Skip to content

fix(axle): don't fail the run for a user-disabled component#4254

Closed
mgazza wants to merge 4 commits into
mainfrom
fix/axle-health-exempt-disabled-components
Closed

fix(axle): don't fail the run for a user-disabled component#4254
mgazza wants to merge 4 commits into
mainfrom
fix/axle-health-exempt-disabled-components

Conversation

@mgazza

@mgazza mgazza commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Problem

When a user turns Axle automation off in the UI but leaves a stored API key, the Axle component stays loaded and keeps polling (component enablement is required_or: [api_key, managed_mode]). If that key has been rotated/expired, every fetch returns HTTP 401, so the component reads active-but-not-alive.

record_final_run_status() treats any active-but-not-alive component as a hard run failure — even though the user opted out and the plan itself computed fine:

AxleAPI: Fetching latest VPP event data (BYOK)
Warn: AxleAPI: Request failed with status 401
Error: Complete run status Charging with component errors: Axle Energy
Warn: record_status Error: Complete run status ... with component errors: Axle Energy

A run stamped Error (had_errors=True) isn't committed cleanly, so battery control thrashes — we've seen instances export at the outgoing rate then re-import at the peak import rate on alternating runs, costing users money — and components_healthy flaps off on every fetch run.

This is the residual case behind a recent fleet incident: an Axle key-rotation left a batch of opted-out users still erroring every run, because turning Axle off stops the planner acting on sessions (automatic=False) but does not stop the component polling and 401-ing.

Fix

Add a health_exempt concept so a component the user has disabled can be degraded without failing the whole run:

  • component_base.pyhealth_exempt() defaults to False.
  • axle.py — override returns not self.automatic (exempt only while automation is user-disabled).
  • components.py — manager delegator health_exempt(name).
  • predbat.py record_final_run_status() — an active-but-not-alive component that is health_exempt is reported as "degraded"; it does not increment error_count, join failed_components, or flip all_healthy. Non-exempt components are unchanged.

Net effect: with Axle automation off, a broken/rotated key surfaces as degraded (still visible, failures_total and warn logs intact) but never fails the run or marks the instance unhealthy — the user opted out. With automation on, an unhealthy Axle still fails the run exactly as before.

Tests

  • test_component_health_status.py: two new cases — an exempt-unhealthy component records a success status; a mix of exempt + non-exempt records an error naming only the non-exempt component. (FakeComponents gains a health_exempt stand-in.)
  • test_axle.py: new health_exempt subtest — True when automatic=False, False when automatic=True.

All component_health_status and axle (31/31) tests pass; black --check clean.

🤖 Generated with Claude Code

mgazza and others added 4 commits July 14, 2026 21:18
When a user turns Axle automation off but leaves a stored API key, the
component stays loaded and keeps polling. A rotated or expired key then
returns HTTP 401 on every fetch, so the component reads active-but-not-alive
and record_final_run_status() treated that as a hard run failure — even
though the user opted out and the plan itself computed fine. A run flagged
"Error" isn't committed cleanly, so battery control thrashes
(export <-> peak-import), costing the user money.

Add a health_exempt concept (the honest fix flagged in the
predbat-saas-images#108 comment): component_base defaults to False, AxleAPI
returns `not self.automatic`, the Components manager exposes a delegator, and
record_final_run_status() surfaces an exempt-but-unhealthy component as
"degraded" without failing the run or flipping components_healthy off.
Non-exempt components are unchanged and still fail the run as before.

Tests: extend test_component_health_status.py (exempt -> success; mixed
exempt + non-exempt -> only the non-exempt component is named) and add a
health_exempt subtest to test_axle.py.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pre-review caught that axle_automatic controls whether Predbat uses the
default event entity name (binary_sensor.predbat_axle_event), not whether
Axle is enabled. A self-hosted user who sets axle_automatic:false to name
axle_session manually is still actively using Axle, so exempting on
`not self.automatic` would mask a genuinely broken/expired key for that
documented, supported configuration.

Exempt only when no axle_session entity is configured — the same
participation check fetch_axle_sessions() uses — which is exactly the
disabled state (api_key present, no session, e.g. the SaaS "Axle off"
toggle). A configured axle_session, manual or the automatic default,
stays non-exempt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ive()

record_final_run_status() already stopped a user-disabled-but-loaded
component (e.g. Axle turned off with a rotated/expired key) from failing
the run. But is_running() -> is_all_alive() did not honour health_exempt,
so the same component still dragged the instance to "unhealthy" on the
/health page (web.py), in the MCP is_running field (web_mcp.py) and on the
dashboard -- contradicting the health_exempt contract in component_base.

Make is_all_alive() treat a health-exempt component as alive. is_alive()
is deliberately left unchanged so record_final_run_status() can still tell
exempt components apart and mark them "degraded".

Adds a unit test exercising the real Components.is_all_alive(): a dead
health-exempt component keeps the instance alive; a dead non-exempt one
still reports unhealthy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to the is_all_alive() health_exempt fix. The built-in /components
page (html_components) classified each component purely on is_alive(), so a
user-disabled-but-loaded component (e.g. Axle off with a rotated key) still
rendered as a red "Error" card and inflated the "N Error" header count --
inconsistent with /health, the MCP and the SaaS admin UI, which now report
it healthy/degraded.

- Route active + not-alive + health_exempt components into a new "Degraded"
  bucket (amber) instead of error_components.
- Add amber .component-card.degraded / .status-degraded CSS (light + dark).

Test hygiene in test_component_health_status:
- _HealthComponent.last_updated_time() now returns now()-1min, not now():
  is_alive() treats a zero timedelta as stale (it is falsy), so returning
  now() could flip an alive component to not-alive on a fast host.
- Save and restore my_predbat.record_status; it is a single shared instance
  reused across the suite, so the no-op lambda must not leak into later tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@springfall2008

Copy link
Copy Markdown
Owner

I think this is incorrect, the component is enabled with the key and not with automatic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants