Skip to content

feat(server): add server side cache - #301

Open
namrataghadi-galileo wants to merge 4 commits into
mainfrom
feature/SAO-19672-ac-server-side-caching
Open

namrataghadi-galileo wants to merge 4 commits into
mainfrom
feature/SAO-19672-ac-server-side-caching

Conversation

@namrataghadi-galileo

@namrataghadi-galileo namrataghadi-galileo commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add a provider-neutral ControlCache interface with NoOpControlCache as the default and an optional Redis implementation for effective-control snapshots.
  • Keep PostgreSQL as the source of truth. Redis stores serialized snapshots keyed by schema version, namespace, agent, target type, and target ID.
  • Support standalone and cluster Redis topologies with generic AGENT_CONTROL_* settings, TTL fallback, and fail-open evaluation when Redis is unavailable.
  • Invalidate affected agent and target scopes after committed control, policy, binding, and agent-data mutations. Redis Cluster Lua keys share a hash tag; invalidation does not scan with KEYS.
  • Add cache hit/miss, Redis failure, invalidation, snapshot-load duration, and snapshot-size metrics.

Compatibility

  • Redis remains an optional server extra (redis>=5.2.1); the default backend is PostgreSQL and OSS deployments do not require Redis.
  • No Orbit, Galileo-private, or provider-specific packages are used by generic Agent Control.

Testing

  • make server-test — 1019 passed.
  • make typecheck — passed.
  • Ruff checks on the changed server source and tests — passed.
  • Added regression coverage for cache behavior, invalidation scopes, fail-open paths, and Agent data updates.

@codecov

codecov Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.28682% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
server/src/agent_control_server/control_cache.py 97.19% 6 Missing ⚠️
...r/src/agent_control_server/endpoints/evaluation.py 90.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@namrataghadi-galileo

Copy link
Copy Markdown
Contributor Author

First request took 4 seconds
Subsequent requests are less than 300 ms

Screenshot 2026-11-02 at 5 19 42 PM Screenshot 2026-11-02 at 5 20 28 PM Screenshot 2026-11-02 at 5 19 35 PM Screenshot 2026-11-02 at 5 19 26 PM

@BipinShetty

Copy link
Copy Markdown
Collaborator

@namrataghadi-galileo nice work on this. The core design looks right to me:

  • The key covers namespace, agent and target.
  • The generation check stops an old DB read from refilling the cache after an invalidation.
  • Every mutating route invalidates after commit.
  • A Redis failure falls back to Postgres.

A few things I would like addressed before we turn it on broadly:

  1. No fast-fail when Redis is unreachable. A miss makes two Redis calls (get, get_generation), and each one can wait the full 0.5 s socket timeout. If Redis black-holes packets, every evaluation pays about +1 s until it recovers. Can we add a short circuit breaker that skips Redis for a few seconds after a failure?
  2. Freshness after a failed invalidation depends only on the TTL. If invalidate_agent fails, a disabled control can keep firing, or a new deny control can stay inactive, until the snapshot expires. Please keep the TTL short (the 60 s default or lower) and document this limit for deployments that raise it.
  3. The per-agent index is not pruned. Each fill adds the snapshot key and extends the index lifetime, but expired snapshot names stay in it. Invalidation then reads every member inside Lua. With many target ids, the set grows without limit and blocks Redis during invalidation. Can we prune it, or put the generation in the key instead of using the index?

Follow-ups, not blocking:

  • Each hit still makes one Redis round trip, re-parses every control with Pydantic and builds a new engine. A small in-process layer that checks the generation would remove that cost.
  • Concurrent misses are not combined, so after an invalidation all in-flight requests go to Postgres.
  • invalidate_namespace_agents swallows a failed SELECT without a rollback. The following db.refresh can then return 500 after the write committed. Running the discovery in its own session would avoid this.
  • Minor: the invalidate script writes stale keys it does not declare in KEYS, the generation keys never expire, and the cached id and name are not validated on read.

self, namespace: str, agent: str, target_type: str | None, target_id: str | None
) -> None:
del namespace, agent, target_type, target_id
_CACHE_MISS.inc()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no cache to miss then why increment this counter?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants