Fix use-after-free in async host ledger reads on shutdown#8003
Conversation
There was a problem hiding this comment.
Pull request overview
This PR addresses a shutdown-only use-after-free risk in the host’s async ledger read path (uv_work_t threadpool jobs) by ensuring worker callbacks never dereference a Ledger* after the owning asynchost::Ledger has been destroyed. It does this by introducing a shared control block that outlives the Ledger and by having ~Ledger() wait for in-flight worker callbacks to finish before allowing member destruction.
Changes:
- Added a
shared_ptr-ownedAsyncReadState(mutex + condition variable +active_workers+ledger_alive) shared betweenLedgerand eachAsyncLedgerGet. - Updated
on_ledger_get_asyncto (a) skip reads once shutdown begins and (b) register/deregister active worker callbacks so~Ledger()can safely drain them. - Added a CHANGELOG entry documenting the shutdown use-after-free fix.
Custom instructions used:
.github/copilot-instructions.md.github/instructions/reviewing.instructions.md
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/host/ledger.h |
Introduces AsyncReadState and shutdown coordination so async libuv workers don’t access a destroyed Ledger during/after shutdown. |
CHANGELOG.md |
Documents the shutdown UAF fix with a PR reference. |
|
No blocking issues found. I reviewed PR #8003 against main, focusing on Correctness:
Non-blocking suggestions:
CI note: At the time of review, the main VM/ACI checks for the current head were still queued or in progress; ASAN/TSAN and long jobs were skipped by workflow configuration. Custom instructions used:
|
Make the active worker accounting exception-safe and clarify that AsyncReadState protects worker access to Ledger while completion callbacks are Ledger-independent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds a ledger_test case that queues an in-flight async committed-range read, holds it active on a libuv threadpool thread, then destroys the Ledger on another thread and asserts ~Ledger blocks until the worker completes - exercising the use-after-free fix from #8003. Validated under ASAN/UBSan. - ledger.h: add guarded (TEST_MODE_LEDGER_ASYNC_HOOK) worker hook and test_queue_async_read trigger; tidy async_read_state comment - ledger_test: new test case; link target against libuv - CHANGELOG: move the #8003 entry to the current 7.0.7 section
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Replace the manual hook/thread teardown with a single RAII guard that, on every exit path (including a failed REQUIRE, which doctest throws), releases the in-flight worker so ~Ledger can finish, joins the destroyer thread (a joinable std::thread must never be destroyed), drains the completion callback, and clears the global worker hook so it can never retain dangling references to the test frame.
test_queue_async_read now takes an optional result callback so a test can observe the read outcome. Adds a test that queues a read after ~Ledger has begun (ledger_alive == false) and asserts it is skipped rather than served, and extracts the shutdown cleanup into a shared helper reused by both async shutdown tests.
Address review feedback on PR #8003: avoid putting test-only code behind an #ifdef in the production header. Replace TEST_MODE_LEDGER_ASYNC_HOOK (a global mutable hook function and a public test_queue_async_read method) with: - An always-compiled, no-op-by-default test_worker_active_hook field on the private AsyncReadState struct, called unconditionally in on_ledger_get_async. - A forward-declared LedgerAsyncTestAccess struct, befriended by Ledger, defined only in src/host/test/ledger.cpp, which uses that friendship to reach the private async_read_state member and queue async reads via the already-public AsyncLedgerGet, on_ledger_get_async and on_ledger_get_async_complete. No behaviour change in production builds; the two async shutdown tests are updated to use the new accessor.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: cjen1-msft <190360281+cjen1-msft@users.noreply.github.com>
Replace the LedgerAsyncTestAccess friend struct and test_worker_active_hook with a standalone WorkerShutdownGate class in src/ds/worker_shutdown_gate.h. The shutdown gate is a reusable concurrency primitive that: - Allows workers to register/unregister - Blocks shutdown until all active workers complete - Rejects new registrations after shutdown begins This removes all test-only surface from the production Ledger class: - No friend struct - No test hook field in production code - No manual job construction in tests The ledger test now uses the real ringbuffer dispatch path to trigger async reads, and exercises concurrent destruction as a stress test (best run under TSAN/ASAN). The shutdown protocol itself is unit-tested independently via deterministic thread-based tests in worker_shutdown_gate_test. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Fix use-after-free in async host ledger reads on shutdown
Problem
When the host
Ledgerobject is destroyed during shutdown, any in-flight asynchronous ledger read workers (dispatched viauv_queue_work) could still be running or about to start on the libuv thread pool. These workers hold a rawLedger*pointer and access ledger state — leading to a potential use-after-free.Solution
Introduce a
WorkerShutdownGateclass (src/ds/worker_shutdown_gate.h) — a reusable cooperative shutdown primitive that:try_register()before accessing the protected resource. Returnsfalseif shutdown has begun (worker must skip).unregister()when done. Notifies the shutdown waiter.shutdown_and_wait()in the destructor — sets the "shutting down" flag and blocks until all registered workers finish.The
Ledgerdestructor now callsshutdown_gate->shutdown_and_wait(), ensuring:Design
The shutdown protocol is extracted as a standalone, independently testable class rather than being baked into Ledger's internals. This means:
friendstructs or test hooks in production codetry_register/unregister/shutdown_and_wait) is its real API, not test scaffoldingChanges
src/ds/worker_shutdown_gate.h— New standalone concurrency primitivesrc/ds/test/worker_shutdown_gate_test.cpp— Deterministic unit tests for the gatesrc/host/ledger.h— Replace hand-rolledAsyncReadStatewithWorkerShutdownGate; removefriend struct LedgerAsyncTestAccessandtest_worker_active_hooksrc/host/test/ledger.cpp— Replace friend-based tests with a stress test using the real ringbuffer message pathCMakeLists.txt— Addworker_shutdown_gate_testtarget; version bumpCHANGELOG.md— Document the fixCloses #8003