[jamesgao-jpg]

Benchmark analysis needs separate compaction timings and stage boundaries so an external collector can associate CPU, I/O and profiles with the work that produced them. This preserves the native benchmark result and adds opt-in, structured observations.

• Record actual compaction request/wait duration and status, dataset metadata and resolved search output fields. Readiness, flush, retry sleeps and load refresh are excluded from compaction time.
• Emit bounded, line-delimited stage events for insert, each compaction, serial search and each search concurrency. Start/end/failure events carry a shared span ID, UTC and monotonic timestamps and run/case/attempt identity. Events are disabled unless the orchestrator opts in, and compaction events travel through the existing worker subprocess.
• When stage events are enabled, sample persistent segment counts before and after each compaction and record their timestamps and availability independently. Metadata reads are bounded and remain outside the compaction duration.
• Preserve existing compaction behavior and retain explicit unavailable/skipped/failed states; historical JSON and other clients remain supported.

The consumer and telemetry result schema are in the orchestrator repository. The dashboard consuming these fields is in test-platform-web PR #51.

Validation: 26 targeted metadata, stage-event, worker-process, mocked segment observation, concurrency-boundary and Milvus optimization tests; Ruff on changed Python files. The segment API is mocked and no new VDBBench workload or live compaction was run. The orchestrator's independent Kubernetes collector was exercised on an existing idle UAT3 pod; this does not establish search-phase native stack quality or benchmark overhead.

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jamesgao-jpg
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment