Choose a Completion Stage Before Setting an Agent Service Objective
By DX Research Group · · Trace evaluation
Define separate objectives for usable proposals, submitted actions and reconciled outcomes.
An agent service objective needs a named completion stage. A response token, a validated proposal and a reconciled fill represent different outcomes. We would choose the stage that matches the task and expose the intermediate stages so a passing early metric cannot conceal a broken downstream path.
What has the owner received?
For a scheduled research review, completion may mean a usable decision record with cited inputs. For an authorized execution task, completion may require submission plus a terminal venue outcome. A passive limit order can legitimately remain open, so the objective should specify acknowledgement and continuing state visibility rather than demand an immediate fill.
An illustrative batch has one hundred assigned reviews. Ninety-five generate a proposal, ninety pass the required validation, and eighty-five produce a reconciled terminal result within the declared observation window. The respective fractions of assigned reviews are 95%, 90% and 85%. A statement about proposal availability answers a different question from a statement about reconciled completion.
The Google SRE chapter supplies the service measurement background. Our proposed contract adds the economic state transition. Each objective should include task eligibility, start clock, terminal condition and treatment of cancellation or expiry.
Use a fixture with a resting order
Create three cases: a review that recommends holding, an immediately filled order, and an acknowledged order that remains open. Record their expected terminal meanings before the run. The hold case completes when its review is persisted. The fill case continues through state reconciliation. The resting order completes its submission objective but enters a separate monitoring obligation.
The operating-layer controls companion reports historical settlement evidence for a stated eligible submission population. The continuous record companion extends the historical trace analysis. Readers should preserve those populations rather than reinterpret a conditional settlement measure as the availability of every scheduled turn.
For a release candidate, we would show stage conversion counts alongside elapsed time. A model can increase proposal availability while a connector regression reduces reconciled outcomes. Conversely, improved reconciliation may repair observability without changing prediction quality. The objective makes the intervention legible.
Declare numerical targets as an operational choice, with rationale from the mandate and infrastructure budget. An illustrative target of 99% has no authority merely because it sounds standard. It requires a population definition and a cost for misses. Urgent exposure checks may justify a tighter deadline than optional news summaries.
This is a proposed measurement contract, rather than a current DXAP service promise. Its final review surface is a compact stage ledger: assigned, usable, admissible, submitted and reconciled, with each task's terminal condition named. That ledger tells an operator where the work stopped and tells a researcher which reliability claim the data can support.