Reconstruct the Instrument Universe an Agent Could Actually Trade
By DX Research Group · · Market data
Build an eligibility ledger that preserves listings, restrictions, and removals before comparing agent selection.
Save instrument eligibility as a changing dataset. A historical trading-agent comparison needs the markets that were available at each decision, including temporary restrictions and subsequent removals. Today's market catalogue cannot supply that history. We would bind the eligibility version to the candidate list before asking whether an agent selected well.
Coinbase's status channel exposes product status and trading-mode fields. That is a useful collection surface; preserving its changes is the research team's responsibility.
Three markets, two different denominators
This synthetic fixture assumes complete local capture of status changes, identical account permissions, and one decision at noon. The labels are fictional.
| Market | Status visible at noon | Entry eligibility | Later catalogue |
|---|---|---|---|
| A/USD | Online | Eligible | Present |
| B/USD | Cancel only | Existing orders only | Present |
| C/USD | Online | Eligible | Removed |
The entry universe has two markets: A and C. The later catalogue also has two: A and B. Equal counts conceal a completely different opportunity set. If the agent chooses A, its selection rate is one of two eligible markets. Replaying the turn against A and B substitutes a restricted market for an available one and changes the comparison.
Maintain separate predicates for opening, closing, and cancelling. A single active flag cannot express an account that must reduce an existing position while entry is restricted. The fixture deliberately distinguishes catalogue membership from the action the venue permits.
Preserve a status transition, not just a daily list
An eligibility row should reference the market, observed status payload, applicable account class, and the rule used to translate status into allowed actions. Close the previous interval when a new observation changes that interpretation. Keep unknown status explicit when collection starts late or a feed disconnects.
A listing announcement can be recorded alongside the venue observation. It serves a different purpose: an announcement supplies expected availability, while an observed tradable status supplies operational evidence. A collector should retain both when their disagreement could affect the candidate list.
For replay, resolve the ledger before ranking candidates. Store the resulting eligible IDs and exclusions with reasons. A model cannot choose a market absent from its rendered list, so attach a second record showing which eligible markets the harness actually displayed. Our candidate-list note explains that distinction.
A small acceptance case worth keeping
Replay the fixture at noon, after B resumes entry, and after C is removed. Require the eligible set to change according to the recorded transitions. Add an unknown-status case whose expected outcome depends on the collector's documented policy. That policy is an implementation choice, rather than a historical fact supplied by the model.
Report coverage by decision and by market. Ten complete markets plus one missing interval can matter more than an impressive row count. The continuous record motivates preserving candidate availability, while the proposed ledger here establishes reconstruction quality only. A subsequent paired evaluation would still be needed to measure any change in agent behavior or economics.