Early feature access should come with a test contract

By DX Research Group · · Incentives and participation

A proposed staged testing design separates research consent, feature exposure and trading authority.

Early access becomes useful research when a participant knows what they are testing and can decline the next stage. We propose a short test contract for experimental trading-agent features: the changed behavior, permitted observations, data use and stopping condition should be explicit before exposure. Access to an experiment and authorization to trade are separate decisions.

The DXAP documentation describes owners defining strategy and limits, then following agent decisions and refining direction through chat. That workflow supplies several places to test a feature without assuming that every test requires live execution. A revised explanation can be evaluated through saved examples. A changed order-path control needs a different protocol with its execution consequences reviewed.

A fictional explanation feature

Consider a proposed interface that expands a policy rejection into a clearer explanation. First, volunteers review synthetic cases with known expected answers. Then consenting participants can inspect explanations on recorded turns. A later stage might expose the interface during ordinary supervised use, under a separate agreement. This sequence is a study proposal, with no claim that DXAP currently offers the feature or a commitment to release it.

For each stage, state what success means. In synthetic review, success could mean correctly identifying which owner limit blocked an action and what information remains unavailable. In recorded-turn review, the question becomes whether the explanation faithfully describes the actual rejection. During supervised use, measure whether owners understand the result without being pushed to relax a limit.

An illustrative case says that a proposed quantity exceeds the owner's size cap. The expected answer identifies that cap. “Increase your allowance to continue” would be a problematic explanation because it turns a diagnosis into an unsolicited recommendation. A user who chooses to leave the cap unchanged has completed the comprehension task successfully.

The historical controls paper companion records linked stages from mandate through policy checks and settlement. Our proposed test contract uses that distinction: changing displayed language should preserve the underlying action and rejection. A reviewer can compare the canonical action before and after rendering, then judge explanation fidelity independently.

Progression needs fresh evidence

Stage advancement would depend on the registered measures and reviewed failures. Participants should receive the next stage's description and choose whether to continue. Declining live exposure would preserve their earlier contribution and any consented attribution. Experimental feedback permission should specify retention and public use of submitted material, rather than bury those choices in an access offer.

For a first comparison, assign volunteers to old or proposed explanations on the same saved cases and score comprehension blind to interface assignment. Keep model output fixed. Record misunderstood restrictions, time to answer and confidence in an incorrect answer. A faster interface that makes owners confidently wrong would fail the intended purpose.

The next measurable advance would be a release decision supported by those observations: revise the explanation, widen the saved-case study or proceed to a separately consented stage. An early-access label alone supplies none of that evidence.

Sources

Related field notes