Measure participation quality beyond trading volume

Volume tells us how much trading occurred. A participation program also needs to tell us what owners understood, what they contributed and whether their evidence helped improve the product. We would keep those questions visible as separate measures rather than collapse them into a single activity rank.

The DXAP documentation describes owners choosing strategies, models, schedules and trading policies. Those choices create several forms of useful participation beyond placing orders: clarifying an ambiguous mandate, reviewing a decision, diagnosing an approval problem or reporting a reproducible failure. An owner whose agent appropriately waits can still contribute valuable evidence.

Two equally large cohorts

Consider two illustrative cohorts of twenty invited owners, each generating $100,000 of eligible volume over the same period. Cohort A contributes sixteen feedback submissions, two of which reproduce distinct issues. Cohort B contributes eight submissions, five of which reproduce distinct issues. Volume is identical. The reproduced-issue yield per invited owner is 2/20, or 0.10, for A and 5/20, or 0.25, for B.

The submitted-report acceptance rates are 2/16, or 12.5%, and 5/8, or 62.5%. Both denominators matter. Acceptance alone rewards a very selective group, while the invitation denominator captures how much useful evidence the program obtained from the population it reached. Neither result establishes that B's agents forecast better or earned more.

Suppose reviewers spend four hours on A and five hours on B. Distinct reproduced issues per review hour become 0.5 and 1.0. That comparison makes reviewer capacity visible without requiring every contribution to become a confirmed defect. Reports that establish correct behavior but reveal unclear explanations should have their own outcome class.

Preserve the owner in the scorecard

We propose four parallel measures for a dated cohort report: useful evidence per invited owner, distinct failure coverage, owner understanding of controls, and continued participation under the owner's chosen limits. Trading volume can sit alongside them. Each measure needs a defined population, observation window and missing-data count.

For understanding, use a short scenario rather than self-reported confidence. Ask whether closing a visualizer pauses the underlying agent, and whether a recorded no-trade decision requires intervention. The Chapter 0 guide makes the visualizer's relationship to trading explicit. A participant who correctly understands that boundary is better equipped to supervise the product, regardless of volume.

For continued participation, record voluntary returns to review decisions and distinguish those from continued live trading. A participant who pauses after identifying a concern may be exercising useful control. Treating every pause as failure would reward persistence even when the evidence calls for investigation.

Finally, connect accepted reports to a versioned repair and withheld evaluation cases. Our controls paper companion describes historical trace-based revision work, which motivates that connection. The proposed scorecard extends the participation measurement; it provides no current causal return estimate. Its practical value is to show whether a program grows the evidence and understanding an agent platform needs to improve responsibly.