A referral is an access event; a research contribution needs evidence

By DX Research Group · · Incentives and participation

Separate community growth from the evidence that helps improve a trading-agent harness.

A community can bring people into DXAP and help improve its harness through different activities. Someone may share an access code. Someone else may identify a confusing policy rejection. Treating both as one participation score hides what each activity accomplished. We propose measuring access and research contribution separately, so the community can grow without making promotional output the definition of useful feedback.

The dated Chapter 0 documentation distinguishes shared codes from redeemed access: the second-week refill review requires at least two new-user redemptions from the current allocation. Its one-time reviewed refill is a documented access mechanism. That condition says something about invitation use; it supplies no finding about the quality of the new users' reports.

Follow two records

Consider an illustrative contributor who shares three codes. Two new users redeem them, and one sends a report that a strategy edit appears to conflict with a displayed limit. The access record contains two redemptions. The research record contains one submitted report, whose evidential status remains pending. A screenshot alone might establish the display mismatch. Reproducing the same state with the recorded strategy version could establish the trigger. These are different achievements with different denominators.

Our proposed research intake would ask for the observed behavior, intended behavior and minimum context needed to reproduce the discrepancy. A reviewer then records whether the report is reproducible, already known, incomplete or a misunderstanding that reveals a documentation problem. A contributor can help by clarifying an existing report rather than producing another public post. Permission to inspect a supplied artifact stays separate from permission to publish it.

The controls paper companion describes a historical cycle of diagnosing failures, revising the harness and checking replayed scenarios. That gives the research record a practical destination: a report should become a fixture, a documentation correction or a recorded unresolved question. Counting submissions without recording that destination would reward intake rather than learning.

A comparison worth running

For a proposed participation study, offer two intake descriptions to otherwise comparable volunteers. One emphasizes sharing; the other asks for a specific observed mismatch and accepts quiet, private submission. Keep access conditions identical. Measure complete reports per invited participant, reproduction rate, reviewer time and the number of distinct failure families covered. Retain people who submit nothing in the denominator and record who declines research participation.

Invitation chains should be grouped during analysis because friends may encounter the same setup and describe the same issue. Ten copied reports would be one failure family with ten affected users, rather than ten independent discoveries. We would also examine whether the intake reaches people who use different devices or remain cautious about automation.

The useful outcome is a clearer allocation decision: which invitation and intake design produces inspectable evidence at a manageable review cost? Referral activity can support access while research participation supports diagnosis. Evidence of better forecasts or economic outcomes would require its own subsequent evaluation.

Sources

Related field notes