A learning network effect needs a measurable spillover

By DX Research Group · · Data and learning flywheels

A donor-recipient experiment tests whether evidence from new participants helps existing owners on untouched cases.

More users can create a learning network effect if their evidence improves the experience of other users. The meaningful claim is a spillover: a recipient benefits from a repair made possible by someone else's contribution. Participation counts and larger archives describe growth. We propose a donor-recipient experiment to measure whether DXAP's expanding evidence base creates that additional value.

DXAP's documented decision path links owner strategy, account context, proposed actions and execution outcomes. That structure offers candidate observations whose mechanisms can be compared. Our continuous-record research also shows why diversity matters: configuration surfaces and rendering choices can shape behavior across a population. The next question is whether new contributors expose mechanisms that a fixed participant group misses.

Separate who supplies evidence from who benefits

In the proposed study, existing participants supply a frozen baseline dataset. A newly consenting group supplies additional cases under a defined contribution scope. Build candidate A using only the baseline and candidate B using baseline plus the new group's accepted evidence. Give both candidates the same curation and development budget, allowing A to spend its budget revisiting baseline gaps. Freeze the changed artifacts before evaluation.

Evaluate on untouched cases from recipient owners who contributed to neither development dataset. Keep a second untouched set from later new owners to see whether any benefit reaches that population too. Group closely related incidents and owners during splitting so a copied scenario cannot appear on both sides. Record which new evidence families actually entered candidate B; an invitation alone adds no training material.

For an illustrative comprehension fixture, candidate A correctly identifies the blocking policy on 72 of 100 recipient cases and candidate B on 80. The eight-case difference would be a descriptive spillover on that fixture. Paired uncertainty and the clustering structure determine whether it supports a reliable benefit. We would additionally examine cases where B makes an owner confidently misunderstand an allowable action, because an average gain can conceal a damaging subgroup change.

Ask whether the marginal contribution survives scale

Repeat the comparison at increasing numbers of donor families, using a prespecified order and fresh assessment sets when earlier results influenced development. Plot recipient improvement against accepted independent families and total review cost. A rising curve followed by a plateau would indicate where more of the same evidence stops helping. A decline could indicate incompatible labels, an overloaded retrieval context or a sampling shift.

This design makes a stronger test than showing that contributors themselves improved after receiving support. Their support exposure could explain their own gains. A recipient who never supplied the diagnostic case gives the shared asset a separate test.

The proposed network effect is therefore conditional on a measurable pathway: new evidence changes a candidate, and that candidate improves held-out recipient behavior at a comparable budget. It would establish the specific assessed behavior. Predictive skill and net economic benefit need their own recipient evaluations before those claims can travel with it.

Sources

Related field notes