Ensemble Agreement Can Hide Shared-Source Dependence

Ensemble agreement is stronger evidence when members have meaningfully different failure paths. If every member reads the same wrong quote or copied news claim, unanimity can express shared-source dependence. We propose auditing source overlap alongside prediction disagreement.

Knowledge distillation discusses ensembles as a predictive mechanism. This note addresses the separate question of what ensemble agreement establishes when evidence sources overlap. Our continuous record remains historical; trace feedback supplies the source-to-decision links required by the proposed audit.

Five votes can contain one observation

Imagine an illustrative ensemble of five models, each recommending buy after reading the same report. The apparent vote is five to zero. At the evidence level, all five recommendations depend on one source claim. If that claim is false, the vote count offers little protection against the shared error.

A simple variance illustration makes the dependence visible. For five equally noisy estimates with common pairwise correlation 0.8, variance of their mean is sigma squared times [1 plus 4 times 0.8]/5, or 0.84 sigma squared. Under independence it would be 0.2 sigma squared. This equicorrelation model is illustrative; actual model dependence needs measurement.

The corresponding effective-count heuristic is 5/[1 plus 4 times 0.8], approximately 1.19. We would use that only when its correlation assumptions fit the measured error process. Shared URLs identify a dependency worth testing; estimating its correlation requires paired outputs.

Remove a source, preserve the task

The proposed test retains one frozen decision case and constructs evidence variants. All members first receive the original source bundle. Then a shared source is removed or replaced with a timestamp-valid alternative conveying the same underlying fact. A third variant introduces a known false claim whose provenance remains visible.

Record which member changes its forecast, which cites the claim, and which detects the contradiction. Separate model diversity from source diversity: switching model families while retaining the same source bundle tests one dimension; changing sources while retaining models tests another. The replacement must avoid giving one condition additional facts.

Aggregate disagreement by source family as well as model identity. Five websites syndicating the same report may belong to one evidence family. Two independent venue observations may deserve separate treatment even when their rendered values agree.

We would report agreement, predictive errors on resolved cases, and source dependency coverage together. An ensemble could improve ordinary accuracy while remaining fragile to a particular shared source. That finding would support the specific source-validation repair tested. The most useful artifact is a dependency graph identifying which supposedly separate votes share the premise that could overturn them.