Quarantining Scenario Memory in Agent Evaluations

A stress scenario needs a memory scope that prevents hypothetical outcomes from entering ordinary account history. We would bind every derived artifact to the scenario branch that created it. The same account identifier can appear in several imagined futures, so account scoping alone provides insufficient separation.

Our state and memory framework separates current facts from retained claims. This note adds a scenario dimension to that separation. Our trace feedback method makes the derivation path inspectable when a later answer unexpectedly recalls a simulated event.

A liquidation that never happened

Imagine a hypothetical research worker evaluating a 20% price decline. In scenario S1, a fictional account loses 200 units from an opening value of 1,000. The worker summarizes, “Account value fell to 800 in the stress case.” A subsequent ordinary turn asks for the actual current account value, still 1,000.

If the summary loses its scenario marker, semantic retrieval can return the vivid loss as recent account history. We would store scenario S1, parent snapshot identity, assumed shock, and evidence class with the summary. All downstream embeddings and derived notes inherit that scope. The ordinary decision query excludes scenario-local claims unless it explicitly asks for scenario analysis.

A second scenario S2 assumes a 5% decline and ends at 950 under the simplified arithmetic. S1 and S2 should remain separate. Combining them into “the account recently ranged from 800 to 950” creates an invented historical range. Scenario arithmetic describes alternative assumptions, rather than an observed time series.

Test leakage in both directions

The fixture first runs an ordinary query after both scenarios. Expected factual value is 1,000, with the simulated values excluded from current-state fields. It then asks specifically about S1 and expects 800 with the shock assumption attached. A third query compares S1 and S2 and should retain both branches instead of selecting one as the latest fact.

We would also test the reverse path: a scenario worker should use the parent snapshot that seeded its scenario. If it silently fetches a newer live balance, its result no longer corresponds to the declared stress input. Branch isolation therefore protects the real decision and the scenario's own reproducibility.

The W3C PROV model supplies derivation concepts useful for linking the scenario result to its inputs. The application must still enforce scope filters and verify that summaries carry those identifiers. A provenance graph that is never consulted at retrieval time leaves the contamination path open.

Measurements should separate retrieval leakage, factual assertion leakage, and action sensitivity. A scenario item can be retrieved and correctly labeled without corrupting the final account statement. Conversely, a model can invent a scenario outcome without retrieving one. The proposed fixture names those mechanisms separately. Its useful final artifact is a comparison showing the same account with three properly labeled states: one observed and two hypothetical. That is a readable way to keep frontier research tools useful without allowing their imagined futures to rewrite the present.