Tool-Output Schema Migrations Need Consumer Tests

A tool-output migration succeeds when its consumers preserve the intended meaning. Valid JSON is only the beginning. We would test old and new consumers against old and new payloads, with explicit adapters for any changed units or field semantics.

A field rename changes the sizing input

Suppose an illustrative quote response originally contains price: 100 with dollars documented externally. A new response contains price_cents: 10000 and currency: USD. An old consumer that reads only price now receives a missing value. A permissive fallback to zero can turn the migration into a sizing error even though the new payload is well formed.

Construct four cells: old consumer with old payload, old consumer with new payload, new consumer with old payload, and new consumer with new payload. Document which combinations are supported. Compatibility can come from an adapter, a dual-field transition or a declared coordinated cutover. Each choice has a different failure and retirement path.

JSON Schema's object reference explains required and additional property behavior. Our proposed test adds semantic assertions: the interpreted quote remains 100 dollars, and the derived quantity for a 1,000-dollar budget remains ten units before trading constraints.

Optional does not mean harmless

An added optional field can still change a model's behavior if it appears in the rendered tool result. A new risk annotation may influence the proposal even when deterministic parsing ignores it. Separate structural backward compatibility from behavioral stability. Both can matter in a model-mediated consumer.

We would include missing-field, explicit-null and unknown-extra-field fixtures. Require the consumer to distinguish unavailable data from an actual numeric zero. A schema adapter should record its source and target versions plus any lossy conversion. If units are unknown, it should produce an unresolved interpretation rather than guess.

The operating-layer controls companion grounds the typed-action boundary. The continuous record companion grounds the importance of rendered inputs. The proposed migration matrix would test a candidate change; the historical publications establish no result for it.

A dual-field payload creates another fixture: price: 100 and price_cents: 9900 conflict. Specify precedence or reject the payload. Silently choosing whichever field the consumer encounters first produces version-dependent economics. A deterministic adapter can instead verify equality after conversion and expose the conflict.

After the transition, count traffic by consumer version before removing the old field. Historical replays should retain their original payload and an explicit compatibility adapter if one is used. Rewriting old traces into the new format can obscure which input originally reached the model.

The reviewable migration artifact is the four-cell matrix, semantic assertions and unresolved combinations. This supports a narrow compatibility claim for the tested consumers. It avoids equating a successful schema validation with preserved trading behavior across every past trace.