Classify Malformed Tool Arguments by the Repair They Need
By DX Research Group · · Trace evaluation
A staged taxonomy distinguishes serialization, schema, domain and authority failures in trading-agent tool calls.
Malformed tool arguments need labels that identify the repair. We would distinguish broken serialization from a schema mismatch, an impossible market value and an unauthorized action. Combining all four into one invalid-call count obscures which part of the harness can change the outcome.
Four mutations of one order
Start with an illustrative valid proposal to buy two units at a limit price of 100. Mutate one property per fixture. Remove a closing brace to create a parsing failure. Change quantity to an array to create a schema failure. Supply a negative quantity to create a domain failure. Finally, use a valid positive quantity above the owner's permitted exposure to create an authority failure.
The first case never becomes an object. The second becomes an object but violates the expected type. The third is structurally accepted by a permissive numeric schema but fails trading semantics. The fourth can satisfy market rules while exceeding user authorization. A successful parser therefore answers only the first question.
JSON Schema's object documentation describes property validation and required fields. We would use that machinery for the interface layer, then preserve separate market and mandate validators. The taxonomy is our proposed evaluation design, rather than a measurement of current DXAP failures.
Keep one primary label and all observed defects
For aggregate reporting, assign the earliest blocking stage as the primary failure. Retain independent flags for later defects discovered during safe offline inspection. A malformed object containing an oversized quantity belongs to parsing failure for runtime attribution; an offline reviewer may additionally flag the intended size violation. That avoids double-counting turns while preserving useful diagnostics.
A fixture with eight calls, two per mutation, has eight first failures and two in each primary class. Counting every defect could produce more than eight flags. The report should name whether its denominator is calls, turns or independent defect observations. Repair success should mean the corrected call reaches the intended admissible action, including any necessary owner review.
The operating-layer controls companion grounds the importance of typed controls. The continuous record companion provides the broader historical decision record in which interface success and economic quality remain separate. A lower parsing-failure rate would establish a narrower improvement than better trading.
For regression, preserve the original bytes, parser error, schema path and first blocking validator. Then run each mutation through the candidate interface. A repair that silently converts an array into a scalar changes the action's meaning and deserves its own review. Prefer an explicit correction request where intent is ambiguous. The audit closes with a repair map: serialization issues go to request construction, structural issues to the contract, domain issues to normalization, and authority issues to policy enforcement. Each label earns its place by pointing to a different engineering response.