As-of Joins for Trading Agents: Enforce Information Availability

An autonomous trading agent often receives market features assembled from several streams. We make each matched observation inspectable so a replay can distinguish model behavior from a feature that arrived too late.

An as-of join attaches a nearby observation to a decision row. The direction and timestamp semantics determine whether that observation was actually available. For a historical reconstruction, the closest observation is not necessarily the permissible observation.

Work through three quotes

Assume an illustrative decision at 10:00:00.500 and three quotes for the same instrument. Quote A was received at .100 with a midpoint of 100. Quote B arrived at .450 with a midpoint of 101. Quote C arrived at .510 with a midpoint of 102. A backward join on receipt time chooses B. A nearest join chooses C because it is only ten milliseconds away, but C arrived after the decision.

This difference can affect every downstream feature even though the join runs successfully. The pandas reference documents backward, forward, and nearest matching, the ascending-key requirement, grouping keys, tolerances, and exact-match control. The library supplies the mechanics; the researcher must supply the correct information boundary.

Define the join explicitly

Use an instrument identifier as the grouping key and local availability time as the temporal key when reconstructing the consumer. Sort by that temporal key as required by the operation. Make direction explicit rather than relying on a default. Set a tolerance based on the intended use and report its effect on coverage.

For illustration, a 200-millisecond tolerance accepts B, which is 50 milliseconds old. A 20-millisecond tolerance leaves the feature missing. Neither threshold is universally correct. The second setting trades coverage for a stricter freshness constraint. Show both the accepted count and missing count when comparing them.

Exact matches need a policy too. If a decision and receipt both have millisecond timestamps of .500, coarse precision may conceal their actual order. Allowing equality is an assumption unless another field establishes which happened first. Record that assumption instead of treating equal timestamps as proof of availability.

Inspect the matched record

Retain the matched quote identifier and matched time alongside the feature. Compute decision_time - matched_received_at for every successful match. Negative ages should be impossible under the stated backward contract. Summarize unmatched rows by instrument and collection interval to keep a single problematic feed visible.

Add fixtures for a future quote, a quote outside tolerance, another instrument at the same time, and duplicate right-hand timestamps. Resolve ties before the join using a documented sequence or stable selection rule. Otherwise the apparent answer may depend on input ordering.

Remaining uncertainty

A correct backward join cannot rescue a misleading availability column. A historical provider's event timestamp may precede delivery substantially. Keep the distinction between event chronology and local knowledge visible in the dataset schema. A reproducible join is valuable because another researcher can inspect its assumptions and reproduce its missingness.

Place this check in the agent loop

Our state and memory framework explains how this input contract fits a persistent trading agent. Use the harness-transfer test design to distinguish a data-adapter change from a change in model behavior.