Replay testing a live trading loop means running the production code path against historical data with the clock and network replaced by deterministic stand-ins. The goal is not a prettier backtest. It is to execute the same run() function that trades live, one bar at a time, and assert that its decisions match an independently computed reference.

This is the harness that caught a +1702% "perfect" backtest. Replaying the production loop over the same period only broke even. The disagreement between the two was the signal: the backtest was lookahead-biased, and the production code was not.

Why Replay the Production Loop

Backtests usually live in their own file. They reimplement signal logic, position sizing, and exits, often subtly differently from the live code. When the backtest looks great, you have learned something about the backtest, not about the system. A rewrite can never find bugs in code it doesn't execute.

Replay testing inverts this. The object under test is the production loop itself. You keep the real run() function, the real state machine, the real order logic. You replace only the boundaries: the network calls, the clock, and persistence. Then you drive it forward bar by bar and check every decision against a reference implementation computed separately.

The payoff is speed and fidelity at the same time. Three months of 4h candles replay in about a minute, because nothing waits on a network or a wall clock.

The Substitution Layer

The production code touches the outside world through a small set of functions. Enumerate them first; everything else is internal. In this system the boundaries were:

  • fetch_klines(symbol, interval, limit) — returns OHLCV history.
  • fetch_price(symbol) — returns the current price.
  • load_state() / save_state(state) — persistence.
  • load_trades() / save_trades(trades) — trade log persistence.
  • now() — the clock.

Monkey-patching these is the whole trick. No dependency injection refactor, no interface layer, no changes to production code. You patch the module attributes the production code resolves at call time, run the replay, and restore them afterward.

import contextlib from unittest import mock @contextlib.contextmanager def patched_market(replay): """Swap every network/clock/persistence boundary for the replay's fakes.""" with mock.patch("trader.market.fetch_klines", replay.fetch_klines), \ mock.patch("trader.market.fetch_price", replay.fetch_price), \ mock.patch("trader.store.load_state", replay.load_state), \ mock.patch("trader.store.save_state", replay.save_state), \ mock.patch("trader.store.load_trades", replay.load_trades), \ mock.patch("trader.store.save_trades", replay.save_trades), \ mock.patch("trader.clock.now", replay.now): yield 

Two details matter. Patch where the name is looked up, not where it is defined. If trader.loop does from trader.market import fetch_klines, you must patch trader.loop.fetch_klines. Second, restore on exit — the context manager handles that, and you should never run a replay without it.

The Replay Object

The replay holds the full candle series in memory and a cursor. Every faked function reads from the cursor, never from the future.

class Replay: def __init__(self, candles, start=90): self.candles = candles # list of dicts: ts, o, h, l, c, v self.cursor = start # index of the current bar self.state = {} self.trades = [] # --- network fakes --- def fetch_klines(self, symbol, interval, limit): # Rolling historical window ending at the current bar, inclusive. lo = max(0, self.cursor - limit + 1) return self.candles[lo:self.cursor + 1] def fetch_price(self, symbol): # Current bar's close. Never the next bar's open. return self.candles[self.cursor]["c"] # --- persistence fakes --- def load_state(self): return dict(self.state) def save_state(self, state): self.state = dict(state) def load_trades(self): return list(self.trades) def save_trades(self, trades): self.trades = list(trades) # --- clock fake --- def now(self): return self.candles[self.cursor]["ts"] 

The single most important line is fetch_price. It returns the current bar's close. A live loop polling mid-bar sees a price that is still forming; on a closed-bar strategy the equivalent is the just-closed bar's close. Returning the next bar's open here is exactly how lookahead bias enters a harness. If your strategy reads the current bar's high or low, stop — those are not known until the bar closes, and the replay must not expose them.

The rolling window in fetch_klines mimics a real exchange call: bounded by limit, anchored at the cursor, containing only bars that have already closed. The production code cannot see bar 500 while the cursor sits at 400.

Driving the Loop

Load 9,009 real BTC 4h candles, start at bar 90 (enough warmup for the EMA windows), and advance one bar at a time.

def replay_run(candles, start=90): replay = Replay(candles, start=start) opens = [] with patched_market(replay): trader = Loop() # production object, unmodified for i in range(start, len(candles)): replay.cursor = i before = len(replay.trades) trader.run() # the live entry point if len(replay.trades) > before: opens.append((i, replay.trades[-1])) return replay, opens 

trader.run() is the production function. It calls fetch_klines, computes indicators, checks entry conditions, calls fetch_price, and persists state and trades through the patched functions. Nothing inside it knows it is being replayed. That is the point.

9,009 candles minus 90 warmup bars is 8,919 invocations of run(). Each one does a bounded amount of work over a window of a few hundred bars. Total runtime lands around a minute on a laptop — no sleeps, no HTTP, no retries.

The Independent Reference

The replay tells you what the production loop did. It does not tell you whether that was correct. For that you need a reference computed by separate code that shares no logic with the loop under test.

Here the reference is an EMA cross: fast EMA crossing above slow EMA is a long signal. Compute both EMAs over the full series once, then derive the exact bars where a cross occurs.

def ema(values, period): k = 2.0 / (period + 1) out = [None] * len(values) acc = None for i, v in enumerate(values): if i < period - 1: continue if acc is None: acc = sum(values[i - period + 1:i + 1]) / period else: acc = v * k + acc * (1 - k) out[i] = acc return out def reference_crosses(candles, fast=12, slow=26): closes = [c["c"] for c in candles] f = ema(closes, fast) s = ema(closes, slow) crosses = [] for i in range(1, len(candles)): if None in (f[i], s[i], f[i - 1], s[i - 1]): continue if f[i - 1] <= s[i - 1] and f[i] > s[i]: crosses.append(("long", i)) return crosses 

This is deliberately simple and written independently. It does not import the strategy module. If both the reference and the production loop used the same helper, a bug in that helper would cancel out and the assertion would pass on broken code.

Asserting Direction and Timing

Every open the production loop produces must correspond to a reference cross at the same bar, in the same direction. Timing matters as much as direction: an open one bar late means the loop is reading a stale window; one bar early means it is reading the future.

def assert_opens_match(opens, crosses): ref = {i: direction for direction, i in crosses} got = {i: t["direction"] for i, t in opens} missing = set(ref) - set(got) extra = set(got) - set(ref) wrong = {i for i in set(ref) & set(got) if ref[i] != got[i]} assert not missing, f"missed opens at bars {sorted(missing)}" assert not extra, f"phantom opens at bars {sorted(extra)}" assert not wrong, f"wrong direction at bars {sorted(wrong)}" 

Three failure modes, three assertions. missing catches signals the loop dropped — usually a warmup or window-length bug. extra catches opens with no reference basis — the classic lookahead signature, where the loop reacts to information the reference never had. wrong catches inverted comparisons or a flipped sign in the cross logic.

Run it end to end:

candles = load_candles("BTCUSDT_4h.json") # 9,009 bars replay, opens = replay_run(candles) crosses = reference_crosses(candles) assert_opens_match(opens, crosses) print(f"{len(opens)} opens matched {len(crosses)} reference crosses") 

What the Harness Exposed

The original backtest reported +1702% and zero losing periods. The replay of the production loop, over the same 9,009 candles, broke even. The assertion step explained why: the backtest produced opens that the reference did not, and those extra opens clustered on bars where the next candle was favorable.

The backtest had been computing the EMA over a slice that included the current bar's close and then entering at that same close — a decision made with the bar's final value before the bar was final. The production loop, fed by fetch_klines bounded at the cursor and fetch_price returning the current close, could not do that. It entered on the next bar. The disagreement between backtest and replay was the tell, and the reference assertion localized it to specific bars.

None of this required changing production code. The loop was correct; the backtest was not. That is the argument for replaying the production loop: the code that runs in production is the only code whose behavior you actually care about, and a rewrite can never find bugs in code it doesn't execute.

Practical Notes

Keep the warmup constant explicit and assert the candle count before starting. A truncated file silently shifts every index. Patch by lookup site, not definition site. Make fetch_price return the current bar's close and nothing later. Write the reference with no imports from the strategy package. And when the assertion fails, print the bar index and both values — the bar number is what lets you open the chart and see the lookahead with your own eyes.

More notes like this ship every week on this site.

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-11-21 08:46:09

Short

Pair Signal Price Take Profit Stop Loss R/R
INX 扫荡做空 $0.0031 $0.003 $0.0032 1:1.2

1 picks selected. Scanner runs every 15 minutes.