Signal Backtests
How Orrery's price-based signals performed historically, replayed on the top 15 markets by volume. Transparent methodology, no cherry-picking.
History
1M CLOB
Universe
Top 15
Cooldown
1h
Cost model
-2pp
Orrery replays the same live rules minute by minute, then measures 1h, 6h, and 24h follow-through. Flow and resolution-risk need event-time history, so they are excluded for now.
Momentum
359 firings across 11 marketsPrice + 1h direction + 24h direction all aligned, with ≥3pp 24h move. Expected to continue in the same direction.
Expected · 24h, after spread
-5.85pp
raw avg − 2pp typical spread
Median · 24h
-0.65pp
robust to outliers
Win rate · 24h
41%
339 measured
Raw avg · 24h
-3.85pp
before spread / fees
Average move by horizon
- 1h-0.55pp
- 6h-2.66pp
- 24h-3.85pp
Centred at zero. Bar length is proportional to the largest of the three windows, so the decay shape is visible without re-scaling.
By category (4 categories fired)
| Category | Firings | Measured | Win 24h | Avg 24h |
|---|---|---|---|---|
| Geopolitics | 263 | 255 | 37% | -4.84pp |
| Sports | 47 | 35 | 34% | -1.96pp |
| Science | 39 | 39 | 64% | -2.54pp |
| Politics | 10 | 10 | 90% | +8.20pp |
Win rate hidden until ≥10 measured firings — thin samples can flip with one outlier.
Per-market breakdown (11 markets fired)
| Market | Firings | Win 24h | Avg 1h | Avg 24h |
|---|---|---|---|---|
| Strait of Hormuz traffic returns to normal by end of May? | 97 | 58% | -0.28pp | +1.78pp |
| Will Trump say "Nuclear" during events with Xi Jinping? | 61 | 33% | -3.73pp | -12.52pp |
| Will Trump say "Iran" during events with Xi Jinping? | 53 | 23% | -1.25pp | -8.97pp |
| Israel x Syria security agreement by June 30? | 52 | 14% | -1.27pp | -3.99pp |
| Trump declassifies new UFO files by May 15? | 39 | 64% | +2.16pp | -2.54pp |
| Cavaliers vs. Pistons | 14 | 17% | +1.23pp | -2.33pp |
| San Diego Padres vs. Seattle Mariners | 14 | 33% | +2.64pp | -4.75pp |
| San Francisco Giants vs. Athletics | 13 | 55% | +2.77pp | +0.55pp |
| Will Trump say "Dumbocrat" during Bret Baier interview? | 10 | 90% | -1.71pp | +8.20pp |
| Spread: Pistons (-4.5) | 3 | — | +11.22pp | +0.00pp |
| Cavaliers vs. Pistons: O/U 205.5 | 3 | — | +0.00pp | +0.00pp |
Divergence
200 firings across 11 markets1h move running against the 24h trend. Expected to continue short-term in the 1h direction before 24h reasserts.
Expected · 24h, after spread
+2.55pp
raw avg − 2pp typical spread
Median · 24h
+0.75pp
robust to outliers
Win rate · 24h
53%
194 measured
Raw avg · 24h
+4.55pp
before spread / fees
Average move by horizon
- 1h+0.04pp
- 6h+2.63pp
- 24h+4.55pp
Centred at zero. Bar length is proportional to the largest of the three windows, so the decay shape is visible without re-scaling.
By category (4 categories fired)
| Category | Firings | Measured | Win 24h | Avg 24h |
|---|---|---|---|---|
| Geopolitics | 153 | 150 | 57% | +6.39pp |
| Sports | 29 | 26 | 54% | +0.92pp |
| Science | 13 | 13 | 15% | +0.98pp |
| Politics | 5 | 5 | — | -21.36pp |
Win rate hidden until ≥10 measured firings — thin samples can flip with one outlier.
Per-market breakdown (11 markets fired)
| Market | Firings | Win 24h | Avg 1h | Avg 24h |
|---|---|---|---|---|
| Strait of Hormuz traffic returns to normal by end of May? | 45 | 33% | +0.06pp | -2.14pp |
| Israel x Syria security agreement by June 30? | 41 | 64% | -0.33pp | +1.83pp |
| Will Trump say "Iran" during events with Xi Jinping? | 37 | 73% | +0.49pp | +14.98pp |
| Will Trump say "Nuclear" during events with Xi Jinping? | 30 | 66% | +1.59pp | +14.84pp |
| Trump declassifies new UFO files by May 15? | 13 | 15% | -0.22pp | +0.98pp |
| Cavaliers vs. Pistons | 11 | 82% | +0.27pp | +2.95pp |
| San Diego Padres vs. Seattle Mariners | 10 | 40% | -0.95pp | +0.20pp |
| San Francisco Giants vs. Athletics | 6 | 20% | -1.25pp | -1.30pp |
| Will Trump say "Dumbocrat" during Bret Baier interview? | 5 | 0% | -6.79pp | -21.36pp |
| Spread: Pistons (-4.5) | 1 | — | -45.10pp | +0.00pp |
| Cavaliers vs. Pistons: O/U 205.5 | 1 | — | +49.45pp | +0.00pp |
Honest caveats
- Small samples. CLOB returns ~30 days of minute-level history; short-dated markets may have very few firings. Anything below 10 measured firings shows a dash instead of a win rate so we don't overstate confidence.
- Survivorship. This backtest runs on markets currently live — not on markets that already resolved and disappeared. Persistent historical coverage is planned so the full universe can be evaluated once enough live history has accumulated.
- No execution costs. Avg-move figures are mid-price to mid-price. Real entry/exit would pay spread, slippage, and Polymarket fees — so a +1.5pp 24h move is not +1.5pp of profit.
- Signal drift. If we tune the live rule, these numbers change with it. That's the point — the page always reflects the rule that's currently running, not a frozen historical version.