From idea to testable evidence
Research process
A trading strategy isn't trustworthy just because it exists — it has to go through several rounds of testing before anyone should consider using real money. Here are the steps, and where each of the 6 strategies on this site currently stands.
How the data was sourced
The first 5 steps all use the same data sourcing and processing, described below so you can judge how much to trust it.
- Real price ticks, not ready-made candles
- The raw data is every XAUUSD price tick (bid/ask at each price change). M15, H1 and H4 candles are rebuilt from ticks by the same code for every source.
- Two independent sources
- Exness: 2016 → Aug 2026. Dukascopy: Jul 2004 → 2023. The two are time-aligned; M15 closing prices differ by a median of only $0.03–0.13 per year (2016–2023).
- Dukascopy has gaps
- In 2016–2023, on 675 of 2,387 days that Exness has data, Dukascopy has no candles at all. So Dukascopy numbers are only used as a cross-check, not as the main numbers.
- Trading costs
- Spread comes from the real ticks. Commission $7/lot. Swap (overnight fee) isn't in the research engine; it's estimated separately at about −0.04R per day held.
- Same risk everywhere
- Every number uses a fixed $50 risk per trade.
- Step 5 runs on real MT5
- MT5 Strategy Tester, Exness-MT5Trial14 server, XAUUSDm M15. Exness only has real ticks from 2026; 2020–2025 use ticks simulated from M1 candles.
Show data weaknesses
- Exness 2016–2023 was used to design and tune the strategies (in-sample), so numbers on that period tend to look better than reality.
- Dukascopy 2005–2015 was partly looked at while researching the older strategy family, so it isn't fully 'clean' either.
- Data from 2024 to Aug 2026 is the “holdout”: it was looked at during exploration, then officially opened exactly ONCE for Every Signal Max 3, then still called research candidate FWD-01 (see the Holdout page). There is no stretch of data left that was never looked at.
How the data was sourced: build-bars/build_bars.py · source-gap/source-gap.md · source-gap/missing-bars.md · duka-2005-2023/REPORT.md · regime-gate/REPORT.md ($7/lot) · atr-and-recent/result.md (swap)(not published) · data/backtests.json (MT5)(not published)
1. Idea
Write the entry and exit rules into a precise, computable formula — based on the ideas listed under Origins.
All 6 strategies have gone through this step.
Show results per strategy
| Strategy | Main entry rule | SL to break-even at | Max open trades |
|---|---|---|---|
| Classic Pullback | Price pulls back to EMA34/89 then bounces, any hour | +2R | 3 |
| Near-Touch Pullback | Like Classic Pullback, price only needs to get close to the EMA | +2R | 3 |
| 3-Bar Pullback | Like Classic Pullback, EMA touched within the last 3 bars | +2R | 3 |
| Every Signal | Like Classic Pullback, takes every signal | +1R | 20 (technical cap) |
| London Open 7AM | Checks 7:00 UTC only each day, H4 rising | +2R | 3 |
| Every Signal Max 3 | Exactly Every Signal, with a cap on open trades | +1R | 3 |
Source: trading-research-lab/.ai/evidence/champion-rerun-2026-09-17/forward-demo/BRIEF-brain-forward-demo-C-D.md(not published) · BRIEF-brain-ea-4-modes-v0-v1.md(not published) · BRIEF-brain-ea-5-modes-v1.md · PROMPT-brain-ea-D_V1_CAP3.md §1(not published)
2. Multi-year backtest, two data sources
Test each strategy against a long history of past prices from two independent data providers — Exness (2016–2026) and Dukascopy (2005–2023) — so results don't depend on one source's quirks.
All 6 strategies have gone through this step.
Show results per strategy
Same period for all: Exness data 2016–2023, $50 risk per trade, before swap. “—” = no data yet.
| Strategy | Trades | Avg per trade | Profit factor | Net profit | Max drawdown | Dukascopy (longer data) |
|---|---|---|---|---|---|---|
| Classic Pullback | 909 | +0.262R | 1.40 | +$10,918 | $2,957 | 2005–2023: +$26,694 · lost 5/19 years |
| Near-Touch Pullback | 1,174 | +0.265R | 1.41 | +$14,129 | $3,295 | 2005–2023: +$32,062 · lost 4/19 years |
| 3-Bar Pullback | 1,082 | +0.273R | 1.42 | +$13,497 | $3,594 | 2005–2023: +$30,428 · lost 6/19 years |
| Every Signal | 1,665 | +0.231R | 1.45 | +$17,050 | $4,351 | 2005–2023: +$42,901 · lost 5/19 years |
| London Open 7AM | 337 | +0.323R | 1.49 | +$4,806 | $1,515 | 2005–2023: +$11,373 · lost 6/19 years |
| Every Signal Max 3 | 1,046 | +0.251R | 1.50 | — | $2,494 | — |
Average profit per trade (R), Exness 2016–2023
Source: trading-research-lab/.ai/evidence/champion-rerun-2026-09-17/web-numbers-2026-09-19/result-24h.md(not published) · web-numbers-24h-2026-09-20/result.md(not published) · web-numbers-2026-09-19/result.md (London Open 7AM). $50 risk per trade, before swap.(not published)
3. Robustness testing
Try changing trading costs (spread, slippage), nudging parameters, running tens of thousands of random simulations (Monte Carlo), and splitting the data in half — to see if results hold up, or only look good under one specific setup.
Released version (24/7): all 4 Pullback/Every Signal strategies pass this preliminary round, and so do London Open 7AM and Every Signal Max 3.
Show results per strategy
| Strategy | Double costs | Parameter tweak: worst drop | First / second half | Prop-account blow-up risk at $100/trade | Result |
|---|---|---|---|---|---|
| Classic Pullback | +0.218R | −17% | +0.295R / +0.230R | 3.2% | |
| Near-Touch Pullback | +0.232R | −24% | +0.260R / +0.270R | 6.6% | |
| 3-Bar Pullback | +0.215R | −23% | +0.304R / +0.241R | 6.3% | |
| Every Signal | +0.217R | −43% | +0.228R / +0.233R | 12.4% | |
| London Open 7AM | +0.359R | −38% | +0.259R / +0.381R | 0.0% | |
| Every Signal Max 3 | +0.197R | −20% | +0.263R / +0.241R | 1.6% |
Small parameter tweaks: worst drop in average profit (%)
— Fail line 50%
Source: trading-research-lab/.ai/evidence/champion-rerun-2026-09-17/web-numbers-2026-09-19/gate-24h.md · web-numbers-2026-09-19/result.md (London Open 7AM). Across the group: removing the 10 best trades costs 42–60% of average profit.(not published)
4. Official statistical validation
The strictest step: a formal statistical test with criteria fixed in advance, Holm-corrected because several strategies are tested at once. It yields a p-value; the pre-registered threshold is p < 0.05.
The first 5 strategies all FAILED. Every Signal Max 3 (formerly FWD-01) PASSED when declared alone but FAILED when declared alongside 4 others, on the same trades. See the RL-14 box below and the Holdout page.
Show results per strategy
Holm-adjusted p-value (threshold p < 0.05)
— Must be under 5%
| Strategy | Trades | Avg per trade | p-value (Holm) | Result |
|---|---|---|---|---|
| Classic Pullback | 909 | +0.262R | 12.8% | |
| Near-Touch Pullback | 1,174 | +0.265R | 12.8% | |
| 3-Bar Pullback | 1,082 | +0.273R | 12.8% | |
| Every Signal | 1,665 | +0.231R | 12.8% | |
| London Open 7AM | 337 | +0.323R | 6.4% |
RL-14: the same trades, two opposite verdicts
| Candidates declared | Candidates | Result | p-value (Holm) | Trades |
|---|---|---|---|---|
| K = 1 | Every Signal Max 3 | Passed (SURVIVES) — but read the note right below | 3.38% | 1046 |
| K = 5 | Classic Pullback · Near-Touch Pullback · 3-Bar Pullback · Every Signal · Every Signal Max 3 | All 5 failed | 16.05% | 909 / 1174 / 1082 / 1665 / 1046 |
Both rows ran on the SAME 1,046 trades of candidate FWD-01 (now Every Signal Max 3, D_V1_CAP3). The only difference is the number of candidates (K) declared in the pre-registration. K = 1 is NOT an honest count of how many things were tried: this candidate was picked AFTER reading RL-13's results, against a background of about 130 exploratory configurations. So the “Passed” on the K = 1 row is NOT evidence of an edge.
Source: trading-research-lab/artifacts/research/rl12-cv1/rl10-d-2b7e1e06b23209bd/outcome.json · trading-research-lab/artifacts/research/rl11-l07s/rl10-d-ef6a339beda7a1c2/outcome.json · rl14-d-cap3-k1/rl10-d-2a8db1fca18c3e1a · rl16-holdout-confirm/rl16-hc-c541b2cac9868d47/outcome.json(not published)
5. Trade-by-trade match against MT5
Compare every single trade between the research engine and the EA actually running on MetaTrader 5 — to make sure the EA does exactly what was calculated, with no coding mismatch.
Trade-by-trade comparison of the EA on MT5 against research's validation data, shared period 2020–2023: 85–97% of trades match on the entry bar, and 98% of matched trades have the same win/loss outcome. Done on the session-filtered version. Separately, the 5-mode EA vs the 4-mode EA in 2025: 129/129 and 180/180 trades match.
Show results per strategy
Source: panda-trading-brain/mql5/forward/EVIDENCE-2026-09-19-web-clmca.md §1 — brain re-ran it twice independently and compared every trade with a script (tools/compare_trades.py) · panda-trading-brain/mql5/forward/clmca-cap3-private/README.md (EA ↔ ledger comparison, self-test)(not published) · clmca-ea CHANGELOG.md v1.1.0 (Vantage 2025 regression)(not published)
6. Real demo run (no money at risk)
Attach the EA to a demo account to measure real costs (slippage, spread) and check whether the system behaves as the backtest predicted.
The original version (fixed-dollar thresholds) is running on a real demo account. Every Signal Max 3 has been running on demo since 21 Sep 2026. No strategy on this site has enough demo results to report yet.
Show results per strategy
- Not yetClassic PullbackOriginal ($-fixed) version running on demo; not enough data to report.
- Not yetNear-Touch PullbackNot started.
- Not yet3-Bar PullbackNot started.
- Not yetEvery SignalOriginal ($-fixed) version running on demo; not enough data to report.
- Not yetLondon Open 7AMNot started.
- Not yetEvery Signal Max 3Running on demo (prospective validation); not enough data to report.
Source: trading-research-lab/.ai/evidence/champion-rerun-2026-09-17/forward-demo/BRIEF-brain-forward-demo-C-D.md §0(not published) · PROMPT-brain-ea-D_V1_CAP3.md (21 Sep 2026, demo only)(not published)
Publication principle
Open research by default. The implementation of candidates under prospective validation may be kept private until validation ends or a different publication decision is made. Keeping a candidate private does NOT mean it is better than the public strategies. Candidate FWD-01 was kept private that way (aggregate numbers only); on 10 Oct 2026 the author chose to publish it as the 6th strategy, Every Signal Max 3 (D_V1_CAP3), in EA v1.1.0, even though prospective validation isn't finished. Being public does not make it more trustworthy.
Research candidate publication policy →About 190 configurations tried on the same data
This number matters as much as the results. The more variants you try on one dataset, the easier it is to land on something that only looks good by luck. That's why a good-looking result isn't enough on its own.
Show results per strategy
- By date: about 104 configurations when building the R-step exit family (Sep 17) → about 130 → about 136 when switching thresholds to ATR → about 142 when loosening the pullback rule → about 150 (Sep 18) → about 40 more on Sep 19, including the 24/7 variant.
- Formal validation corrects for the number of tests within one family (Holm correction), but it does NOT correct for choosing the 24/7 variant after seeing the numbers on that same data — research states this explicitly.
- All of it ran on the same Exness 2016–2023 data. There is still no stretch of data that was never looked at, for a truly independent check.
Source: trading-research-lab/.ai/evidence/champion-rerun-2026-09-17/CANDIDATE-step-r-2026-09-17.md · robustness-atr/REPORT.md (old numbers: EMA-style ATR, session-filtered version — history only) · robustness-pullback/REPORT.md (old numbers: EMA-style ATR, session-filtered version — history only) · trading-research-lab/.ai/CHECKPOINT-2026-09-18b-atr-p1b-swap-random-baseline.md(not published) · CHECKPOINT-2026-09-19-web-numbers-va-rl13-24h.md(not published)
Real money should only be considered after a strategy has gone through ALL of the steps above. None has reached the final step yet. This process is an attempt to get it right, NOT a guarantee. The process itself can be wrong: criteria that don't fit, gaps in the data, or something the people doing it simply missed. This site records a few such cases — a hidden filter that prettified numbers, and an ATR computed the wrong way. Both were caught by re-checking our own work, and there may be mistakes nobody has spotted yet.
