A one-time measurement, data Jan 2024 → Aug 2026
Holdout: Every Signal Max 3 (formerly FWD-01)
The holdout is a slice of data kept locked and opened exactly once to test a chosen candidate. This time it was opened for the research candidate then called FWD-01 and kept private. On 10 Oct 2026 it was published as the EA's 6th strategy (v1.1.0): Every Signal Max 3, code D_V1_CAP3 — exactly Every Signal, capped at 3 open trades. Publishing changes nothing in the result or the caveats below.
Status: Forward testing · Frozen specification · No confirmed production edge.
Research candidate publication policy →
Read these 5 points BEFORE looking at any number
- 1. This is not clean out-of-sample evidence. The holdout manifest records the “never-seen” data for this strategy family as EMPTY. There is no stretch of data that is genuinely new to this strategy.
- 2. The result had already been seen — with a specific number. On 19 Sep 2026, one day BEFORE this candidate was chosen, a very close version of it was measured on exactly the 2024 → Aug 2026 period: average +0.418R per trade. The holdout came out at +0.462R — close to what had already been seen. Most of this result was predictable.
- 3. One candidate declared, hundreds tried. The test was registered with K = 1 (a single candidate), while the research line had tried hundreds of configurations — about 130 in one branch alone. Declared alongside 5 candidates, this very strategy FAILED (see the RL-14 box under Validation).
- 4. The formal Dukascopy run once showed half the edge — caused by missing days in the dataset. Formal run on days present in both sources: Exness +0.277R per trade, Dukascopy only +0.143 to +0.160R. A later diagnosis (11 Oct 2026) found the frozen Dukascopy dataset is missing 706 weekdays in 2016–2023, which distorted the result; on continuous Dukascopy data it matches Exness (+0.241R vs +0.251R). Every number on this page is from Exness.
- 5. It may simply suit this period. 2024–2026 was a strong, steady gold uptrend, and this is a trend-following strategy. The test cannot separate “has an edge” from “suits this period”. The holdout average (+0.46R) is nearly double the training data (+0.25R) — exactly the shape of a favourable period that had been seen in advance.
Not a recommendation to trade real money. Whether the result looks good or bad.
DECISION block
The rule below was written down and locked BEFORE the run. Only this block decides the verdict.
- Average per trade
- +0.462R
- One-sided p
- 0.0054
- Trades
- 441
- Weeks
- 107
Verdict: SURVIVES — meaning: this test, with exactly this threshold, could NOT reject the strategy. It does NOT mean the strategy is profitable, nor that it can be traded with real money.
Rule: Average profit per trade must exceed +0.05R at significance 0.05; weekly block bootstrap with 10,000 iterations, seed 20260911; at least 200 trades and 50 weeks.
DESCRIPTION block — computed in the same run, NOT used for the decision
The numbers below are descriptive only and play no part in the verdict above.
- 95% interval of average profit
- +0.172R … +0.767R
- Without the 5 best trades
- +0.332R
- With double spread
- +0.452R
- Largest drawdown
- 25.7R
By year
| Year | Trades | Avg/trade | Win rate | Profit factor |
|---|---|---|---|---|
| 2024 | 167 | +0.376R | 24.0% | 1.79 |
| 2025 | 181 | +0.590R | 25.4% | 2.61 |
| 2026 | 93 | +0.368R | 17.2% | 1.78 |
By month
13/31 losing months is NORMAL for this strategy: on the 2016–2023 training data, 54% of months were negative. It loses in most months and makes up for it with a few very large trades. One month — about 10 trades — tells you nothing.
| Month | Trades | Avg/trade | Win rate | Profit factor |
|---|---|---|---|---|
| 2024-01 | 8 | +0.375R | 25.0% | 2.50 |
| 2024-02 | 21 | −0.571R | 0.0% | 0.00 |
| 2024-03 | 14 | +2.929R | 50.0% | 6.86 |
| 2024-04 | 14 | −0.377R | 7.1% | 0.53 |
| 2024-05 | 13 | +0.907R | 23.1% | 3.36 |
| 2024-06 | 7 | +0.286R | 28.6% | 3.00 |
| 2024-07 | 24 | +0.792R | 37.5% | 3.37 |
| 2024-08 | 19 | −0.053R | 31.6% | 0.90 |
| 2024-09 | 15 | −0.200R | 20.0% | 0.57 |
| 2024-10 | 19 | +0.912R | 31.6% | 4.47 |
| 2024-11 | 9 | −0.667R | 11.1% | 0.14 |
| 2024-12 | 4 | −1.000R | 0.0% | 0.00 |
| 2025-01 | 23 | +0.640R | 34.8% | 3.79 |
| 2025-02 | 18 | +0.611R | 33.3% | 2.83 |
| 2025-03 | 20 | +0.850R | 30.0% | 2.89 |
| 2025-04 | 6 | +2.833R | 50.0% | 9.50 |
| 2025-05 | 9 | −0.444R | 0.0% | 0.00 |
| 2025-06 | 12 | −0.417R | 8.3% | 0.29 |
| 2025-07 | 14 | +0.143R | 21.4% | 1.40 |
| 2025-08 | 17 | +0.941R | 17.6% | 3.67 |
| 2025-09 | 18 | +1.111R | 33.3% | 5.00 |
| 2025-10 | 12 | +2.167R | 58.3% | 14.00 |
| 2025-11 | 16 | −0.563R | 6.3% | 0.10 |
| 2025-12 | 16 | +0.062R | 12.5% | 1.20 |
| 2026-01 | 20 | +2.250R | 35.0% | 7.43 |
| 2026-02 | 17 | +0.118R | 23.5% | 1.29 |
| 2026-03 | 4 | −0.500R | 0.0% | 0.00 |
| 2026-04 | 19 | −0.669R | 0.0% | 0.00 |
| 2026-05 | 8 | −0.500R | 0.0% | 0.00 |
| 2026-07 | 11 | −0.639R | 0.0% | 0.00 |
| 2026-08 | 14 | +0.929R | 35.7% | 4.25 |
Data through: 2026-08-30 · This page deliberately shows no equity curve: a rising line makes it easy to forget the 5 points above.
Before the run, the chance of SURVIVES was only about 14–21%
Research measured this before running: if the strategy is exactly as good as on the training data (+0.25R per trade), this test comes out SURVIVES only about 14–21% of the time, because 32 months of data is too little. Panda saw this table and chose to run it anyway. It came out SURVIVES because the holdout average was nearly double the training average — matching points 2 and 5 above, not a happy surprise. Conversely, a fail would not have meant the strategy was broken.
Timeline — including the failed attempt
- 1. RL-15: build the holdout data and lock it.
- 2. RL-16: build the one-time measurement. The decision rule and the power calculation were registered FIRST.
- 3. The first run was REJECTED after being claimed — a technical bug on an empty day (Saturday 6 Jan 2024). No result was counted.[ace01e3]
- 4. Panda granted ONE exception, recorded verbatim. RL-17 fixed the bug; the retry was machine-proven to be the same test — differing in 5 technical fields only, none touching the candidate, the rule or the data.[e3c0147 · 8c30072]
- 5. Retry → outcome committed BEFORE it was read; an independent verifier passed 40/40.[f69182b]
The research repo is currently private, so commits are listed as text. The original outcome file is downloadable under Documents.
How to read the words on this page
- SURVIVES ≠ profitable ≠ tradeable
- SURVIVES only means one specific test, with one specific threshold, could not reject the strategy.
- What K is
- K is the number of candidates declared before validation. The more declared, the stricter the bar. RL-14 is a live example: the same trades pass at K = 1 and fail at K = 5.
- What “contamination” is
- If you've already seen the data before choosing the strategy, a measurement on that same data is no longer independent.
- What “power” is
- The chance a test detects an edge if one really exists. With a weak test, a fail doesn't mean broken; and when it passes despite being hard to pass, that needs explaining.
- Why the holdout opens only once
- Open it repeatedly and you keep trying until it looks good, which defeats the purpose. This time there was one exception due to a technical bug, and it's recorded publicly instead of hidden.
- Price-source effect
- The same strategy run on two different price feeds can give materially different results. Here the half-the-edge gap turned out to come from missing days in the dataset, not the price feed — which is why data gaps must be checked before trusting a number.
Source: trading-research-lab/artifacts/research/rl16-holdout-confirm/rl16-hc-c541b2cac9868d47/outcome.json(not published) · sha256 bdf8f475de966e3d3a7e1d48aa43134d4f15e1d483e42b30ef77d270e059a314
