Quant Strategy Validation Bench
Most backtesting tools are built to make a strategy look good. This one is built to catch a strategy lying. The core insight is arithmetic: a binary option pays out less than it takes, so a 50% hit rate is a guaranteed loss and even 55% loses at a typical payout. Optimising for win rate therefore selects for strategies that are quietly unprofitable. The bench refuses to report win rate as a success metric and evaluates expectancy per unit staked instead, with a deliberately worthless coin-flip strategy running alongside as a control.
Python package with a strategy library, evaluation engine, configuration and data pipeline
Documented as an architecture record. The reasoning behind the methodology is the deliverable; published backtest numbers would be meaningless without the data behind them.
How a strategy gets rejected
Follow a hypothesis through evaluation. The control path exists so you can see what noise scores — and how often a 'working' strategy fails to beat it.
A declarative YAML hypothesis: indicators, thresholds, asset and timeframe.
Why win rate never appears as the answer.
Highlights
A harness designed to reject
The pipeline is deliberately simple — spec, data, evaluate, report — because the value is in the metric choice, not in machinery. Every design decision exists to make a bad strategy look bad.
Strategy specifications
Declarative definitions of entry conditions, indicator parameters and the asset a hypothesis applies to.
- YAML rather than code, so variants are cheap to generate and diff.
- Indicator families are parameterised, exposing sensitivity to tuning.
- A control specification with no predictive content sits alongside the real ones.
Evaluation engine
Runs a specification over historical data and computes payout-aware outcomes.
- Expectancy per unit staked is the headline metric.
- The breakeven threshold is derived from the payout, not assumed to be 50%.
- Win rate is reported only as context, never as a verdict.
Data pipeline
Supplies real historical market data to the evaluation engine.
- All computation is local; nothing is sent to a broker.
- Data and results are kept separate from the strategy definitions.
Every module in the bench
Note that validation and risk are separate top-level modules, not helpers inside the backtester — the whole point of the tool is that evaluation is not the same thing as simulation. 11 units across 4 groups.
Evaluation
4Strategy
3Data
2Interfaces
2Stack by layer
Delivered work
Methodology
- Payout-aware breakeven derivation replacing the naive 50% assumption.
- Expectancy per unit staked as the primary reported metric.
- A coin-flip control strategy shipped as a first-class citizen.
- Explicit documentation of why the usual success metric is misleading.
Harness
- Declarative YAML strategy specifications across multiple indicator families.
- Seventeen strategy configurations spanning eight assets.
- Local-only evaluation with no order execution path at all.
Modules and demo strategy
Payout-aware evaluation
Prevents the most common backtesting self-deception.
- Breakeven is computed from the payout, not assumed.
- Turns 'it wins more than half the time' into a non-argument.
Control strategy
Establishes what noise scores.
- Any result that fails to beat it is discarded.
- Cheap to run and surprisingly hard to beat.
Declarative specs
Makes hypotheses cheap to generate and compare.
- Variants across assets expose overfitting quickly.