2af8e02fd8
Phase E.3 Task 23. Trains the Phase E execution-policy DQN on the first
80% of fxcache snapshots, then evaluates the frozen policy (ε=0) on the
held-out 20% across a transaction-cost sweep. Compares absolute Sharpe
vs the Phase 1d.4 always-market-when-confident baseline.
Pipeline pieces:
- Shared loaders extracted into crates/ml/src/env/loaders.rs (used by
both alpha_dqn_h600_smoke and alpha_compose_backtest)
- alpha_compose_backtest.rs: train DQN on first n_train bars, then
frozen-eval n_eval episodes per cost level
- cost grid: [0.0, 0.0625, 0.125, 0.25, 0.5] (price units per
contract round-turn)
- Annualised Sharpe via per-episode Sharpe × sqrt(episodes/year)
where episodes/year ≈ 252 · 6.5h · 3600s / (horizon · 12s)
Run (horizon=600, 1000 train ep, 500 eval ep/cost, 1.5M snapshots):
cost n_ep mean_R std_R Sharpe/ep Sharpe_ann win_rate
0.0000 500 -11.09 8.03 -1.380 -39.50 0.090
0.0625 500 -20.68 9.88 -2.093 -59.89 0.012
0.1250 500 -29.38 9.49 -3.095 -88.58 0.000
0.2500 500 -48.23 11.90 -4.052 -115.98 0.000
0.5000 500 -84.59 17.43 -4.854 -138.92 0.000
Phase 1d.4 baseline for comparison: +4.4 ann. at cost=0, -4.0 at half-tick.
The Phase E policy LOSES MONEY across the whole cost grid — even at
frictionless cost=0. This is not a contradiction with the H=600 PASS
verdict (rvr=+1.04σ): the smoke's rvr is RELATIVE TO RANDOM, while
backtest Sharpe is ABSOLUTE. "Better than random by 1 std" is still
losing if random loses big.
The diagnostic that the E.2 controller already surfaced:
ISV[543] STACKER_THRESHOLD saturated at upper clamp (0.5) — policy
trades 85% of the time vs the 8% target. Over-trading pays spread on
every bar regardless of alpha confidence. Even with perfect alpha
(Phase 1d.3 AUC=0.673), trading 85% × spread cost > alpha edge.
The Phase 1d.4 baseline beats us at cost=0 because it WAITS unless
|stacker_logit| > threshold — the threshold gate filters bars with
weak alpha signal. The Phase E controller PRODUCES slot 543 but the
DQN's action selection doesn't CONSUME it.
This is exactly what the E.3 backtest is FOR: revealing that the
Phase E.1/E.2 producer-side architecture without consumer-side gating
is incomplete. The composition backtest validates the architecture's
weak link.
NEXT (E.3 task 24-28 or a side fix): wire slot 543 consumption into
the action selection. At each step:
if |ISV[543] − 0.5| > |stacker_logit − 0.5|:
action = Wait // confidence below threshold, sit out
else:
action = argmax(Q)
Or equivalently: action = if confidence_high(alpha_logit, ISV[543])
{ argmax(Q) over Buy/Sell actions } else { Wait }.
Once slot 543 is consumed, re-run alpha_compose_backtest and expect
Sharpe to move toward / past the Phase 1d.4 baseline.
Loader refactor: extracted load_fill_model_from_json, load_alpha_cache,
load_snapshots_from_fxcache from alpha_dqn_h600_smoke.rs into
crates/ml/src/env/loaders.rs. The smoke now calls the shared module
via ml::env::loaders::*. ~150 lines of duplicated code removed.
Build + run verified: smoke still builds clean. Backtest runs in ~30s
(train 8s + eval 20s + setup).
Branch: sp20-aux-h-fixed, pushed.
77 lines
2.0 KiB
JSON
77 lines
2.0 KiB
JSON
{
|
|
"bins": [
|
|
{
|
|
"cost": 0.0,
|
|
"mean_reward": -11.089629173278809,
|
|
"n_episodes": 500,
|
|
"p05": -22.522815704345703,
|
|
"p50": -11.164673805236816,
|
|
"p95": 3.1911258697509766,
|
|
"sharpe_annualised": -39.49838638305664,
|
|
"sharpe_per_episode": -1.3801853656768799,
|
|
"std_reward": 8.034883499145508,
|
|
"win_rate": 0.09000000357627869
|
|
},
|
|
{
|
|
"cost": 0.0625,
|
|
"mean_reward": -20.684438705444336,
|
|
"n_episodes": 500,
|
|
"p05": -36.075843811035156,
|
|
"p50": -19.98797607421875,
|
|
"p95": -7.718697547912598,
|
|
"sharpe_annualised": -59.89125061035156,
|
|
"sharpe_per_episode": -2.0927696228027344,
|
|
"std_reward": 9.88376235961914,
|
|
"win_rate": 0.012000000104308128
|
|
},
|
|
{
|
|
"cost": 0.125,
|
|
"mean_reward": -29.383296966552734,
|
|
"n_episodes": 500,
|
|
"p05": -44.815711975097656,
|
|
"p50": -28.835330963134766,
|
|
"p95": -15.50295352935791,
|
|
"sharpe_annualised": -88.58140563964844,
|
|
"sharpe_per_episode": -3.095284938812256,
|
|
"std_reward": 9.492920875549316,
|
|
"win_rate": 0.0
|
|
},
|
|
{
|
|
"cost": 0.25,
|
|
"mean_reward": -48.22906494140625,
|
|
"n_episodes": 500,
|
|
"p05": -67.93758392333984,
|
|
"p50": -48.01160430908203,
|
|
"p95": -29.78984832763672,
|
|
"sharpe_annualised": -115.97515106201172,
|
|
"sharpe_per_episode": -4.052499771118164,
|
|
"std_reward": 11.9010648727417,
|
|
"win_rate": 0.0
|
|
},
|
|
{
|
|
"cost": 0.5,
|
|
"mean_reward": -84.58942413330078,
|
|
"n_episodes": 500,
|
|
"p05": -110.6795883178711,
|
|
"p50": -84.19437408447266,
|
|
"p95": -58.62353515625,
|
|
"sharpe_annualised": -138.9206085205078,
|
|
"sharpe_per_episode": -4.854279041290283,
|
|
"std_reward": 17.425745010375977,
|
|
"win_rate": 0.0
|
|
}
|
|
],
|
|
"cost_grid": [
|
|
0.0,
|
|
0.0625,
|
|
0.125,
|
|
0.25,
|
|
0.5
|
|
],
|
|
"horizon": 600,
|
|
"n_eval_episodes": 500,
|
|
"n_train_episodes": 1000,
|
|
"phase": "E.3 Task 23",
|
|
"train_cost": 0.0625,
|
|
"train_frac": 0.800000011920929
|
|
} |