cd5aa3402b
Phase E.1 Task 12b complete. The H=600 DQN smoke now consumes real
alpha_logit from the Phase 1d.3 stacker (Mamba2 + 7-input MLP stacker
trained for AUC=0.673 on test), and PASSES all four kill criteria:
Q_SPREAD_EMA = 10.92 ≥ 0.05 PASS
ACTION_ENTROPY_EMA = 1.97 ≥ 1.099 PASS
RETURN_VS_RANDOM_EMA = +1.043 ≥ 0.0 PASS ← jumped +3.62σ
EARLY_Q_MOVEMENT_EMA = 0.099 ≥ 0.01 PASS
Overall: PASS (H=6000 scale-up VIABLE)
Before/after comparison (same env, same DQN, only alpha_logit changed):
alpha_logit=0 alpha_logit=Phase1d.3
rollout_R_mean (final) -18,272 -18
RETURN_VS_RANDOM_EMA -2.58σ +1.04σ
Overall verdict FAIL PASS
The 1000× reduction in episode loss + the +3.62σ rvr swing definitively
proves the "first-best-action lock-in" hypothesis from the previous FAIL
analysis was a SYMPTOM, not the cause. The cause was alpha_logit=0
placeholder starving the policy of directional signal. With real Phase
1d.3 alpha, the linear Q-network learns to use it cleanly — no
NoisyNet, no MLP, no architectural change needed.
Integration pieces in this commit:
1. Cargo workspace registration: ml-alpha added as a workspace dep,
ml's manifest now depends on ml-alpha for FxCacheReader access.
(ml-alpha already depends only on ml-core, so no circular risk.)
2. alpha_dqn_h600_smoke.rs: two new CLI args
--fxcache-path <PATH> load snapshots from precomputed fxcache
(mid from raw_close, bid/ask synthesized
at fixed half-tick, 81-dim features extracted
for spread_bps / l1_imbalance / ofi / mid_drift)
--alpha-cache <PATH> load Phase 1d.3 stacker logit cache produced
by `alpha_train_stacker --alpha-cache-out`.
Each cache entry aligns to the corresponding
fxcache bar, populates SnapshotRow.alpha_logit
(and derives alpha_confidence = |sigmoid(z)-0.5|).
3. Snapshot source selection: in main(), --fxcache-path takes priority
when both paths are set; --alpha-cache requires --fxcache-path
(alignment guarantee). Original --mbp10-dir path unchanged for
non-cached runs.
4. Two new helper fns: load_alpha_cache (binary [u32 n] + [f32; n]
reader), load_snapshots_from_fxcache (FxCacheReader → Vec<SnapshotRow>
with synthesized bid/ask and alpha_logit/alpha_confidence from cache).
alpha_logits_cache.bin (7.6 MB, 1.97M f32 entries) is .gitignore'd —
regenerable from `cargo run -p ml-alpha --release --example
alpha_train_stacker -- --fxcache-path <FXC> --alpha-cache-out
config/ml/alpha_logits_cache.bin` (~2 min on RTX 3050 Ti).
Reproduction of this PASS verdict:
cargo run -p ml --release --example alpha_dqn_h600_smoke -- \
--fxcache-path /home/jgrusewski/Work/foxhunt/test_data/feature-cache/9297....fxcache \
--alpha-cache config/ml/alpha_logits_cache.bin \
--horizon 600 --n-episodes 1000
Total run time ~10s after fxcache load. Verdict + per-checkpoint KC
trajectory in config/ml/alpha_dqn_h600_smoke.json.
NEXT: Task 13 — scale to H=6000 (the production horizon). Per the plan,
PASS at H=600 unlocks H=6000.
166 lines
4.2 KiB
JSON
166 lines
4.2 KiB
JSON
{
|
|
"action_entropy_ema": 1.968935489654541,
|
|
"all_pass": true,
|
|
"alpha_m": 0.8999999761581421,
|
|
"early_q_movement_ema": 0.09944231063127518,
|
|
"eps_end": 0.05000000074505806,
|
|
"eps_start": 0.5,
|
|
"gamma": 0.9900000095367432,
|
|
"grad_clip": 1.0,
|
|
"horizon": 600,
|
|
"kc_log": [
|
|
{
|
|
"early_mvmt": 0.011208697222173214,
|
|
"entropy": 2.0113048553466797,
|
|
"episode": 50,
|
|
"q_spread": 38.21281433105469,
|
|
"rvr": 1.0437408685684204
|
|
},
|
|
{
|
|
"early_mvmt": 0.016528302803635597,
|
|
"entropy": 2.0112295150756836,
|
|
"episode": 100,
|
|
"q_spread": 27.53325080871582,
|
|
"rvr": 1.0437407493591309
|
|
},
|
|
{
|
|
"early_mvmt": 0.022869938984513283,
|
|
"entropy": 2.0082004070281982,
|
|
"episode": 150,
|
|
"q_spread": 21.13676643371582,
|
|
"rvr": 1.0437486171722412
|
|
},
|
|
{
|
|
"early_mvmt": 0.029317570850253105,
|
|
"entropy": 2.005980968475342,
|
|
"episode": 200,
|
|
"q_spread": 20.710119247436523,
|
|
"rvr": 1.04374098777771
|
|
},
|
|
{
|
|
"early_mvmt": 0.0356033593416214,
|
|
"entropy": 2.0036354064941406,
|
|
"episode": 250,
|
|
"q_spread": 19.258197784423828,
|
|
"rvr": 1.0437567234039307
|
|
},
|
|
{
|
|
"early_mvmt": 0.041733257472515106,
|
|
"entropy": 2.001204252243042,
|
|
"episode": 300,
|
|
"q_spread": 26.219839096069336,
|
|
"rvr": 1.0437005758285522
|
|
},
|
|
{
|
|
"early_mvmt": 0.047568317502737045,
|
|
"entropy": 1.9986319541931152,
|
|
"episode": 350,
|
|
"q_spread": 16.797637939453125,
|
|
"rvr": 1.0436869859695435
|
|
},
|
|
{
|
|
"early_mvmt": 0.053121890872716904,
|
|
"entropy": 1.9962358474731445,
|
|
"episode": 400,
|
|
"q_spread": 12.861183166503906,
|
|
"rvr": 1.0436040163040161
|
|
},
|
|
{
|
|
"early_mvmt": 0.05835384875535965,
|
|
"entropy": 1.993706464767456,
|
|
"episode": 450,
|
|
"q_spread": 11.837430000305176,
|
|
"rvr": 1.043595314025879
|
|
},
|
|
{
|
|
"early_mvmt": 0.06333397328853607,
|
|
"entropy": 1.9912099838256836,
|
|
"episode": 500,
|
|
"q_spread": 14.526165008544922,
|
|
"rvr": 1.0435247421264648
|
|
},
|
|
{
|
|
"early_mvmt": 0.0680142194032669,
|
|
"entropy": 1.988599419593811,
|
|
"episode": 550,
|
|
"q_spread": 13.646886825561523,
|
|
"rvr": 1.043499231338501
|
|
},
|
|
{
|
|
"early_mvmt": 0.07242720574140549,
|
|
"entropy": 1.9862360954284668,
|
|
"episode": 600,
|
|
"q_spread": 13.742671012878418,
|
|
"rvr": 1.0434374809265137
|
|
},
|
|
{
|
|
"early_mvmt": 0.07657017558813095,
|
|
"entropy": 1.9838505983352661,
|
|
"episode": 650,
|
|
"q_spread": 17.030311584472656,
|
|
"rvr": 1.0434088706970215
|
|
},
|
|
{
|
|
"early_mvmt": 0.08046558499336243,
|
|
"entropy": 1.9814624786376953,
|
|
"episode": 700,
|
|
"q_spread": 19.744157791137695,
|
|
"rvr": 1.043370008468628
|
|
},
|
|
{
|
|
"early_mvmt": 0.08412447571754456,
|
|
"entropy": 1.9791860580444336,
|
|
"episode": 750,
|
|
"q_spread": 16.830333709716797,
|
|
"rvr": 1.0432881116867065
|
|
},
|
|
{
|
|
"early_mvmt": 0.08755383640527725,
|
|
"entropy": 1.9769866466522217,
|
|
"episode": 800,
|
|
"q_spread": 18.11051368713379,
|
|
"rvr": 1.0432313680648804
|
|
},
|
|
{
|
|
"early_mvmt": 0.09078100323677063,
|
|
"entropy": 1.9748573303222656,
|
|
"episode": 850,
|
|
"q_spread": 21.77625846862793,
|
|
"rvr": 1.0432629585266113
|
|
},
|
|
{
|
|
"early_mvmt": 0.09382475167512894,
|
|
"entropy": 1.972854495048523,
|
|
"episode": 900,
|
|
"q_spread": 14.12209415435791,
|
|
"rvr": 1.0431597232818604
|
|
},
|
|
{
|
|
"early_mvmt": 0.09670793265104294,
|
|
"entropy": 1.9708701372146606,
|
|
"episode": 950,
|
|
"q_spread": 12.469049453735352,
|
|
"rvr": 1.0431069135665894
|
|
},
|
|
{
|
|
"early_mvmt": 0.09944231063127518,
|
|
"entropy": 1.968935489654541,
|
|
"episode": 1000,
|
|
"q_spread": 10.922320365905762,
|
|
"rvr": 1.0431302785873413
|
|
}
|
|
],
|
|
"lr": 0.00009999999747378752,
|
|
"n_episodes": 1000,
|
|
"pass_early": true,
|
|
"pass_entropy": true,
|
|
"pass_q_spread": true,
|
|
"pass_rvr": true,
|
|
"phase": "E.1 Task 12",
|
|
"q_init_norm": 2.5735182762145996,
|
|
"q_spread_ema": 10.922320365905762,
|
|
"return_vs_random_ema": 1.0431302785873413,
|
|
"reward_scale": 1000.0,
|
|
"target_update_every": 10,
|
|
"tau": 0.029999999329447746
|
|
} |