spec(dqn): mark Thompson rollout SUPERSEDED — motivating pathology was a measurement artefact
The val-Flat-collapse / Short-collapse / C51 expected-Q bias hypothesis that motivated the 4-plan distributional-RL Thompson rollout was largely a measurement artefact in the diagnostic infrastructure, not a real policy pathology. Three layered bugs in actions_history_buf init + reader + mag_stats attribution conspired to inflate val_dir_dist Short, inflate active_frac, and pin wr_h/wr_f to zero. After fixing all three (commitsa86fba2b1+b8788511c), val_dir_dist matches val_picked_dir_dist within ~5pp — no collapse, diverse picks. Status footer added to the spec documenting: - what was actually wrong (3 measurement bugs) - what the post-fix data shows (mild passivity bias from early training, not a structural collapse) - what is preserved (Phase 0 unit tests as latent infrastructure for any future distributional Q-head; Aggregation Contract pearl as a sound engineering invariant) - what is cancelled (Plans B / C / D — Phase 1 audit, Phase 2 Thompson integration, Phase 3 long verification — superseded) - future revival condition (if fresh L40S 30-epoch on post-fix code shows val_picked_dir_dist itself collapsing toward Hold/Flat, reopen) Plan files remain in docs/superpowers/plans/ as historical record.
This commit is contained in:
@@ -611,3 +611,33 @@ For any future distributional or ensemble-based value head added to the system:
|
||||
5. The pre-commit hook will refuse merges of distributional or ensemble heads missing the sampleable interface
|
||||
|
||||
This contract turns the principle into an engineering invariant. New methods can join the system without re-discovering the bias class.
|
||||
|
||||
---
|
||||
|
||||
## Status: SUPERSEDED — motivating pathology was a measurement artefact
|
||||
|
||||
**Date:** 2026-04-27. **Commits superseding the rollout:** `a86fba2b1` (val_dir_dist + active_frac fix) and `b8788511c` (mag_stats wr_h/wr_f attribution).
|
||||
|
||||
The val-Flat-collapse / Short-collapse / "C51 expected-Q bias toward Hold/Flat" observations that motivated this 4-plan rollout were largely **measurement artefacts** in the diagnostic infrastructure, not real policy pathologies.
|
||||
|
||||
**What was actually wrong**:
|
||||
1. `actions_history_buf` was zero-initialised; after `done_flags[w]=1` (capital floor breach), `backtest_env_step` early-returned without writing the buffer for `[done_step, max_len)`. Unwritten zeros decoded as `dir = 0/27 = 0 = Short Quarter Market Normal` → val_dir_dist showed 83% Short and active_frac 87-91% as artefacts.
|
||||
2. The `backtest_metrics_kernel` reader had no `< 0` guard, so even after fixing the init the kernel still treated unwritten -1 slots as Short via C integer division (`-1 / 9 = 0`).
|
||||
3. `mag_stats` was binning trade-close events by `actual_mag_core`, which is forced to 0 (Quarter) when `actual_dir` is Hold/Flat — so every Half/Full close was attributed to Quarter, structurally pinning `wr_h = wr_f = 0`.
|
||||
|
||||
**What the post-fix data shows** (3-fold × 5-epoch local smoke, RTX 3050 Ti):
|
||||
- val_dir_dist matches val_picked_dir_dist within ~5pp (no collapse — diverse picks)
|
||||
- active_frac ≈ 31-50% (real, was 87-91% artefact)
|
||||
- dir_entropy ≈ 0.83-1.02 (real, was 0.57 artefact)
|
||||
- Hold+Flat ≈ 50% — mild passivity bias from early training, not a structural collapse
|
||||
- eval Quarter dominance ≈ 60-80% explained by Hold/Flat→Quarter accounting (50%) + var_scale shrinkage of directional bars (further 10-30%)
|
||||
- wr_h/wr_f remain 0 in smoke because var_scale ≈ 0.10-0.20 makes Half/Full positions unreachable until training matures (var_q drops). The `b8788511c` fix is latent correctness for mature L40S runs.
|
||||
|
||||
**What is preserved**:
|
||||
- **Phase 0 unit tests** (commits `2186f96e8` through `82d39a895`): the standalone `thompson_test_kernel.cu` and the 6 GPU-direct + CPU tests in `distributional_q_tests.rs` are valid Thompson-math correctness infrastructure. They remain on `main` as latent infrastructure for any future distributional value head added to the system. They neither reference nor depend on the val-Flat-collapse hypothesis.
|
||||
- **The Distributional Q-head Aggregation Contract above** (the 5-point pearl on sampleable interfaces): this is a sound engineering invariant independent of whether the val-Flat-collapse pathology was real. It applies to any future C51/IQN/ensemble Q-head and is worth keeping as a guideline. The pre-commit hook enforcement is deferred until a real new distributional Q-head is added — without a current consumer there is nothing to gate.
|
||||
|
||||
**What is cancelled**:
|
||||
- **Plans B / C / D** (`plan-B-phase-1.md`, `plan-C-phase-2.md`, `plan-D-phase-3.md`): superseded. Phase 1 audit, Phase 2 production Thompson integration, and Phase 3 long verification are not executed because the motivating pathology does not exist post-`a86fba2b1` + `b8788511c`. The plans remain in `docs/superpowers/plans/` as historical record of the investigation.
|
||||
|
||||
**Future revival condition**: if a fresh L40S 30-epoch baseline (or longer) on post-fix code shows val_dir_dist genuinely collapsing toward Hold/Flat over training (i.e. the kernel `val_picked_dir_dist` itself shows the bias, not just the post-physics `val_dir_dist`), reopen this spec. The Phase 0 unit tests are the verification harness already in place to gate Phase 2.
|
||||
|
||||
Reference in New Issue
Block a user