spec(dqn): mark Thompson rollout SUPERSEDED — motivating pathology was a measurement artefact

The val-Flat-collapse / Short-collapse / C51 expected-Q bias hypothesis
that motivated the 4-plan distributional-RL Thompson rollout was
largely a measurement artefact in the diagnostic infrastructure, not
a real policy pathology. Three layered bugs in actions_history_buf
init + reader + mag_stats attribution conspired to inflate val_dir_dist
Short, inflate active_frac, and pin wr_h/wr_f to zero. After fixing
all three (commits a86fba2b1 + b8788511c), val_dir_dist matches
val_picked_dir_dist within ~5pp — no collapse, diverse picks.

Status footer added to the spec documenting:

  - what was actually wrong (3 measurement bugs)
  - what the post-fix data shows (mild passivity bias from early
    training, not a structural collapse)
  - what is preserved (Phase 0 unit tests as latent infrastructure
    for any future distributional Q-head; Aggregation Contract
    pearl as a sound engineering invariant)
  - what is cancelled (Plans B / C / D — Phase 1 audit, Phase 2
    Thompson integration, Phase 3 long verification — superseded)
  - future revival condition (if fresh L40S 30-epoch on post-fix
    code shows val_picked_dir_dist itself collapsing toward Hold/Flat,
    reopen)

Plan files remain in docs/superpowers/plans/ as historical record.
This commit is contained in:
jgrusewski
2026-04-27 12:23:34 +02:00
parent b8788511ce
commit 42ffd6aadc

View File

@@ -611,3 +611,33 @@ For any future distributional or ensemble-based value head added to the system:
5. The pre-commit hook will refuse merges of distributional or ensemble heads missing the sampleable interface
This contract turns the principle into an engineering invariant. New methods can join the system without re-discovering the bias class.
---
## Status: SUPERSEDED — motivating pathology was a measurement artefact
**Date:** 2026-04-27. **Commits superseding the rollout:** `a86fba2b1` (val_dir_dist + active_frac fix) and `b8788511c` (mag_stats wr_h/wr_f attribution).
The val-Flat-collapse / Short-collapse / "C51 expected-Q bias toward Hold/Flat" observations that motivated this 4-plan rollout were largely **measurement artefacts** in the diagnostic infrastructure, not real policy pathologies.
**What was actually wrong**:
1. `actions_history_buf` was zero-initialised; after `done_flags[w]=1` (capital floor breach), `backtest_env_step` early-returned without writing the buffer for `[done_step, max_len)`. Unwritten zeros decoded as `dir = 0/27 = 0 = Short Quarter Market Normal` → val_dir_dist showed 83% Short and active_frac 87-91% as artefacts.
2. The `backtest_metrics_kernel` reader had no `< 0` guard, so even after fixing the init the kernel still treated unwritten -1 slots as Short via C integer division (`-1 / 9 = 0`).
3. `mag_stats` was binning trade-close events by `actual_mag_core`, which is forced to 0 (Quarter) when `actual_dir` is Hold/Flat — so every Half/Full close was attributed to Quarter, structurally pinning `wr_h = wr_f = 0`.
**What the post-fix data shows** (3-fold × 5-epoch local smoke, RTX 3050 Ti):
- val_dir_dist matches val_picked_dir_dist within ~5pp (no collapse — diverse picks)
- active_frac ≈ 31-50% (real, was 87-91% artefact)
- dir_entropy ≈ 0.83-1.02 (real, was 0.57 artefact)
- Hold+Flat ≈ 50% — mild passivity bias from early training, not a structural collapse
- eval Quarter dominance ≈ 60-80% explained by Hold/Flat→Quarter accounting (50%) + var_scale shrinkage of directional bars (further 10-30%)
- wr_h/wr_f remain 0 in smoke because var_scale ≈ 0.10-0.20 makes Half/Full positions unreachable until training matures (var_q drops). The `b8788511c` fix is latent correctness for mature L40S runs.
**What is preserved**:
- **Phase 0 unit tests** (commits `2186f96e8` through `82d39a895`): the standalone `thompson_test_kernel.cu` and the 6 GPU-direct + CPU tests in `distributional_q_tests.rs` are valid Thompson-math correctness infrastructure. They remain on `main` as latent infrastructure for any future distributional value head added to the system. They neither reference nor depend on the val-Flat-collapse hypothesis.
- **The Distributional Q-head Aggregation Contract above** (the 5-point pearl on sampleable interfaces): this is a sound engineering invariant independent of whether the val-Flat-collapse pathology was real. It applies to any future C51/IQN/ensemble Q-head and is worth keeping as a guideline. The pre-commit hook enforcement is deferred until a real new distributional Q-head is added — without a current consumer there is nothing to gate.
**What is cancelled**:
- **Plans B / C / D** (`plan-B-phase-1.md`, `plan-C-phase-2.md`, `plan-D-phase-3.md`): superseded. Phase 1 audit, Phase 2 production Thompson integration, and Phase 3 long verification are not executed because the motivating pathology does not exist post-`a86fba2b1` + `b8788511c`. The plans remain in `docs/superpowers/plans/` as historical record of the investigation.
**Future revival condition**: if a fresh L40S 30-epoch baseline (or longer) on post-fix code shows val_dir_dist genuinely collapsing toward Hold/Flat over training (i.e. the kernel `val_picked_dir_dist` itself shows the bias, not just the post-physics `val_dir_dist`), reopen this spec. The Phase 0 unit tests are the verification harness already in place to gate Phase 2.