spec(dqn): reframe Phase 3 as direction-branch dead-code cleanup

User correctly challenged the "band-aid removal" framing. All fixes
shipped during the val-Flat-collapse investigation addressed real bugs
at their respective layers and should be preserved:

  - Kelly cap warm-branch (0c9d1ee39): post-decision physics layer.
    Thompson-independent. KEEP.
  - Train Return display + Sharpe annualization (non-tau parts of
    7a3d88646): display/metric layer. Thompson-independent. KEEP.
  - Direction Boltzmann tau-floor (tau part of 7a3d88646) +
    adaptive eps_dir floor (d54b49efc): gates inside direction-branch
    action selection. Phase 2 replaces direction-branch action
    selection wholesale (eps-greedy + Boltzmann → Thompson), so these
    direction-only code paths become structurally unreachable.

The latter two are NOT band-aids being removed because Thompson is
better. They are dead code being cleaned up because Thompson replaces
the surrounding mechanism. Magnitude/order/urgency branches keep their
existing eps-greedy + Boltzmann + tau-floor + EPS_FLOOR paths intact.

Reframed Phase 3 deliverable: "direction-branch dead-code cleanup"
with explicit rationale (per feedback_no_legacy_aliases.md and
feedback_no_partial_refactor.md). 0.5 day budget instead of 1.

Also clarified eval action selection: argmax of (E[Q_C51]+E[Q_IQN])/2
is correct. Bellman backup is a Q-learning UPDATE rule, not an
action-selection rule. Once Q is learned, optimal policy is greedy
argmax of learned Q. Online Bellman lookahead at eval would require
a forward model of market dynamics — not available, not standard.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-26 23:41:05 +02:00
parent 5c325f8947
commit 1c63c32297

View File

@@ -3,7 +3,10 @@
**Date**: 2026-04-26
**Status**: Design approved, awaiting implementation plans (4 sub-plans)
**Origin**: Task #178 — Design proper C51 expected-Q Hold/Flat bias mitigation
**Predecessor symptom-layer fixes (revisited in Phase 3)**: Kelly cap warm-branch (`0c9d1ee39`), eps-floor adaptive boost (`d54b49efc`), ISV-adaptive Boltzmann tau (`7a3d88646`)
**Predecessor fixes from val-Flat-collapse investigation (all KEPT — real fixes for real bugs)**:
- Kelly cap warm-branch (`0c9d1ee39`) — post-decision physics layer
- Train Return display + Sharpe annualization (non-tau parts of `7a3d88646`) — display/metric layer
- Direction-branch Boltzmann tau-floor (tau part of `7a3d88646`) and adaptive eps_dir floor (`d54b49efc`) — gates inside direction-branch action selection. Phase 2 replaces direction-branch action selection wholesale (Thompson replaces eps-greedy + Boltzmann), so these gates become dead code FOR THE DIRECTION BRANCH ONLY. Magnitude/order/urgency keep their existing eps-greedy + Boltzmann + tau-floor + EPS_FLOOR paths intact. Phase 3 cleanup deletes only the direction-branch unreachable code, preserving all logic for other branches.
---
@@ -297,19 +300,22 @@ Pearl entry to memory: `pearl_thompson_for_distributional_action_selection.md`.
**Time budget**: 2-3 days.
### Phase 3 — Verification + Band-Aid Removal
### Phase 3 — Verification + Direction-Branch Dead-Code Cleanup
**Deliverables**:
- L4 multi-fold deploy (1 seed × 6 folds × 30 epoch) — long enough to see edge discovery via Thompson exploration
- L5 full validation matrix (per Plan 5 Task 5): Tier 1/2/3 PASS
- **Band-aid audit + removal**: explicit task to revisit:
- eps-floor adaptive boost (`d54b49efc`): if Thompson generates sufficient directional exploration, REMOVE
- Boltzmann tau-floor on direction (`7a3d88646`): direction branch no longer uses Boltzmann (Thompson replaces it), so this code path becomes dead — REMOVE
- Kelly cap fix (`0c9d1ee39`): KEEP — addresses different layer (post-decision physics), still needed
- **Direction-branch dead-code cleanup** (NOT band-aid removal — these are real fixes whose direction-branch code paths simply become unreachable after Phase 2):
- **Kelly cap warm-branch fix (`0c9d1ee39`)**: KEEP. Post-decision physics layer; Thompson-independent. Real fix for a real bug; would break Long/Short trades again if reverted.
- **Train Return display + Sharpe annualization fixes (non-tau parts of `7a3d88646`)**: KEEP. Display/metric layer; Thompson-independent. Real fixes for real bugs.
- **Direction-branch Boltzmann tau-floor (tau-floor part of `7a3d88646`)**: code path becomes structurally unreachable for direction once Thompson replaces direction Boltzmann. The tau-floor remains active for magnitude/order/urgency branches (which keep Boltzmann). Cleanup pass: delete direction-only tau-floor code; keep mag/ord/urg paths intact. Not a band-aid removal — dead-code elimination per `feedback_no_legacy_aliases.md` and `feedback_no_partial_refactor.md`.
- **Adaptive eps_dir floor (`d54b49efc`)**: same — gates the random-sample path of direction-branch eps-greedy. Direction branch no longer uses eps-greedy under Thompson; this code becomes structurally unreachable for direction. Static `EPS_FLOOR=0.02` for magnitude/order/urgency branches stays. Cleanup pass: remove direction-only adaptive boost; keep the static EPS_FLOOR for other branches.
- nsys regression check (Phase H continues independently; Thompson overhead measured)
- Final architecture doc updates
**Time budget**: per Plan 5 existing budget + 1 day for band-aid removal.
**Why "cleanup" not "removal"**: every band-aid fix shipped during the val-Flat-collapse investigation addressed a real bug at its layer and should be preserved. Phase 2 replaces the direction-branch action-selection mechanism wholesale (eps-greedy + Boltzmann → Thompson). The eps_dir adaptive floor and the direction Boltzmann tau-floor are both inside the direction-branch action-selection path that Thompson replaces. Their direction-only code becomes dead — not because they were unnecessary, but because the larger surrounding mechanism changed shape. Dead code that runs but does nothing is a maintenance hazard (same anti-pattern as backward-compatibility shims for removed features). Phase 3 cleans it up.
**Time budget**: per Plan 5 existing budget + 0.5 day for dead-code cleanup.
---
@@ -419,7 +425,7 @@ The single 5-epoch L40S smoke is INSUFFICIENT to verify Thompson — long-term e
| Plan A | Phase 0: TDD hypothesis + synthetic edge verification | 2 days |
| Plan B | Phase 1: existing-lever audit | 1 day (longer if bug) |
| Plan C | Phase 2: Thompson sampling integration | 2-3 days |
| Plan D | Phase 3: long verification + band-aid removal | per Plan 5 + 1 day |
| Plan D | Phase 3: long verification + direction-branch dead-code cleanup | per Plan 5 + 0.5 day |
Each plan gets its own writing-plans cycle.