spec(dqn): reframe Phase 3 as direction-branch dead-code cleanup
User correctly challenged the "band-aid removal" framing. All fixes shipped during the val-Flat-collapse investigation addressed real bugs at their respective layers and should be preserved: - Kelly cap warm-branch (0c9d1ee39): post-decision physics layer. Thompson-independent. KEEP. - Train Return display + Sharpe annualization (non-tau parts of7a3d88646): display/metric layer. Thompson-independent. KEEP. - Direction Boltzmann tau-floor (tau part of7a3d88646) + adaptive eps_dir floor (d54b49efc): gates inside direction-branch action selection. Phase 2 replaces direction-branch action selection wholesale (eps-greedy + Boltzmann → Thompson), so these direction-only code paths become structurally unreachable. The latter two are NOT band-aids being removed because Thompson is better. They are dead code being cleaned up because Thompson replaces the surrounding mechanism. Magnitude/order/urgency branches keep their existing eps-greedy + Boltzmann + tau-floor + EPS_FLOOR paths intact. Reframed Phase 3 deliverable: "direction-branch dead-code cleanup" with explicit rationale (per feedback_no_legacy_aliases.md and feedback_no_partial_refactor.md). 0.5 day budget instead of 1. Also clarified eval action selection: argmax of (E[Q_C51]+E[Q_IQN])/2 is correct. Bellman backup is a Q-learning UPDATE rule, not an action-selection rule. Once Q is learned, optimal policy is greedy argmax of learned Q. Online Bellman lookahead at eval would require a forward model of market dynamics — not available, not standard. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -3,7 +3,10 @@
|
||||
**Date**: 2026-04-26
|
||||
**Status**: Design approved, awaiting implementation plans (4 sub-plans)
|
||||
**Origin**: Task #178 — Design proper C51 expected-Q Hold/Flat bias mitigation
|
||||
**Predecessor symptom-layer fixes (revisited in Phase 3)**: Kelly cap warm-branch (`0c9d1ee39`), eps-floor adaptive boost (`d54b49efc`), ISV-adaptive Boltzmann tau (`7a3d88646`)
|
||||
**Predecessor fixes from val-Flat-collapse investigation (all KEPT — real fixes for real bugs)**:
|
||||
- Kelly cap warm-branch (`0c9d1ee39`) — post-decision physics layer
|
||||
- Train Return display + Sharpe annualization (non-tau parts of `7a3d88646`) — display/metric layer
|
||||
- Direction-branch Boltzmann tau-floor (tau part of `7a3d88646`) and adaptive eps_dir floor (`d54b49efc`) — gates inside direction-branch action selection. Phase 2 replaces direction-branch action selection wholesale (Thompson replaces eps-greedy + Boltzmann), so these gates become dead code FOR THE DIRECTION BRANCH ONLY. Magnitude/order/urgency keep their existing eps-greedy + Boltzmann + tau-floor + EPS_FLOOR paths intact. Phase 3 cleanup deletes only the direction-branch unreachable code, preserving all logic for other branches.
|
||||
|
||||
---
|
||||
|
||||
@@ -297,19 +300,22 @@ Pearl entry to memory: `pearl_thompson_for_distributional_action_selection.md`.
|
||||
|
||||
**Time budget**: 2-3 days.
|
||||
|
||||
### Phase 3 — Verification + Band-Aid Removal
|
||||
### Phase 3 — Verification + Direction-Branch Dead-Code Cleanup
|
||||
|
||||
**Deliverables**:
|
||||
- L4 multi-fold deploy (1 seed × 6 folds × 30 epoch) — long enough to see edge discovery via Thompson exploration
|
||||
- L5 full validation matrix (per Plan 5 Task 5): Tier 1/2/3 PASS
|
||||
- **Band-aid audit + removal**: explicit task to revisit:
|
||||
- eps-floor adaptive boost (`d54b49efc`): if Thompson generates sufficient directional exploration, REMOVE
|
||||
- Boltzmann tau-floor on direction (`7a3d88646`): direction branch no longer uses Boltzmann (Thompson replaces it), so this code path becomes dead — REMOVE
|
||||
- Kelly cap fix (`0c9d1ee39`): KEEP — addresses different layer (post-decision physics), still needed
|
||||
- **Direction-branch dead-code cleanup** (NOT band-aid removal — these are real fixes whose direction-branch code paths simply become unreachable after Phase 2):
|
||||
- **Kelly cap warm-branch fix (`0c9d1ee39`)**: KEEP. Post-decision physics layer; Thompson-independent. Real fix for a real bug; would break Long/Short trades again if reverted.
|
||||
- **Train Return display + Sharpe annualization fixes (non-tau parts of `7a3d88646`)**: KEEP. Display/metric layer; Thompson-independent. Real fixes for real bugs.
|
||||
- **Direction-branch Boltzmann tau-floor (tau-floor part of `7a3d88646`)**: code path becomes structurally unreachable for direction once Thompson replaces direction Boltzmann. The tau-floor remains active for magnitude/order/urgency branches (which keep Boltzmann). Cleanup pass: delete direction-only tau-floor code; keep mag/ord/urg paths intact. Not a band-aid removal — dead-code elimination per `feedback_no_legacy_aliases.md` and `feedback_no_partial_refactor.md`.
|
||||
- **Adaptive eps_dir floor (`d54b49efc`)**: same — gates the random-sample path of direction-branch eps-greedy. Direction branch no longer uses eps-greedy under Thompson; this code becomes structurally unreachable for direction. Static `EPS_FLOOR=0.02` for magnitude/order/urgency branches stays. Cleanup pass: remove direction-only adaptive boost; keep the static EPS_FLOOR for other branches.
|
||||
- nsys regression check (Phase H continues independently; Thompson overhead measured)
|
||||
- Final architecture doc updates
|
||||
|
||||
**Time budget**: per Plan 5 existing budget + 1 day for band-aid removal.
|
||||
**Why "cleanup" not "removal"**: every band-aid fix shipped during the val-Flat-collapse investigation addressed a real bug at its layer and should be preserved. Phase 2 replaces the direction-branch action-selection mechanism wholesale (eps-greedy + Boltzmann → Thompson). The eps_dir adaptive floor and the direction Boltzmann tau-floor are both inside the direction-branch action-selection path that Thompson replaces. Their direction-only code becomes dead — not because they were unnecessary, but because the larger surrounding mechanism changed shape. Dead code that runs but does nothing is a maintenance hazard (same anti-pattern as backward-compatibility shims for removed features). Phase 3 cleans it up.
|
||||
|
||||
**Time budget**: per Plan 5 existing budget + 0.5 day for dead-code cleanup.
|
||||
|
||||
---
|
||||
|
||||
@@ -419,7 +425,7 @@ The single 5-epoch L40S smoke is INSUFFICIENT to verify Thompson — long-term e
|
||||
| Plan A | Phase 0: TDD hypothesis + synthetic edge verification | 2 days |
|
||||
| Plan B | Phase 1: existing-lever audit | 1 day (longer if bug) |
|
||||
| Plan C | Phase 2: Thompson sampling integration | 2-3 days |
|
||||
| Plan D | Phase 3: long verification + band-aid removal | per Plan 5 + 1 day |
|
||||
| Plan D | Phase 3: long verification + direction-branch dead-code cleanup | per Plan 5 + 0.5 day |
|
||||
|
||||
Each plan gets its own writing-plans cycle.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user