From d1068de2b89350e75f4ca37b8038c9a1f47498cc Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Wed, 22 Apr 2026 11:18:28 +0200 Subject: [PATCH] plan(policy-quality): reframe DELETE tasks per no-functionality-removal rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit User directive: "we don't remove functionality at all, we fix what's broken". Standing rule captured in ~/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_no_functionality_removal.md. Phase 2 plan reframed: * Task 2.3 (R5 micro-reward): was DELETE; now FIX — either (a) set a non-zero scale that delivers real signal, or (b) document why it's intentionally disabled while preserving the code path. * Task 2.4 (R6 loss-aversion): unchanged — the relocation to C51 Bellman target is consolidation, not removal. Invariant preserved. * Task 2.6 (E4 entropy-reg): was DELETE-CANDIDATE; now TUNE — if the signal doesn't reach magnitude, re-route it rather than remove. * Task 2.7 (C4 grad-clip): was DELETE-CANDIDATE; now KEEP-AND-DOCUMENT — gradient clipping is a safety feature; run the ablation for diagnostic, not for removal. * H9 delete-magnitude-branch fallback: rejected permanently. On magnitude convergence failure the fallback is per-magnitude reward shaping / per-bin advantage weighting / curriculum / state enrichment. Task 2.5 wiring-bug sweep unchanged (correctness fixes, not removal). This commit updates only the planning narrative. Task execution hasn't started on 2.3/2.6/2.7 yet, so no code changes required here. When those tasks run, they will implement the fix-paths, not the delete-paths. --- .../plans/2026-04-21-policy-quality-phase2.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/plans/2026-04-21-policy-quality-phase2.md b/docs/superpowers/plans/2026-04-21-policy-quality-phase2.md index 41e726439..146878698 100644 --- a/docs/superpowers/plans/2026-04-21-policy-quality-phase2.md +++ b/docs/superpowers/plans/2026-04-21-policy-quality-phase2.md @@ -78,10 +78,22 @@ The Track 1 triage scored H4 CONFIRMED based on `grad_ratio_mag_dir = 0.0000 acr - **Task 2.1 is DEFERRED / likely unnecessary.** Magnitude doesn't need architectural unblocking. Branch A (Xavier rescale for magnitude head) would make over-feeding worse; Branch B (per-component weight rebalance) targets a non-existent starvation; Branch C (`w_b1fc` direction-conditioning rewrite) is a structural change the data doesn't justify. - **Task 2.2 (H10 argmax tie-break) is PROMOTED TO PRIMARY FIX.** The observable symptom (eval `eq=1.000 eh=0.000 ef=0.000` despite training `ent_mag ≈ 0.99`) IS argmax tie-breaking. Fix the tie-break and the smoke test likely passes. -- **New fallback if Task 2.2 alone is insufficient**: promote H9's spec §5.1 proposed fix to a new Task — **delete the magnitude branch entirely** (108 → 36 action space: direction × order × urgency only). The data says magnitude doesn't differentiate trades; removing it is a strict simplification with no expected loss. -- **Task 2.3 / 2.4 / 2.5 / 2.6 / 2.7 are unchanged.** The Task 2.0 data also confirms R5 TD-propagation is fine (C51+CQL reach magnitude robustly), so R5 DELETE (Task 2.3) is unblocked. +- **Fallback if Task 2.2 alone is insufficient — make magnitude differentiate, do NOT delete.** Per standing rule `~/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_magnitude_must_be_useful.md`: the magnitude branch is not deletable. If eval `eh + ef` stays below 0.30 after Task 2.2, the follow-up work is to **force magnitude to matter** via one or more of: + - Per-magnitude reward shaping — amplify step-return signal per bin (e.g., scale by per-magnitude realized-return variance observed in forced-exploration) + - Per-bin advantage weighting in the branch loss — deliberately bias gradient so Half / Full bins see more training signal per trade + - Magnitude curriculum — force ε-exploration to visit Half / Full for a warm-up window before argmax takes over + - State-vector enrichment — add regime-differentiation signals that make Quarter vs Half vs Full learnable (Phase 2 implementation extends the feature vector) + - Q-regularization preventing the `Q_full ≈ Q_half ≈ Q_quarter` degenerate fixpoint + H9's original spec §5.1 "delete the magnitude branch" proposal is **rejected as a non-starter** — the project relies on position sizing as a real lever; simplifying it away permanently forecloses later gains when signals become learnable. +- **Task 2.3 / 2.6 / 2.7 DELETE verdicts are REJECTED per the no-functionality-removal rule** (`~/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_no_functionality_removal.md`). Reframed: + - **Task 2.3 R5 micro-reward**: was planned as DELETE because `reward_contrib[3] = 0.000` and `micro_reward_scale = 0` in smoke. Reframed to **FIX R5**: either (a) set a non-zero scale that delivers real signal, or (b) document why it's intentionally disabled while preserving the code path for future activation. Do not delete the kernel code or buffer infrastructure. + - **Task 2.4 R6 loss-aversion**: remains as planned. The plan's "relocate negative-tail compression to C51 Bellman target" is **consolidation**, not removal — same invariant, better location. Keep the upper `+10` cap inline. Functionality is preserved; only the wiring point moves. + - **Task 2.6 E4 entropy-regularization**: DELETE-CANDIDATE reframed to **TUNE**. If Task 2.0 data shows C51 entropy doesn't reach magnitude, the fix is to re-route the entropy signal so it does (increase coefficient, redirect through a term that writes to the magnitude branch) rather than remove. If it's truly redundant with NoisyNets + count bonus, document the redundancy rationale while keeping the mechanism wired at a smaller coefficient as a safety net. + - **Task 2.7 C4 adaptive grad-clip**: DELETE-CANDIDATE reframed to **KEEP-AND-DOCUMENT**. Gradient clipping is a safety feature; even at 20% fire rate, the epochs it fires on are protecting against unbounded gradient norms. Rather than ablate-then-delete, run the ablation to understand *what* it's clipping (diagnostic), then tune the threshold if over-firing or document the load-bearing invariant if genuinely needed. +- **Task 2.5 wiring-bug sweep** is unchanged — those are correctness fixes, not functionality removal. +- **Task 2.0 data also confirms R5 TD-propagation is fine** (C51+CQL reach magnitude robustly), so R5 remediation (Task 2.3, reframed) is unblocked. -Net effect: Phase 2 gets a big simplification. The hardest task (2.1 with three architectural branches) is dropped. Primary fix becomes Task 2.2 (~25 LOC), with H9-delete-magnitude-branch as fallback. +Net effect: Phase 2 gets a simplification in its happy path — the hardest task (2.1 with three architectural branches) is dropped as not matching the real diagnostic data. Primary fix becomes Task 2.2 (~25 LOC). The fallback is a new magnitude-differentiation task (per-magnitude reward shaping / per-bin advantage weighting / curriculum / state enrichment — not yet scoped in detail; planned only if Task 2.2 alone is insufficient). **No DELETE paths in any fallback.** ---