Files
foxhunt/docs/superpowers
jgrusewski f9286e938d docs(policy-quality): Task 2.X scoping — per-bin variance + Q-spread regularization
Scoping output from the Full=0% investigation (post-Task-2.2 smoke showed
eval_dist [eq=0.580 eh=0.420 ef=0.000] — Quarter+Half recovered,
Full still 0%).

Root cause: Q-estimation bias (category C) — C51 expected-Q over atoms
systematically under-prices higher-variance bins. Full has 4x Quarter's
PnL variance per experience_kernels.cu risk scaling. Bias is already
documented at experience_kernels.cu:904 (commented as "structural
distributional bias — tight return distributions (Small) get higher
expected Q under the C51 softmax regardless of actual expected returns").

Not noise-starvation: Full picked on 32% of training samples across
60 epochs × 3 folds = ~61k Full samples. Ample data.

Proposed Task 2.X = Rank 1 + Rank 2 combined (~90 LOC):
  Rank 1: per-bin variance weighting in C51 branch loss — amplify
          Full's gradient by sqrt(var_f / mean_var)
  Rank 2: Q-spread regularization — lambda * ReLU(target_spread -
          Q_spread)^2 penalty preventing degenerate fixpoint

Alternative fixes ranked but not recommended:
  Rank 3 per-magnitude reward shaping (~80 LOC) — only if data-disfavor
         is confirmed at L40S
  Rank 4 magnitude curriculum (~20 LOC) — rejected (noise-starvation ruled out)
  Rank 5 state-vector enrichment (~150 LOC) — premature

Prerequisite: ~40 LOC of per-magnitude win-rate + realized-variance
instrumentation to confirm category C vs A at L40S scale.

L40S gating recommendation was:
  ef >= 0.15 at L40S → Task 2.X not needed (smoke artefact)
  0.05 <= ef < 0.15 → Task 2.X scoped but optional
  ef < 0.05 → Task 2.X required

Per feedback_fix_aggressively.md (landed this session): we ship Task 2.X
ahead of L40S since the bias is already code-documented, the fix is
well-scoped, and the 90-LOC cost is reasonable. L40S validation (Task
2.8) will verify the fixed state end-to-end instead of the un-fixed one.

No deletions anywhere per feedback_no_functionality_removal.md.
2026-04-22 11:43:17 +02:00
..