docs(sp11): close-out — pearl_symmetric_clamp_audit + audit doc

Documents the SP11 sharpe-degradation root cause investigation:
  - 35db31089: symmetric reward cap (THE bug)
  - 348f6078b: plan_isv symmetric clamp mirrors (4 sites)
  - b92dcc3df: diagnostic instrumentation cleanup
  - this commit: close-out

New memory pearl: pearl_symmetric_clamp_audit
  Distinct from pearl_bounded_modifier_outputs_require_structural_activation:
  - That pearl: model OUTPUTS need structural activation (sigmoid/tanh)
  - This pearl: intermediate kernel scalars need bilateral fminf/fmaxf
  Both share: bounded contracts must be enforced structurally.

Audit doc: cross-references bug, fix, validation smoke, both pearls.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-04 15:20:26 +02:00
parent b92dcc3dfc
commit 6a259942e9

View File

@@ -5600,3 +5600,117 @@ This is the policy-input mirror of the reward-side bug fixed in
`35db31089`. Both share the same architectural lesson — see
`pearl_symmetric_clamp_audit` (added in the SP11 close-out commit)
and `pearl_bounded_modifier_outputs_require_structural_activation`.
## SP11 B1b sharpe-degradation close-out (2026-05-04)
The within-fold sharpe degradation flagged in the B1b smoke-recovery
"Open follow-up" above is now resolved. Root cause + fix chain
documented for future audit.
### Root cause: asymmetric reward cap
`experience_kernels.cu:2788` (pre-fix):
```cuda
float capped_pnl = fminf(base_reward, 10.0f);
```
`base_reward` was P&L-derived and could be negative on losing
trades; the upper-only cap left the loss tail unbounded. Diagnostic
instrumentation (commit `774d7552a`, since removed) found the
post-cap reward reaching **-210336** by F0 ep3 — five orders of
magnitude past the spec bound on the loss side.
The unbounded loss tail propagated through:
1. **PopArt running magnitude EMA (slot 63)** inflated to track
extreme magnitudes, breaking C51/IQN normalisation across
epochs. HEALTH_DIAG `popart` jumping from O(10) to O(10⁵) within
one epoch was the smoking gun.
2. **C51 atom-span (per-branch v_range)** widened to cover the
pollution, then never recovered, holding subsequent epochs at
miscalibrated atom resolution.
3. **Loss-balance controller (SP7)** read inflated reward
magnitudes, mis-allocating per-loss-term budgets.
4. **Within-fold sharpe degraded** from ~6 (clean) to ~4.26 across
the fold — the residual flagged in the B1b close-out.
### Fix: symmetric reward cap
Commit `35db31089` at `experience_kernels.cu:2788`:
```cuda
float capped_pnl = fmaxf(-10.0f, fminf(base_reward, 10.0f));
```
Bilateral clamp at the canonical reward-emission site. Spec bound
`reward_per_bar ∈ [-10, 10]` is now enforced structurally on both
sides; downstream PopArt/C51/IQN normalisers receive a guaranteed
bounded signal.
### Validation: smoke-test-trk72
L40S smoke at commit `35db31089` (5-epoch × 3-fold):
| Metric | Pre-fix (degraded) | Post-fix (smoke-test-trk72) | Delta |
|---|---|---|---|
| F0 sharpe | 4.26 | 6.13 | **+44%** |
| `popart` magnitude | unbounded (peaks 10⁵+) | bounded ±10 throughout | clean |
| `curiosity_b` | unstable, drift-prone | stable 0.18-0.31 across folds | clean |
| Cross-fold convergence | F0 sharpe degraded mid-fold | All 3 folds converge cleanly | PASSED |
The smoke validated the fix end-to-end: PopArt slot 63 bounded,
C51/IQN normalisation stable, sharpe recovered.
### Mirror fix: plan_isv 4 sites (commit `348f6078b`)
The implementer of `35db31089` flagged 4 additional asymmetric-clamp
sites in `plan_isv` policy-input features (table above). These are
the feature-side mirror of the reward-side bug — same shape, same
fix recipe, same architectural lesson.
### Diagnostic cleanup (commit `b92dcc3df`)
Diagnostic instrumentation from `774d7552a` served its purpose
(empirically identified the asymmetric cap as the inflater) and was
removed:
- `reward_chain_diag_reduce_kernel.cu` (file deletion)
- 12 per-sample diagnostic buffers in `gpu_experience_collector`
- Kernel parameter threading in `experience_kernels.cu`
- Launcher + reader + mapped-pinned output in `gpu_dqn_trainer`
- Wire-up site + HEALTH_DIAG emit in `training_loop`
The worktree returned to its pre-instrumentation state on the SP11
reward chain.
### Cross-references — both pearls
The SP11 sharpe-degradation investigation produced two
complementary memory pearls:
| Pearl | Domain | Fix recipe |
|-------|--------|-----------|
| `pearl_bounded_modifier_outputs_require_structural_activation` | Model OUTPUTS — learned scalars feeding multiplicative modifiers | Structural activation (sigmoid/tanh/softplus) on the architecture |
| `pearl_symmetric_clamp_audit` | Kernel-derived intermediate scalars (rewards, ISV features) | Bilateral runtime clamp `fmaxf(lo, fminf(x, hi))` |
Both share the same architectural principle: **bounded contracts at
architectural boundaries must be enforced structurally, not by
hopes about input distribution**. They are siblings in the same
class — only the implementation site differs (model architecture
vs. kernel arithmetic).
### Commit chain summary
| Commit | Description |
|--------|-------------|
| `774d7552a` | diag: reward-chain instrumentation (transient) |
| `35db31089` | fix: symmetric reward cap (THE bug) |
| `348f6078b` | fix: plan_isv 4-site symmetric clamp mirrors |
| `b92dcc3df` | chore: remove reward-chain instrumentation |
| (this commit) | docs: pearl + audit close-out |
The B1b sharpe-degradation residual is **resolved**. The B1b
controller (z-score normalised mag-ratio canary) and the SP11
reward cap fix together close the within-fold degradation root
cause.