docs(sp11): close-out — pearl_symmetric_clamp_audit + audit doc
Documents the SP11 sharpe-degradation root cause investigation: -35db31089: symmetric reward cap (THE bug) -348f6078b: plan_isv symmetric clamp mirrors (4 sites) -b92dcc3df: diagnostic instrumentation cleanup - this commit: close-out New memory pearl: pearl_symmetric_clamp_audit Distinct from pearl_bounded_modifier_outputs_require_structural_activation: - That pearl: model OUTPUTS need structural activation (sigmoid/tanh) - This pearl: intermediate kernel scalars need bilateral fminf/fmaxf Both share: bounded contracts must be enforced structurally. Audit doc: cross-references bug, fix, validation smoke, both pearls. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -5600,3 +5600,117 @@ This is the policy-input mirror of the reward-side bug fixed in
|
||||
`35db31089`. Both share the same architectural lesson — see
|
||||
`pearl_symmetric_clamp_audit` (added in the SP11 close-out commit)
|
||||
and `pearl_bounded_modifier_outputs_require_structural_activation`.
|
||||
|
||||
## SP11 B1b sharpe-degradation close-out (2026-05-04)
|
||||
|
||||
The within-fold sharpe degradation flagged in the B1b smoke-recovery
|
||||
"Open follow-up" above is now resolved. Root cause + fix chain
|
||||
documented for future audit.
|
||||
|
||||
### Root cause: asymmetric reward cap
|
||||
|
||||
`experience_kernels.cu:2788` (pre-fix):
|
||||
|
||||
```cuda
|
||||
float capped_pnl = fminf(base_reward, 10.0f);
|
||||
```
|
||||
|
||||
`base_reward` was P&L-derived and could be negative on losing
|
||||
trades; the upper-only cap left the loss tail unbounded. Diagnostic
|
||||
instrumentation (commit `774d7552a`, since removed) found the
|
||||
post-cap reward reaching **-210336** by F0 ep3 — five orders of
|
||||
magnitude past the spec bound on the loss side.
|
||||
|
||||
The unbounded loss tail propagated through:
|
||||
|
||||
1. **PopArt running magnitude EMA (slot 63)** inflated to track
|
||||
extreme magnitudes, breaking C51/IQN normalisation across
|
||||
epochs. HEALTH_DIAG `popart` jumping from O(10) to O(10⁵) within
|
||||
one epoch was the smoking gun.
|
||||
2. **C51 atom-span (per-branch v_range)** widened to cover the
|
||||
pollution, then never recovered, holding subsequent epochs at
|
||||
miscalibrated atom resolution.
|
||||
3. **Loss-balance controller (SP7)** read inflated reward
|
||||
magnitudes, mis-allocating per-loss-term budgets.
|
||||
4. **Within-fold sharpe degraded** from ~6 (clean) to ~4.26 across
|
||||
the fold — the residual flagged in the B1b close-out.
|
||||
|
||||
### Fix: symmetric reward cap
|
||||
|
||||
Commit `35db31089` at `experience_kernels.cu:2788`:
|
||||
|
||||
```cuda
|
||||
float capped_pnl = fmaxf(-10.0f, fminf(base_reward, 10.0f));
|
||||
```
|
||||
|
||||
Bilateral clamp at the canonical reward-emission site. Spec bound
|
||||
`reward_per_bar ∈ [-10, 10]` is now enforced structurally on both
|
||||
sides; downstream PopArt/C51/IQN normalisers receive a guaranteed
|
||||
bounded signal.
|
||||
|
||||
### Validation: smoke-test-trk72
|
||||
|
||||
L40S smoke at commit `35db31089` (5-epoch × 3-fold):
|
||||
|
||||
| Metric | Pre-fix (degraded) | Post-fix (smoke-test-trk72) | Delta |
|
||||
|---|---|---|---|
|
||||
| F0 sharpe | 4.26 | 6.13 | **+44%** |
|
||||
| `popart` magnitude | unbounded (peaks 10⁵+) | bounded ±10 throughout | clean |
|
||||
| `curiosity_b` | unstable, drift-prone | stable 0.18-0.31 across folds | clean |
|
||||
| Cross-fold convergence | F0 sharpe degraded mid-fold | All 3 folds converge cleanly | PASSED |
|
||||
|
||||
The smoke validated the fix end-to-end: PopArt slot 63 bounded,
|
||||
C51/IQN normalisation stable, sharpe recovered.
|
||||
|
||||
### Mirror fix: plan_isv 4 sites (commit `348f6078b`)
|
||||
|
||||
The implementer of `35db31089` flagged 4 additional asymmetric-clamp
|
||||
sites in `plan_isv` policy-input features (table above). These are
|
||||
the feature-side mirror of the reward-side bug — same shape, same
|
||||
fix recipe, same architectural lesson.
|
||||
|
||||
### Diagnostic cleanup (commit `b92dcc3df`)
|
||||
|
||||
Diagnostic instrumentation from `774d7552a` served its purpose
|
||||
(empirically identified the asymmetric cap as the inflater) and was
|
||||
removed:
|
||||
|
||||
- `reward_chain_diag_reduce_kernel.cu` (file deletion)
|
||||
- 12 per-sample diagnostic buffers in `gpu_experience_collector`
|
||||
- Kernel parameter threading in `experience_kernels.cu`
|
||||
- Launcher + reader + mapped-pinned output in `gpu_dqn_trainer`
|
||||
- Wire-up site + HEALTH_DIAG emit in `training_loop`
|
||||
|
||||
The worktree returned to its pre-instrumentation state on the SP11
|
||||
reward chain.
|
||||
|
||||
### Cross-references — both pearls
|
||||
|
||||
The SP11 sharpe-degradation investigation produced two
|
||||
complementary memory pearls:
|
||||
|
||||
| Pearl | Domain | Fix recipe |
|
||||
|-------|--------|-----------|
|
||||
| `pearl_bounded_modifier_outputs_require_structural_activation` | Model OUTPUTS — learned scalars feeding multiplicative modifiers | Structural activation (sigmoid/tanh/softplus) on the architecture |
|
||||
| `pearl_symmetric_clamp_audit` | Kernel-derived intermediate scalars (rewards, ISV features) | Bilateral runtime clamp `fmaxf(lo, fminf(x, hi))` |
|
||||
|
||||
Both share the same architectural principle: **bounded contracts at
|
||||
architectural boundaries must be enforced structurally, not by
|
||||
hopes about input distribution**. They are siblings in the same
|
||||
class — only the implementation site differs (model architecture
|
||||
vs. kernel arithmetic).
|
||||
|
||||
### Commit chain summary
|
||||
|
||||
| Commit | Description |
|
||||
|--------|-------------|
|
||||
| `774d7552a` | diag: reward-chain instrumentation (transient) |
|
||||
| `35db31089` | fix: symmetric reward cap (THE bug) |
|
||||
| `348f6078b` | fix: plan_isv 4-site symmetric clamp mirrors |
|
||||
| `b92dcc3df` | chore: remove reward-chain instrumentation |
|
||||
| (this commit) | docs: pearl + audit close-out |
|
||||
|
||||
The B1b sharpe-degradation residual is **resolved**. The B1b
|
||||
controller (z-score normalised mag-ratio canary) and the SP11
|
||||
reward cap fix together close the within-fold degradation root
|
||||
cause.
|
||||
|
||||
Reference in New Issue
Block a user