gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs(sp4): signal-driven magnitude control design spec

Browse Source 
Comprehensive design replacing every hardcoded magnitude multiplier in
the SP3 mechanism stack (Mechs 1, 2, 5, 6, 9, 10) plus pre-SP3 mechs in
the same magnitude-control surface, per feedback_isv_for_adaptive_bounds
and feedback_adaptive_not_tuned.

Core principle: the BOUND lives in an ISV slot, computed by a producer
kernel as p99 EMA of observed signal magnitude. Consumer reads the slot
and clamps directly — no multiplier between ISV read and clamp. Cold-
start ε from theoretical-init bootstrap (Xavier, etc.) — same theoretical-
constant category as Adam β values, not tuning knobs.

Architecture:
- 28 new ISV slots (7 base bounds × per-param-group split where appropriate)
- Per-signal P² (Jain-Chlamtac) quantile producer kernels
- Diagnostic = clamp engagement (sticky flag from producer's max-comparison)
- Migration in 3 layers: additive infra → atomic consumer flip → smoke

Out of scope: theoretical/structural constants (Adam β, Xavier formula,
attention 1/√d, hidden_dim, num_atoms). EMA rates stay as documented
statistical-design parameters (half-life ≈ observation time-window).

Estimated: ~2-3000 LOC across 3 layers, 1-2 weeks, 1 L40S smoke.

Awaiting user spec review before writing implementation plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-30 20:49:09 +02:00
parent 6e71576537
commit e00736ce9b
1 changed files with 236 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

236
docs/superpowers/specs/2026-04-30-sp4-signal-driven-magnitude-control-design.md Normal file
View File

@@ -0,0 +1,236 @@
# SP4 — Signal-Driven Magnitude Control Design
**Date:** 2026-04-30
**Branch:** `plan-c-phase-2-thompson` (post-SP3 close-out)
**Predecessor:** SP3 close-out v2 (Mech 9 across 5 Adam kernels) + Mech 10 (h_s2 clamp)
**Status:** Design — awaiting user spec review before implementation plan.
## Goal
Replace every hardcoded magnitude multiplier in the SP3 mechanism stack (and pre-SP3 mechs in the same magnitude-control surface) with **signal-driven ISV bounds**. The bound itself lives in an ISV slot computed by a producer kernel from observed signal statistics. Consumers read the slot and clamp directly — no multiplier between ISV read and clamp.
This closes the principle violation in `feedback_isv_for_adaptive_bounds.md` and `feedback_adaptive_not_tuned.md` that accumulated across Mechs 1, 2, 5, 6, 9, 10. Every clamp/diagnostic in this category becomes a "p99 of recent observations" EMA — outliers above the typical-99% magnitude get clipped to the typical-99% magnitude.
This is also the long-term answer to the F0=21.09 plateau observed in `smoke-test-jxckl` (Mech 10 at 100×): Mech 10's hardcoded multiplier was binding heavy-tail h_s2 elements during F0 training. Under SP4, the bound becomes p99(|h_s2|) — outliers above the observed typical maximum get clipped, but the typical distribution flows through unaltered.
## Scope
**In scope (category B per brainstorm):** every hardcoded magnitude multiplier and every hardcoded ε floor that controls a clamp, diagnostic threshold, gradient clip, weight decay, or L1 lambda across:
- **Mech 1** — target_q clamp (currently `10 × Q_ABS_REF.max(1.0)`)
- **Mech 2** — atom positions (3 sites: `atoms_update_kernel`, `iql_value_kernel`, `experience_kernels`; currently `10 × Q_ABS_REF.max(1.0)`)
- **Mech 5** — diagnostic thresholds (slots 36-49: Adam m at 100×, Adam v at 1e6× squared, weights at 1e3×, target_q post-clip at 95×, atom span at 190×, IQN weight at 1e3×, h_s2 max at 1e3×)
- **Mech 6** — adaptive grad-clip upper bound (`100 × Q_ABS_REF.max(1.0) × prev_slow_ema`)
- **Mech 9** — post-Adam weight clamp across 5 Adam kernels (`100 × Q_ABS_REF.max(1.0)`)
- **Mech 10** — h_s2 activation clamp (`100 × H_S2_RMS_EMA.max(1.0)`)
- **EMA rates** (α values in `h_s2_rms_ema_update`, `grad_norm_slow_ema`, `moe_gate_entropy`, `vsn_mask_ema`, `aux_heads_loss_ema`, `iqn_quantile_ema`) — kept as **statistical-design parameters** (each documented as "observation time-window in epochs/folds"); not a multiplier on a magnitude.
- **ε floors** (the `.max(1.0)` SP1 pearl, `MIN_CLIP=1.0`, MoE λ floor) — replaced by per-slot **theoretical-init bootstrap values** (Xavier-derived, etc.).
- **Gradient clip values** (`CLIP_MULTIPLIER`, `MAX_GRAD_NORM` static floors)
- **Weight decay constant** (AdamW `weight_decay` field)
- **L1 lambda** (`l1_lambda = 1e-3` in `dqn_adam_update_kernel`)
**Out of scope (categories D + E):** mathematically derived theoretical constants (Adam β₁=0.9, β₂=0.999; Adam ε=1e-8 numerical denominator; Xavier init formula `√(2/K)`; attention 1/√d), and architectural shape constants (hidden_dim=256, num_atoms=51, num_quantiles=5, layer count, batch size). These are not "tuning knobs" — they come from convergence theorems, variance analysis, or model-shape definition.
## Architecture
### The universal contract
Every magnitude bound follows the same producer-consumer pattern:
```
Producer kernel (per training step):
step_p99 = P²-quantile(|signal|, p=0.99)
isv[BOUND_X] = (1 - α_X) · isv[BOUND_X] + α_X · step_p99
Consumer (clamp or diagnostic):
bound = isv[BOUND_X].max(ε_X) // ε_X = theoretical-init bootstrap, not tuned
out = clamp(v, ±bound) // or diagnostic-fire if |v| > bound
```
**Why p99 and not max-abs:** the EMA of `max(|v|)` is outlier-driven by construction — a single bad sample inflates the EMA, the bound chases the inflation, and the clamp ends up defined by the outliers it's supposed to control. p99 is a robust statistic: a single outlier (in the top 1%) doesn't move the 99th percentile. Clamping at p99 means clipping the worst 1% of magnitudes to the top-of-typical, which is the precise statistical statement of "control the tail without disturbing the body".
**Why P² and not radix-select / sort:** P² (Jain-Chlamtac, 1985) is a single-pass approximate quantile estimator using 5 markers (~20 floats of state per slot). No scratch buffer, no sort, single-block kernel, deterministic. Approximation error is typically <2% of the true p99 — well within the precision needed for clamping. Same algorithmic category as Adam β values (a published algorithm whose constants are theorem-derived, not tuned).
**Why no multiplier between ISV read and clamp:** the multiplier was the tuning knob. Removing it means the bound IS the observed signal statistic, with no per-mech "headroom" choice. The cold-start ε floor handles bootstrap; the EMA tracks distribution shifts.
### Per-bound vs per-param-group slots
Some signals (target_q, h_s2, grad_norm) are scalar-per-step — single ISV slot suffices.
Others (weights, Adam m, Adam v) are param-group-specific — IQN weights have a different natural distribution than DQN trunk weights. Sharing one bound across all param groups would over-clamp some and under-clamp others. **These signals get one ISV slot per param group.**
Param groups (7 total):
1. DQN trunk + branches (main `dqn_adam_update_kernel` params buffer)
2. DQN value head (within main params, but distinct param-group for bound purposes)
3. IQN-internal (separate param buffer + Adam state, `iqn_adam_kernel`)
4. IQL high-tau (`iql_adam_kernel`)
5. IQL low-tau (separate `iql_adam_kernel` instance)
6. Attention/TLOB (`attn_adam_kernel`, shared cubin)
7. Curiosity (`curiosity_adam_step`)
## Migration scope
### ISV slot allocation (~30 new slots)
| Slot family | Count | Tracks | Consumed by |
|---|---|---|---|
| `TARGET_Q_BOUND` | 1 | p99(\|target_q\|) | Mech 1 clamp + slot 46 diagnostic |
| `ATOM_POS_BOUND` | 4 (per branch) | p99(\|atom_positions[branch]\|) | Mech 2 clamps (3 sites × 4 branches) + slot 47 diagnostic |
| `WEIGHT_BOUND` | 7 (per param-group) | p99(\|params\|) per group | Mech 9 clamps × 5 Adam kernels + slots 44/45/48 diagnostics |
| `ADAM_M_BOUND` | 7 (per param-group) | p99(\|adam_m\|) per group | slots 36-39 diagnostics |
| `ADAM_V_BOUND` | 7 (per param-group) | p99(\|adam_v\|) per group | slots 40-43 diagnostics |
| `GRAD_CLIP_BOUND` | 1 | p99(grad_norm) | Mech 6 + adaptive_clip itself |
| `H_S2_BOUND` | 1 | p99(\|h_s2\|) | Mech 10 clamp + slot 49 diagnostic |
| Total | **28** | | |
### Diagnostic = clamp engagement, not separate threshold
Existing Mech 5 diagnostic slots (36-49) currently fire at `1e3 × Q_ABS_REF` etc. — a separate threshold above the clamp. Under SP4, **the diagnostic threshold = the clamp bound itself**. Slot N fires when the clamp engaged on this step (any element exceeded the bound and got clipped). Sticky flag captures "did clamping happen this step". This naturally fires rarely under stable training (because typical p99 is stable) and often during regime shifts (because the EMA hasn't caught up to the new distribution). No separate "headroom multiplier" needed.
**Implementation:** the producer kernel reads the buffer for p99 estimation; in the same pass it computes `step_max = max(|v|)` and compares to `prev_isv[BOUND]` (the bound used by the consumer this step). If `step_max > prev_isv[BOUND]`, the producer writes `nan_flags_buf[diag_slot] = 1` (sticky-flag, never cleared). Cost: one extra max-reduce + one comparison per producer launch — already in the same kernel that estimates p99, so adds <100 ns. Diagnostic ordering preserved per SP1 pearl (the firing reflects the buffer's pre-clamp state at the moment the consumer ran this step).
This drops the `1e3 ×` and `× 10` headroom patterns entirely.
### EMA rates as statistical-design parameters
The α value in each producer kernel is documented as the "observation time-window" — half-life of the EMA in training steps:
| Producer | α | Half-life | Time-window |
|---|---|---|---|
| `target_q_p99` | 0.05 | ~14 steps | sub-batch (fast) — Q values shift quickly |
| `atom_pos_p99` | 0.01 | ~70 steps | one batch — atom-position drift is slow |
| `weight_p99` | 0.001 | ~700 steps | ~1 fold — weights move slowly |
| `adam_m_p99`, `adam_v_p99` | 0.001 | ~700 steps | matches weight pace |
| `grad_norm_p99` | 0.05 | ~14 steps | per-batch grad noise |
| `h_s2_p99` | 0.05 | ~14 steps | matches existing `h_s2_rms_ema` |
Each α is a DESIGN choice ("how much history to retain") tied to the time-scale of the underlying signal, not a magnitude. Documented inline in each producer kernel. Not iterated via smokes.
### Cold-start ε bootstrap values
Each ISV bound slot has a bootstrap value applied at construction time and at fold-boundary reset (via `StateResetRegistry`). Bootstrap derives from theoretical-init scale (Xavier formula or domain knowledge):
| Bound | Bootstrap | Source |
|---|---|---|
| `TARGET_Q_BOUND` | `1.0` | Q values start near zero (untrained network output ~0); 1.0 floors the cold-start clamp at "no Q learned yet". |
| `ATOM_POS_BOUND[branch]` | `v_max[branch] - v_min[branch]` | Atom range itself, read from existing v-range slots [23+2*b, 24+2*b]. |
| `WEIGHT_BOUND[group]` | `√(2/K_in) × √K_in = √2` | Xavier init: σ = √(2/K_in), expected max-element ≈ σ × √K_in. Per param-group's K_in. |
| `H_S2_BOUND` | `1.0` | Post-Linear+ReLU, expected magnitude ~O(1) per Xavier analysis. |
| `ADAM_M_BOUND[group]` | `WEIGHT_BOUND[group] × LR_init` | Adam m ≈ lr × g_init; g_init ≈ Xavier init magnitude. |
| `ADAM_V_BOUND[group]` | `ADAM_M_BOUND[group]²` | Adam v ≈ m². |
| `GRAD_CLIP_BOUND` | `1.0` | Matches existing `MIN_CLIP=1.0` convention. |
`LR_init` is the optimizer's initial learning rate — already an ISV-driven parameter (Plan 1 Task 14, slot 91), so reading at bootstrap time is consistent.
These bootstraps are derived from Xavier init or domain semantics — same theoretical-constant category as Adam β values. Not tuning knobs.
## Producer architecture
### Per-signal producer kernels
Each ISV bound has a dedicated producer kernel mirroring the existing `h_s2_rms_ema_update` shape:
```
extern "C" __global__ void <signal>_p99_update(
const float* __restrict__ buf,
int N,
float* __restrict__ isv,
int isv_bound_index,
float ema_alpha
);
```
Single-block, 256-thread, P² quantile estimation:
- Block-local: each thread processes a strided slice of `buf`, contributes to a 5-marker P² state in shared memory
- Block reduce: 5 markers merged across threads (P² has a defined merge operation for parallel updates)
- Thread 0: writes `isv[isv_bound_index] = (1-α)·prev + α·current_p99`
Cold path: launches sit in `training_loop.rs` alongside `launch_h_s2_rms_ema`. Not in the captured graph (matches existing convention for ISV producers).
### P² state representation
Per-slot P² state: 5 markers (positions q[0..4]) and 5 desired marker positions n_d[0..4]. Stored as 10 floats per slot in a dedicated mapped-pinned buffer (per `feedback_no_htod_htoh_only_mapped_pinned`). State persists across training steps; producer reads + updates each step.
State size: `28 slots × 10 floats × 4 bytes = 1120 bytes` — negligible.
## Migration sequencing
Per `feedback_no_partial_refactor`, the contract change is one atomic commit. To manage risk, the implementation plan structures into three sequential layers:
### Layer A — additive infrastructure (no behavior change)
- Add 28 ISV slots to the bus
- Add producer kernels (compiled but not yet launched in critical path)
- Add `StateResetRegistry` entries with bootstrap values
- Add bootstrap initialization at construction
- Wire producer launches in `training_loop.rs` (slot updates happen each step but no consumer reads them yet)
After Layer A: ISV slots are populated correctly each step; nothing else changes. cargo check clean.
### Layer B — atomic consumer migration (the contract change)
ALL consumer sites simultaneously:
- Mech 1 clamp: replace `10 × Q_ABS_REF.max(1.0)` → `isv[TARGET_Q_BOUND].max(ε_target_q)`
- Mech 2 clamps (3 × 4 branches = 12 sites): replace → `isv[ATOM_POS_BOUND[branch]].max(ε_atom)`
- Mech 5 diagnostic kernel: thresholds replaced by per-slot ISV reads
- Mech 6 upper bound: replace → `isv[GRAD_CLIP_BOUND].max(ε_grad_clip)`
- Mech 9 across 5 Adam kernels: each kernel takes `weight_clamp_max_abs` from its `WEIGHT_BOUND[group]` slot
- Mech 10 clamp: replace `100 × H_S2_RMS_EMA.max(1.0)` → `isv[H_S2_BOUND].max(ε_h_s2)`
Single commit. Hardcoded multipliers all removed in this step. `feedback_no_partial_refactor` honoured.
### Layer C — validation + cleanup
- L40S smoke validation (multi_fold_convergence)
- Remove the now-dead Mech 5 fused-kernel ISV parameters (`q_abs_ref_eff`, `h_s2_rms_ema_eff`) — replaced by per-slot ISV reads
- Audit doc updates (mechanism table, slot allocation table, ISV bindings)
- Memory entry capturing the SP4 close-out
## Validation
### Pass criteria for the post-Layer-C smoke
1. **F0, F1, F2 all complete 5 epochs** — no early-stop on NaN (zero NaN-CLAMPED log lines)
2. **F1 Best Sharpe > 0** — fixes the original SP3 motivation; no h_s2-overflow cascade
3. **F0 Best Sharpe ≥ 45** — no regression from v2's 45.47; SP4 should restore (or exceed) the v2 baseline because clamps no longer over-bind heavy-tail magnitudes
4. **All Mech 5 diagnostic slots (36-49) fire only on regime shifts** — sticky-flag pattern; clamps engaging is the "diagnostic"
5. **No new ISV slots beyond the 28 in this design**
6. **No new kernels beyond producer-per-signal**
7. **Each consumer site reads exactly one ISV slot** (no compound expressions like `K × isv[X]`)
### Regression sentinels (not pass/fail, but logged)
- Per-step count of clamp engagements per bound — if `WEIGHT_BOUND[group]` clamps >5% of params, weights are saturating somewhere upstream; investigate.
- p99-EMA values over training — should be smooth; sudden jumps signal regime shifts.
## Out of scope / open questions
### EMA-rate signal-drivenness
Per the brainstorm, EMA rates (α=0.05 etc.) stay as **statistical-design parameters**, not signal-driven. Making α itself signal-derived creates a recursion (what controls α's update rate?). The pragmatic resolution: α is a "time-window choice" (how much history to retain), analogous to choosing 1-second vs 1-hour averages — a design-time decision, not a behavioral tuning knob.
If a future principle escalation requires α to also be signal-driven, that's a follow-up project; the architecture here doesn't preclude it.
### Param-group granularity vs cross-group sharing
The design uses 7 param-groups for `WEIGHT_BOUND`, `ADAM_M_BOUND`, `ADAM_V_BOUND` (= 21 of the 28 slots). This is the maximum granularity. If smoke validation shows certain groups have nearly-identical p99 distributions, slots could be merged in a follow-up. **Default: per-group split** to avoid over/under-clamping mismatch.
### Pre-existing ISV slots — leave or migrate
Slots 23+2*b, 24+2*b (atom v-range) are already directly in ISV without multipliers. They serve as the bootstrap for `ATOM_POS_BOUND` and stay as-is.
Slot 16 (`Q_ABS_REF`) was the input to several Mech multipliers. After SP4, slot 16 is no longer read by any Mech (its consumers all migrate to dedicated bounds). Slot 16 may still be used by HEALTH_DIAG / monitoring for its information value — leave the producer running, drop only the consumer reads in Mech 1/2/5/6/9/10.
Slot 96 (`H_S2_RMS_EMA`) — same treatment. Was read by Mech 10. After SP4, only `H_S2_BOUND` (slot 49 follow-on or new slot) is read. Slot 96 stays for monitoring + `mag_concat_qdir` adaptive scale (P4.T2c.3c.6 consumer remains).
### Mech 10 short-term
Until SP4 lands, the SP3 close-out v2 (Mech 10 at 100×, F1 fixed, F0 plateau accepted) remains in place. SP4 supersedes Mech 10's hardcoded multiplier; the activation-clamp behavior is preserved by the new `H_S2_BOUND` consumer.
## Estimated effort
- **Layer A:** ~28 producer kernels + ISV plumbing + reset entries + bootstraps ≈ 1500-2000 LOC, multi-day work
- **Layer B:** ~15 consumer sites × ~10 LOC change each ≈ 200 LOC, one focused day
- **Layer C:** smoke + audit doc + memory entry ≈ 100 LOC + validation
Total: ~2-3000 LOC across 3 layers, 1-2 weeks of focused work, 1 L40S smoke validation. Mid-large project.