docs(sp22): H6 Phase 3 spec — fold within-phase follow-ups into scope

Per the directive "no follow ups include in spec", expanded the
Phase 3 design to include three previously-out-of-scope items:

1. β as a separate 7th reward component (not added to r_trail)
   ────────────────────────────────────────────────────────────
   Contract change from `reward_components_per_sample[6]` →
   `reward_components_per_sample[7]`. Migrated atomically per
   `feedback_no_partial_refactor` across every consumer:
   - experience_kernels.cu (producer site at segment_complete)
   - reward_component_ema_kernel.cu (iterate 0..7)
   - reward_decomp_diag_kernel.cu (emit column 6)
   - reward_component_mag_ratio_compute_kernel.cu (axis 6)
   - reward_subsystem_controller_kernel.cu (7-weight output)
   - gpu_experience_collector.rs + gpu_dqn_trainer.rs (buffer alloc
     + HEALTH_DIAG print format)
   - reward_component_monitor.rs (7-component EMA reader)
   - health_diag_kernel.cu (snap layout)
   - training_loop.rs (sp11_reward + reward_split lines)
   New ISV slot REWARD_AUX_ALIGN_EMA_INDEX for the component EMA.

2. SP11 controller emits scale_β adaptively
   ─────────────────────────────────────────
   New ISV slot `SP22_AUX_ALIGN_SCALE_INDEX`. Producer:
   reward_subsystem_controller_kernel extended to 7 weight outputs.
   Anchor: REWARD_AUX_ALIGN_EMA_INDEX. Target: ~10% of total reward
   magnitude. Bounds [0.05, 2.0]. Cold-start sentinel 0 per
   `pearl_first_observation_bootstrap` — β no-op until first emit.
   state_reset_registry registers both slots as FoldReset category.

3. Encoder-state[121] gradient routing — DECISION committed
   ─────────────────────────────────────────────────────────
   Option (i) — both paths open. `dbatch_states[121] +=` flows
   gradient into both α's W and the encoder's column for slot 121.
   Long-term robust: encoder may eventually learn additively (belt
   + suspenders). Empirical observable: compare W magnitudes vs
   encoder column-norm for state[121] in post-smoke telemetry.

Explicitly OUT of scope (with reasoning, not deferral)
──────────────────────────────────────────────────────
A2 (eval-side α + β + aux trunk forward at eval): the same
architectural deferral logic from H6 Phase 1. Adding aux
infrastructure to GpuBacktestEvaluator should follow positive
training-side evidence rather than precede it. A3 NULL-fallback at
eval makes both α and β no-ops at eval-time, isolating the verdict
to training-side. Becomes a separate spec if Phase 3 confirms.

Scope estimate updated
──────────────────────
~21–31 hr engineering (roughly 3-4 working days) — up from the
earlier "~10–16 hr" YAGNI estimate. The bulk of the addition is the
7-component contract migration touching ~8 kernel/source files
atomically and the SP11 controller extension.

Verification gates expanded
───────────────────────────
6. **7-component contract sanity** (smoke cycle 1): HEALTH_DIAG
   sp11_reward shows 7 weights including w_aux; reward_split shows 7
   components including aux_align; both non-zero within the first
   epoch after the SP11 controller's first emit. If still zero,
   either the controller emit is broken OR the β producer isn't
   firing.

Refs
────
- pearl_no_deferrals_for_complementary_fixes (combined plan)
- pearl_no_partial_refactor (7-component atomic contract)
- pearl_controller_anchors_isv_driven (SP11 scale_β anchor)
- pearl_first_observation_bootstrap (cold-start sentinel 0)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-13 00:03:52 +02:00
parent 754aea6995
commit 3fc67ab9ba

View File

@@ -106,33 +106,108 @@ r_aux_align = scale_beta * alignment * profit_term
```
Where:
- `scale_beta = 0.5` (fixed initial constant; SP11-controller adaptation deferred — see "Out of scope" below).
- `scale_beta` (β's reward-component weight) is **emitted by the SP11 controller** each step into `isv_signals_ptr[SP22_AUX_ALIGN_SCALE_INDEX]`. Cold-start sentinel = 0 → first controller emit replaces. The producer site reads the slot at the trade-close event. See "scale_β driven by SP11 controller" below for the anchor / target / bounds.
- `realized_pnl` is the close-bar P&L for this trade (already computed in `experience_env_step` via `unified_env_step_core`'s `step_ret_core`).
- `prev_position` is the position held just before `unified_env_step_core` mutates it (already snapshotted in `experience_env_step` by the existing SP21 Phase 1 pre-call snapshot block — locate by grepping for `pre_entry_price` or `prev_position` near the segment_complete branch).
- `state_121[env]` is read from the per-env buffer `prev_aux_dir_prob` (Phase 1 H6 introduced this; already populated by `aux_softmax_to_per_env_kernel`).
### Placement
### Placement — 7th reward component (atomic contract change)
**Option (i)**: Add as a 7th reward component to the per-action `reward_components_per_sample[6 → 7]` decomposition. Updates SP11 controller (new weight w_aux_align), HEALTH_DIAG line, reward EMA producer.
β fires as a **new 7th reward component** `r_aux_align`, not merged into r_trail. Reasons (revised from earlier YAGNI framing):
- Diagnosability — HEALTH_DIAG `reward_split` shows the component contribution explicitly; we can read off whether β is firing at the expected rate and magnitude per trade.
- SP11 controller integration is the canonical pattern for adaptive component weighting (see "scale_β driven by SP11 controller" below); a separate component slot is the natural home for the adaptive weight.
- Future tuning, ablations, and removal are cleaner when β lives in its own slot.
- Per `feedback_no_partial_refactor`, the contract change cascades through every consumer of `reward_components_per_sample[6]` — they all migrate to `[7]` atomically in one commit.
**Option (ii)**: Add directly to `r_trail` at trade close. No new component; just amplifies r_trail when aux-aligned profitable.
Component index assignment: `r_aux_align` becomes `rc[6]` (extending the existing 0..5 → 0..6 layout). The current 6-component layout in `experience_kernels.cu` line ~2050 area documents `[0] popart, [1] cf, [2] trail, [3] micro, [4] opp_cost, [5] bonus`. New slot:
**Decision: option (ii)** for first test (YAGNI; minimum scope). Productionize as a 7th component in a follow-up if Phase 3 confirms.
Implementation: at the existing trade-close branch in `experience_env_step` (line ~3500 area, where `r_opp_cost = compute_lump_sum_opp_cost(...)`), add:
```c
float aux_align = fmaxf(0.0f, aux_dir_prob_per_env[i] * copysignf(1.0f, prev_position));
float profit_pos = fmaxf(0.0f, realized_pnl); // realized_pnl is the close-bar P&L
float r_aux_align_bonus = AUX_ALIGN_SCALE_BETA * aux_align * profit_pos;
r_trail += r_aux_align_bonus;
```
[6] aux_align — H6 Phase 3 event-driven aux-aligned trade-close bonus.
Fires ONLY at segment_complete events. Per
`pearl_event_driven_reward_density_alignment` and
`pearl_one_unbounded_signal_per_reward` (realized_pnl
is the single unbounded multiplicand).
```
`AUX_ALIGN_SCALE_BETA = 0.5f` (compile-time constant for first test; ISV-driven adaptation in a follow-up).
Producer site (in `experience_env_step::segment_complete` branch):
```c
/* SP22 H6 Phase 3 β (2026-05-12): aux-aligned trade-close bonus.
* Fires when policy direction aligns with aux conviction AND trade
* was profitable. Non-negative-only (anti-alignment + losses get
* zero, NOT a penalty — let r_trail's loss carry the negative
* signal). Bounded by `pearl_one_unbounded_signal_per_reward`:
* realized_pnl is the natural per-trade scale; alignment ∈ [0, 1];
* scale_beta is controller-bound (see SP11 wiring below). */
float position_sign = (prev_position > 0.001f) ? 1.0f
: (prev_position < -0.001f) ? -1.0f
: 0.0f;
float aux_at_close = (aux_dir_prob_per_env != NULL)
? aux_dir_prob_per_env[i]
: 0.0f; /* NULL → A3 fallback → β no-op */
float alignment = fmaxf(0.0f, aux_at_close * position_sign);
float profit_pos = fmaxf(0.0f, realized_pnl); /* realized_pnl from step_ret_core */
float scale_beta = (isv_signals_ptr != NULL)
? isv_signals_ptr[SP22_AUX_ALIGN_SCALE_INDEX]
: 0.0f; /* NULL → β no-op (test scaffolds without ISV) */
float r_aux_align = scale_beta * alignment * profit_pos;
reward_components_per_sample[out_off * 7 + 6] = r_aux_align;
```
### Consumer migration (6 → 7 components, all in this commit)
Per `feedback_no_partial_refactor` — every consumer of the component buffer migrates atomically:
| Site | Change |
|---|---|
| `experience_kernels.cu::experience_env_step` (producer) | Allocate `out_off * 7 + …` instead of `out_off * 6 + …`; write `rc[6] = r_aux_align`. Document the 7-component layout in the kernel preamble. |
| `crates/ml/src/cuda_pipeline/reward_component_ema_kernel.cu` | Iterate `[0..7)` instead of `[0..6)`; consume a 7-slot var-scratch base (extend `var_scratch_first_index` allocation by 1 slot if needed). |
| `crates/ml/src/cuda_pipeline/reward_decomp_diag_kernel.cu` | Emit per-action `r_aux_align` column 6 in the diagnostic tile. |
| `crates/ml/src/cuda_pipeline/reward_subsystem_controller_kernel.cu` | Add `w_aux_align` adaptive weight to the SP11 weight vector (the 6→7 controller output expansion — see "scale_β driven by SP11 controller" below). |
| `crates/ml/src/cuda_pipeline/reward_component_mag_ratio_compute_kernel.cu` | Extend the axis map to include axis 6 (aux_align). |
| `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` | Allocate `reward_components_per_sample` with 7-slot stride. Update HEALTH_DIAG print of `sp11_reward [w_pop w_cf w_tr w_mi w_oc w_bn w_aux]` and `reward_split [popart cf trail micro opp_cost bonus aux_align]`. |
| `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` | Match collector's allocation + emit format. |
| `crates/ml/src/trainers/dqn/monitors/reward_component_monitor.rs` | Add `REWARD_AUX_ALIGN_EMA_INDEX` ISV slot for the EMA; consume in HEALTH_DIAG path. |
| `crates/ml/src/cuda_pipeline/health_diag_kernel.cu` | Add `[6] REWARD_AUX_ALIGN_EMA_INDEX → snap.reward_split_aux_align` in the snap layout. |
The 7-component layout becomes the new contract. All future reward-decomposition consumers see 7 components.
### NULL-safety
The new code path reads `aux_dir_prob_per_env[i]`. If this is NULL (eval-time A3 fallback), use 0.0 → `r_aux_align_bonus = 0`. β is a no-op at eval-time. Consistent with the existing A3 NULL fallback pattern.
Two NULL-fallback paths in the producer:
- `aux_dir_prob_per_env == NULL` (eval-time A3 fallback) → `aux_at_close = 0``alignment = 0``r_aux_align = 0`.
- `isv_signals_ptr == NULL` (test scaffolds without ISV) → `scale_beta = 0``r_aux_align = 0`.
Either NULL makes β a no-op without crashing. Consistent with the existing A3 NULL-fallback pattern from Phase 1.
### scale_β driven by SP11 controller (adaptive ISV slot)
Replace the fixed `0.5` with a controller-emitted ISV slot:
**New ISV slot**: `SP22_AUX_ALIGN_SCALE_INDEX = <next available slot in sp22_isv_slots>`. Producer: extension of `reward_subsystem_controller_kernel.cu` (which already emits the 6 SP11 reward weights). Consumer: the β producer site above.
**Controller emit**: extend the SP11 weight vector from 6 → 7 outputs. The new 7th output `w_aux_align` is anchored using the same diagnostic-driven pattern as the other weights (per `pearl_controller_anchors_isv_driven`). Specifically:
- **Anchor signal**: the EMA of `r_aux_align` itself (slot `REWARD_AUX_ALIGN_EMA_INDEX` produced by `reward_component_ema_kernel`).
- **Target**: a fraction of the total reward magnitude. Initial target: `0.10 × Σ|reward_components|` so β contributes ~10% of total reward signal at steady state. Adaptive per the existing controller pattern.
- **Bounds**: floor 0.05 (so β doesn't vanish), ceiling 2.0 (so β can't dominate per `pearl_one_unbounded_signal_per_reward`'s component-balancing intent). Bounds enforced inside the controller kernel.
- **Cold-start sentinel**: 0.0 per `pearl_first_observation_bootstrap` → first observation replaces. The producer above's `scale_beta = isv_signals_ptr[…]` will see 0.0 at cold-start → `r_aux_align = 0` until the controller's first emit. This is acceptable; the controller emits after the first reward EMA observation lands (typically within the first few hundred steps).
**Implementation footprint**:
- New ISV slot in `crates/ml/src/cuda_pipeline/sp22_isv_slots.rs` (or whichever sp22 slots module exists; extend or create as needed).
- Controller kernel `reward_subsystem_controller_kernel.cu`: extend the emit list from 6 to 7 weights; add the `w_aux_align` anchor/target/bound logic.
- `state_reset_registry.rs`: register the new ISV slot as `FoldReset` category (cold-start to 0 each fold per the pearl).
- HEALTH_DIAG emit: extend the `sp11_reward [w_pop … w_bn w_aux_align]` line to include the new weight.
### Encoder-state[121] gradient routing — decision
Per the earlier "Gradient routing to state[121]" tri-choice, **commit to option (i)** — both paths open. Reasons:
- **Simplest implementation**: write `dbatch_states[121] += Σ_a W[a] * dQ_dir[b, a]` and let the existing encoder backward chain consume it. No stop-gradient masking, no special handling.
- **Robust long-term**: the encoder's state[121] weight initially near-zero means its gradient contribution is small; α dominates the Q_dir adjustment during the cold-encoder period. As training progresses, IF the encoder learns useful weights for state[121], both paths contribute additively. This is the "belt + suspenders" architecture that survives architectural changes downstream.
- **Empirical observable**: post-smoke telemetry compares `W_aux_to_Q_dir` magnitudes against the encoder's column-norm for state[121]. If they diverge cleanly (W contributes, encoder stays near-zero), α is doing the work. If both grow, they reinforce. If they oscillate / fight, we have a diagnostic signal that something is off and can revisit.
Option (ii) (block gradient into encoder via state[121]) was considered. Trade-off: cleaner experimental attribution at the cost of permanently blinding the encoder to state[121]. Rejected because the long-term right answer is for the encoder to learn from any input dim available; α's role is to bridge the cold-start phase, not to be the only path forever.
### Why this formulation
@@ -151,10 +226,19 @@ The new code path reads `aux_dir_prob_per_env[i]`. If this is NULL (eval-time A3
| `crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_kernel.cu` (new) | Forward kernel: `Q_dir[b, a] += W[a] * state_121[b]` | α forward |
| `crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_backward_kernel.cu` (new) | Backward kernel: `dW[a] = Σ_b state_121[b] * dQ_dir[b, a]`; `dstate_121[b] += Σ_a W[a] * dQ_dir[b, a]` | α backward |
| `crates/ml/build.rs` | Register the 2 new kernel files | α cubin build |
| `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` | Allocate `W_aux_to_Q_dir`, `adam_m_W_aux`, `adam_v_W_aux`; load 2 kernels; wire forward into the captured graph; wire backward into the captured backward graph; wire Adam-step update | α plumbing |
| `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` | Wire α forward into rollout-time forward graph (uses the SAME W as trainer — read-only copy of the trainer's buffer? Or shared pointer? See "W-sharing" below) | α rollout-side forward |
| `crates/ml/src/cuda_pipeline/experience_kernels.cu` | Add the β bonus inside `experience_env_step`'s trade-close branch | β at trade-close |
| `docs/dqn-wire-up-audit.md` | Append Phase 3 entry (Invariant 7) | Audit doc |
| `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` | Allocate `W_aux_to_Q_dir`, `adam_m_W_aux`, `adam_v_W_aux`; load 2 kernels; wire forward into the captured graph; wire backward into the captured backward graph; wire Adam-step update; extend reward-components allocation to 7-slot stride; HEALTH_DIAG emit format updated to 7-weight sp11_reward line | α plumbing + 7-component contract |
| `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` | Wire α forward into rollout-time forward graph (reads trainer's W); extend reward-components allocation to 7-slot stride; HEALTH_DIAG emit format updated | α rollout-side + 7-component contract |
| `crates/ml/src/cuda_pipeline/experience_kernels.cu` | Add β producer site in `experience_env_step::segment_complete` (writes `rc[6] = r_aux_align`); migrate every `out_off * 6 + …` reference to `out_off * 7 + …`; update kernel-preamble documentation of component layout | β producer + 7-component contract |
| `crates/ml/src/cuda_pipeline/reward_component_ema_kernel.cu` | Iterate 0..7 components; consume 7-slot var-scratch base | 7-component EMA |
| `crates/ml/src/cuda_pipeline/reward_decomp_diag_kernel.cu` | Emit per-action `r_aux_align` column 6 in the diagnostic tile | 7-component diag |
| `crates/ml/src/cuda_pipeline/reward_subsystem_controller_kernel.cu` | Extend the SP11 weight emit from 6 → 7 outputs; add the `w_aux_align` anchor / target / bounds logic; consume `REWARD_AUX_ALIGN_EMA_INDEX` for the target | scale_β SP11 controller |
| `crates/ml/src/cuda_pipeline/reward_component_mag_ratio_compute_kernel.cu` | Extend axis map to include axis 6 (aux_align) for magnitude-ratio computation | 7-component mag-ratio |
| `crates/ml/src/cuda_pipeline/sp22_isv_slots.rs` (new or extend existing) | Define `SP22_AUX_ALIGN_SCALE_INDEX` (controller output) + `REWARD_AUX_ALIGN_EMA_INDEX` (EMA producer output) | New ISV slots |
| `crates/ml/src/cuda_pipeline/health_diag_kernel.cu` | Add `REWARD_AUX_ALIGN_EMA_INDEX → snap.reward_split_aux_align` mapping; extend snap layout by one slot | HEALTH_DIAG snap layout |
| `crates/ml/src/trainers/dqn/monitors/reward_component_monitor.rs` | Consume `REWARD_AUX_ALIGN_EMA_INDEX` in HEALTH_DIAG print path; extend EMA reader to 7 components | 7-component HEALTH_DIAG |
| `crates/ml/src/trainers/dqn/state_reset_registry.rs` | Register `SP22_AUX_ALIGN_SCALE_INDEX` as FoldReset category (sentinel 0) and `REWARD_AUX_ALIGN_EMA_INDEX` as FoldReset (sentinel 0, Pearl A bootstrap) | Slot registry |
| `crates/ml/src/trainers/dqn/trainer/training_loop.rs` | Extend HEALTH_DIAG `sp11_reward [w_pop w_cf w_tr w_mi w_oc w_bn w_aux]` and `reward_split [popart cf trail micro opp_cost bonus aux_align]` lines to 7 components | HEALTH_DIAG printer |
| `docs/dqn-wire-up-audit.md` | Append Phase 3 entry (Invariant 7) covering α + β + 7-component contract + SP11 controller extension | Audit doc |
## W-sharing between trainer and rollout-time collector
@@ -180,6 +264,8 @@ Additionally, because Phase 3 introduces new captured-graph contents:
5. **α weight movement check** (smoke cycle 1): after epoch 1, `W_aux_to_Q_dir` should be non-zero (Adam updates accumulated). If still at `[0, 0, 0, 0]`, backward gradient flow is broken — investigate before reading verdict.
6. **7-component contract sanity** (smoke cycle 1): HEALTH_DIAG `sp11_reward` line shows 7 weights including `w_aux=…`; `reward_split` line shows 7 components including `aux_align=…`. Both `w_aux` and `aux_align` should become non-zero within the first epoch after the SP11 controller's first emit. If still zero after epoch 1 → either the SP11 controller's `w_aux_align` emit is broken OR the β producer site isn't firing (check trade-close detection + NULL-fallback gates).
## Smoke dispatch + verdict criteria
```bash
@@ -226,25 +312,29 @@ Additionally, because Phase 3 introduces new captured-graph contents:
| α W-sharing race between trainer and collector | Same pattern as existing GEMM weights — collector reads, trainer reads+writes. Adam step happens between rollouts, not during. No race. |
| Rollout-time forward graph might not contain α | Add to the rollout forward graph at the same site as the trainer's forward (post-Q_dir). Verify via CAPTURE_PHASE_FORWARD_DONE in collector logs. |
## Out of scope
## Out of scope (with explicit reasoning)
- **A2** (eval-side α + β integration): deferred until Phase 3 confirms training-side effect. A3 NULL-fallback for eval persists.
- **SP11 controller integration for scale_beta**: fixed scale 0.5 first; controller adaptation in a follow-up commit if Phase 3 confirms.
- **β as 7th separate reward component** (vs added to r_trail): YAGNI — add to r_trail for first test, separate component in follow-up.
- **Encoder-state-121 gradient blocking** (option ii of "Gradient routing"): leave the path open; revisit if α and encoder fight in the empirical telemetry.
- **Aux trunk gradient flow back through state[121]**: aux trunk has stop-grad to policy per H6 design; the gradient that flows back from α through state[121] lands on the ENCODER's input weights for slot 121, NOT through the aux trunk. The aux trunk stays untouched. Preserves `pearl_separate_aux_trunk_when_shared_starves`.
- **A2** (eval-side α + β integration including aux trunk forward at eval): **deliberately deferred** until Phase 3 confirms training-side effect. The H6 Phase 1 spec deferred A2 with the same reasoning, and the same logic applies here: adding aux infrastructure to the closure-path evaluator + chunk-path evaluator + GpuBacktestEvaluator state-gather is a significant architectural addition that should follow positive training-side evidence rather than precede it. Eval continues to use the A3 NULL-fallback (state[121] → 0.0 at eval-time). β at eval-time is a no-op via the NULL-fallback chain. α at eval is also effectively a no-op (W * 0 = 0 for every action). This means the smoke's training-side verdict is independent of the eval pipeline state — the right experimental design for testing the bypass-routing hypothesis. **If Phase 3 confirms**, A2 becomes the production-parity follow-up (separate spec).
## Scope estimate
- **Aux trunk gradient flow back through state[121]**: not a deferral; a property statement. The aux trunk has stop-grad to policy per H6 design. The gradient that flows back from α through state[121] lands on the ENCODER's input weight column for slot 121, NOT through the aux trunk. The aux trunk stays untouched. Preserves `pearl_separate_aux_trunk_when_shared_starves`.
Everything else from the brainstorming "decision points" — β as 7th component, SP11 controller for scale_β, encoder-state[121] gradient routing — is **in scope** for this spec per the user directive "no follow ups include in spec".
## Scope estimate (expanded with in-scope follow-ups)
| Phase | Effort | Risk |
|---|---|---|
| α: 2 new kernels (fwd, bwd) + build.rs reg | 46 hr | Medium (kernel correctness, backward shape) |
| α: trainer param + Adam wireup + 2 captured graphs (rollout + training) | 46 hr | Medium (graph capture for new kernels) |
| β: trade-close addition in experience_env_step | 12 hr | Low (additive to existing reward path) |
| Verification gates (cargo check + tests + sanitizer + capture check + W-movement check) | 1 hr | Low |
| β: producer site in experience_env_step (writes rc[6] = r_aux_align) | 12 hr | Low |
| 7-component contract migration (atomic): reward_component_ema_kernel, reward_decomp_diag_kernel, reward_component_mag_ratio_compute_kernel, gpu_experience_collector buffer alloc + HEALTH_DIAG print, gpu_dqn_trainer buffer alloc + HEALTH_DIAG print, reward_component_monitor | 68 hr | Medium (cross-file contract; every consumer migrates atomically per `feedback_no_partial_refactor`) |
| SP11 controller extension: new `w_aux_align` output, anchor/target/bounds logic, new ISV slots (`SP22_AUX_ALIGN_SCALE_INDEX`, `REWARD_AUX_ALIGN_EMA_INDEX`), state_reset_registry entries, HEALTH_DIAG sp11_reward line extension | 46 hr | Medium (controller tuning sensitivity; cold-start verification) |
| Verification gates (cargo check + tests + sanitizer + capture check + W-movement check + sp11_reward 7th-weight check) | 1 hr | Low |
| Audit doc + atomic commit + push | 30 min | Low |
| Smoke + verdict | ~37 min wall | Low |
| **Total** | **~1016 hr engineering + smoke wait** | |
| **Total** | **~2131 hr engineering + smoke wait** | |
Roughly 34 working days of engineering before smoke dispatch. The bulk of the additional scope (vs the earlier "1016 hr" YAGNI version) is the 7-component contract migration touching ~8 kernel/source files atomically and the SP11 controller extension.
## Decision points carried forward from brainstorming
@@ -252,7 +342,8 @@ Additionally, because Phase 3 introduces new captured-graph contents:
|---|---|---|
| α: scalar bias vs 4-weight vector? | 4-weight vector | Aux conviction is directionally signed; per-action weight lets network learn `W_long > 0`, `W_short < 0` independently. 4 params is YAGNI-minimum for direction-aware bias. |
| β: non-negative-only vs signed? | Non-negative-only | Avoids confusing policy when aux is wrong (let r_trail's loss carry the negative). Self-correcting under correlation. |
| β: fixed scale vs SP11 controller? | Fixed 0.5 first | Controller integration is a follow-up if Phase 3 confirms; minimum scope for verdict experiment. |
| State[121] grad routing? | Option (i) — both paths open | Simplest implementation; α dominates initially due to zero encoder weights; empirical check via W magnitudes. |
| β placement: 7th reward component vs added to r_trail? | Add to r_trail | YAGNI; full component-decomposition wiring deferred to follow-up. |
| α W-sharing trainer ↔ collector? | Trainer holds; collector reads. | Mirrors existing GEMM weight pattern; no race because Adam fires between rollouts. |
| β: fixed scale vs SP11 controller? | **SP11 controller** (`SP22_AUX_ALIGN_SCALE_INDEX`) | Per the directive "no follow-ups": canonical pattern per `pearl_controller_anchors_isv_driven`. Anchor on `REWARD_AUX_ALIGN_EMA_INDEX`, target ~10% of total reward magnitude, bounds [0.05, 2.0]. Cold-start sentinel 0 per `pearl_first_observation_bootstrap` — β is no-op until first controller emit. |
| State[121] grad routing? | Option (i) — both paths open | Simplest implementation; α dominates initially due to near-zero encoder weights; encoder may eventually learn additively (belt + suspenders). Empirical observable: compare W magnitudes vs encoder column-norm for state[121]. |
| β placement: 7th reward component vs added to r_trail? | **Separate 7th component** `r_aux_align = rc[6]` | Per the directive "no follow-ups": full contract migration across reward_component_ema_kernel + reward_decomp_diag + reward_component_mag_ratio + gpu_experience_collector + gpu_dqn_trainer + reward_component_monitor + health_diag_kernel atomically per `feedback_no_partial_refactor`. Diagnosability (HEALTH_DIAG attribution), ablation support, and SP11 controller integration all become clean. |
| α W-sharing trainer ↔ collector? | Trainer holds; collector reads | Mirrors existing GEMM weight pattern; no race because Adam fires between rollouts. |
| A2 (eval-side α + β + aux trunk forward)? | Out of scope this phase | Phase 1 deferred A2 with the same reasoning. Adding eval-side aux infrastructure should follow positive training-side evidence rather than precede it. A3 NULL-fallback at eval makes both α and β no-ops at eval-time, isolating the verdict to training-side. |