gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(sp5): Task A2 — Pearl 3 per-branch NoisyNet sigma

Browse Source 
Per-branch sigma scales with per-branch Q magnitude (v_half from Pearl 1's
ATOM_V_HALF, populated in A1). SIGMA_FRACTION adapts via entropy-deficit
controller targeting 70% of max action entropy (target_entropy =
log(n_actions[b]) * 0.7).

8 ISV slots (NOISY_SIGMA[210..214), SIGMA_FRACTION[214..218)). Reuses
BRANCH_ENTROPY (218..222) from Task A1's q_branch_stats_kernel.

producer_step_scratch_buf grew 103 -> 111 (8 new outputs). wiener_state_buf
already at 543 (A1 sized for entire SP5 block).

NoisyLinear consumer migration deferred to Layer B.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-01 21:16:11 +02:00
parent 3b4ce05465
commit ab3e17f4a2
7 changed files with 472 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/dqn-wire-up-audit.md
View File

@@ -2,6 +2,8 @@
**Status:** Populated during Plan 1 Task 6 (A.5 orphan audit). Updated on every commit per Invariant 7.
SP5 Task A2 — Pearl 3 per-branch NoisyNet sigma GPU producer (Layer A, 2026-05-01): one new CUDA kernel (`pearl_3_sigma_kernel.cu`) lands as a Layer A additive producer that feeds ISV slots [210..218) with per-branch NoisyNet sigma and sigma-fraction controller state. No NoisyLinear consumer migration in this commit — Layer B wires the noisy_linear_kernel.cu consumer. `pearl_3_sigma_update`: single-block 4-thread kernel (one thread per branch), reads ATOM_V_HALF[b] (Pearl 1, ISV[178..182)) + BRANCH_ENTROPY[b] (q_branch_stats, ISV[218..222)) + current SIGMA_FRACTION[b] (ISV[214..218), Pearl A sentinel 0 → bootstrap to 0.1); applies entropy-deficit controller `new_sf = clamp(sf + SF_LR * (target_entropy - entropy), 0.001, 0.5)` where `target_entropy = log(n_actions[b]) * 0.7` (SF_LR=0.005 and target_entropy_factor=0.7 are Invariant 1 structural anchors); computes `new_sigma = new_sf * max(v_half, EPS_CLAMP_FLOOR=1.0)` (EPS_CLAMP_FLOOR is an Invariant 1 anchor); writes (sigma[4], SF[4]) to scratch[103..111) (SCRATCH_PEARL_3_SIGMA=103, SCRATCH_PEARL_3_SF=107). `launch_sp5_pearl_3_sigma` fires 8 `launch_apply_pearls` calls: sigma[4] → ISV[210..214) (NOISY_SIGMA_BASE), SF[4] → ISV[214..218) (SIGMA_FRACTION_BASE). Wiener offset formula: `base_wiener_offset + (isv_slot - SP5_SLOT_BASE) * 3` where `base_wiener_offset = SP4_PRODUCER_COUNT * 3 = 213`. Pearl 3 must chain AFTER Pearl 1 (reads ATOM_V_HALF which Pearl 1 populates via apply_pearls_ad). Buffer growth: `producer_step_scratch_buf` 103→111 (SP5_SCRATCH_TOTAL updated; new constants SCRATCH_PEARL_3_SIGMA=103, SCRATCH_PEARL_3_SF=107). wiener_state_buf already at 543 (A1 sized for entire SP5 block — no further growth). StateResetRegistry gains 2 new FoldReset entries: sp5_noisy_sigma (ISV[210..214)) and sp5_sigma_fraction (ISV[214..218)) — Pearl A sentinel 0 at fold boundary triggers bootstrap to 0.1 on new fold's first launch. Wire-up in training_loop.rs: `launch_sp5_pearl_3_sigma()` called immediately after `launch_sp5_pearl_1_atom()` with `warn!` on error. Two GPU-only `#[ignore = "requires GPU"]` unit tests: (3) bootstrap + target-entropy equality case (sentinel ISV SF=0→bootstrap 0.1, target==actual → no update, sigma=0.1*2.0=0.2); (4) entropy-below-target case (SF=0.1, entropy=0 → SF increases by SF_LR*target_entropy, sigma increases proportionally). No CPU compute, no HtoD/HtoH, no atomicAdd, no stubs. Touched: `cuda_pipeline/pearl_3_sigma_kernel.cu` (new), `build.rs` (+1 cubin entry), `cuda_pipeline/gpu_dqn_trainer.rs` (static cubin + SP5_SCRATCH_TOTAL 103→111 + 2 new constants + struct field + cubin loader + struct initializer + launch method), `trainers/dqn/state_reset_registry.rs` (+2 FoldReset entries), `trainers/dqn/trainer/training_loop.rs` (+4 LOC after Pearl 1 launch), `tests/sp5_producer_unit_tests.rs` (+2 GPU-only tests + cubin loader helper). cargo check -p ml --lib clean. No consumer migration in this commit.
SP5 Task A1 fix-up — pearl_1 clip_rate denominator must be per-branch (2026-05-01): code-quality review caught the headroom controller's clip_rate computation using `batch_size × 13` (sum of per-branch action counts) instead of `batch_size × action_counts[b]` (per-branch). The TARGET_CLIP_RATE=0.01 anchor is calibrated against the per-branch event rate; using a denominator 3.25–4.3× too large made the controller systematically under-responsive — at actual 1% per-branch clipping it would read ~0.23–0.31% and contract headroom toward the 2.0 floor instead of holding the target. Currently masked because `atoms_clip_count` is zero in Layer A (Layer B's atoms_update_kernel migration populates it); without this fix Layer B would inherit a silently-wrong formula. Fix plumbs `action_counts[4]` ({4, 3, 3, 3}) through pearl_1_atom_kernel signature and uses `batch_size × action_counts[b]` as the per-branch denominator, matching q_branch_stats_kernel's existing parameter pattern. Test #2 launcher updated to allocate action_counts_buf and pass its dev_ptr through the new parameter slot. Touched: `cuda_pipeline/pearl_1_atom_kernel.cu` (+1 param, denominator math), `cuda_pipeline/gpu_dqn_trainer.rs` (+1 launch arg in `launch_sp5_pearl_1_atom`), `tests/sp5_producer_unit_tests.rs` (+5 LOC for action_counts_buf + extra arg). cargo check + cargo build --release + cargo test --no-run all clean.
SP5 Task A1 — Pearl 1 atom-span + Q-stats GPU producers (Layer A, 2026-05-01): two new CUDA kernels (`q_branch_stats_kernel.cu`, `pearl_1_atom_kernel.cu`) land as Layer A additive producers that feed ISV slots [174..226) with per-branch C51 atom-span signals. No consumer migration in this commit — Layer B (atoms_update_kernel wiring) is a separate task. `q_branch_stats_update`: single-block 4-thread kernel (one thread per branch: Direction/Magnitude/Order/Urgency), reads `q_out_buf[B × 13]` row-major (confirmed layout), computes 3 serial passes over batch × branch-actions to produce (mean, max_abs_dev, var, entropy) per branch, writes 16 floats to `producer_step_scratch_buf[71..87)` (SCRATCH_Q_STATS=71; slot 70 = SP4 q_dir_grad_p99 was the last SP4 slot — collision-free). No atomicAdd per `feedback_no_atomicadd.md` — one thread per branch with independent scratch writes. `pearl_1_atom_update`: single-block 4-thread kernel, reads q_branch_stats scratch [71..87), reads current HEADROOM from ISV (Pearl A sentinel: ISV=0.0 → bootstrap to 6.0), reads `atoms_clip_count_buf[4]` (zero-init until Layer B wires atoms_update_kernel — clip_rate=0, headroom holds at bootstrap), runs headroom controller `new_headroom = clamp(headroom + 0.01*(clip_rate - 0.01), 2.0, 20.0)` (TARGET_CLIP_RATE=0.01 and HEADROOM_LR=0.01 are Invariant-1 structural anchors, not tuned constants), writes `(v_center, v_half, headroom, clip_rate)` to scratch [87..103) (SCRATCH_ATOM_OUT=87). `launch_sp5_pearl_1_atom` then fires 24 `launch_apply_pearls` calls: 4 groups × 4 contiguous Pearl 1 outputs (ISV[174..178)=v_center, [178..182)=v_half, [182..186)=headroom, [186..190)=clip_rate) looped, then 8 single-slot launches for per-branch entropy ([218..222)) and var ([222..226)) which are non-contiguous in scratch. Wiener offset: `(SP4_PRODUCER_COUNT=71 + (isv_slot − SP5_SLOT_BASE=174)) × 3` — correct SP5 block within wiener_state_buf. Buffer growth per `feedback_no_partial_refactor.md`: `wiener_state_buf` 213→543 floats ((71+110)×3), `producer_step_scratch_buf` 71→103 slots (SP4_PRODUCER_COUNT → SP5_SCRATCH_TOTAL), both in the same commit that adds the kernels that use them. New module-level constants: `SP5_WIENER_TOTAL_FLOATS=543`, `SP5_SCRATCH_TOTAL=103`, `SCRATCH_Q_STATS=71`, `SCRATCH_ATOM_OUT=87`. New struct fields: `q_branch_stats_kernel`, `pearl_1_atom_kernel` (CudaFunction), `atoms_clip_count_buf` (MappedI32Buffer[4], zero-init), `action_counts_buf` ([4,3,3,3]), `action_offsets_buf` ([0,4,7,10]). StateResetRegistry gains 7 new FoldReset entries: sp5_atom_v_center/v_half/headroom/clip_rate (ISV[174..190)), sp5_branch_entropy (ISV[218..222)), sp5_q_var_per_branch (ISV[222..226)), sp5_wiener_state (wiener_state_buf[213..543) — SP5 block, resets companion Wiener state with Pearl A sentinel per `pearl_first_observation_bootstrap.md`). Wire-up in training_loop.rs: `launch_sp5_pearl_1_atom()` called after `launch_sp4_param_group_oracles_all_groups` with `warn!` on error (Layer A — non-fatal until Layer B consumers land). Two GPU-only `#[ignore = "requires GPU"]` unit tests in `tests/sp5_producer_unit_tests.rs`: (1) uniform-input analytical ground truth for q_branch_stats (mean=2.0, max_abs_dev=0.0, entropy=log(n_actions)); (2) headroom bootstrap + controller step for pearl_1_atom (sentinel ISV=0→bootstrap 6.0, clip_rate=0, one step adjustment). No CPU compute per `feedback_no_cpu_compute_strict.md`, no HtoD/HtoH per `feedback_no_htod_htoh_only_mapped_pinned.md`, no atomicAdd per `feedback_no_atomicadd.md`, no stubs per `feedback_no_stubs.md`. Touched: `cuda_pipeline/q_branch_stats_kernel.cu` (new), `cuda_pipeline/pearl_1_atom_kernel.cu` (new), `build.rs` (+2 cubin entries), `cuda_pipeline/gpu_dqn_trainer.rs` (static cubins + SP5 constants + struct fields + buffer growth 213→543 + 71→103 + cubin loaders + buffer allocs + struct initializer + launch method + docstring), `trainers/dqn/state_reset_registry.rs` (+7 FoldReset entries + sp4_wiener_state description 213→543), `trainers/dqn/trainer/training_loop.rs` (+3 LOC wire-up after SP4 param_group_oracles), `tests/sp5_producer_unit_tests.rs` (new, 2 GPU-only tests). cargo check -p ml --lib clean (11 pre-existing warnings, none new). Validation deferred to one L40S smoke; expected sp5_atom_headroom stays [2.0,20.0], sp5_atom_v_half tracks C51 atom spread, no NaN flags.