foxhunt

Files

jgrusewski dd9c70df98 feat(sp20): Phase 3 Task 3.4 — replay buffer schema for per-bar aux_conf

Lands the producer→ring→consumer schema plumbing for per-bar aux_conf
(the K=2 peak-softmax-above-uniform confidence value the Phase 5
Aux→Q gate consumes at the Bellman target). Atomic across all struct
boundaries per `feedback_no_partial_refactor`.

Data flow (TWO struct boundaries, not one — plan errata Gap 11):

  ExperienceCollector  →  GpuExperienceBatch  →  insert_batch()
                            (.aux_conf field)      (8th arg)
       →  ring  →  sample()  →  GpuBatchPtrs  →  Trainer
                                  (.aux_conf_ptr)   (Phase 5 consumer)

Phase 5 lands the consumer (Aux→Q gate kernel at the Bellman target);
this commit is the producer-side schema plumbing only.

Spec §4.4 Phase 5 forward-reference contract:

  At each replay batch step:
    aux_conf  = ptrs.aux_conf_ptr[k]                 # SAMPLED bar
    threshold = ISV[AUX_CONF_THRESHOLD_INDEX]   # [0.01, 0.20]
    temp      = ISV[AUX_GATE_TEMP_INDEX]
    gate      = sigmoid((aux_conf - threshold) / temp)  # ∈ [0, 1]
    Q_target  = gate × Q_full + (1 - gate) × mean_a Q(s, a)

Producer (experience_env_step):
  - 1 new kernel arg `aux_conf_per_sample: float* [N×L×cf_mult]`.
  - Per-bar write at kernel entry: `aux_conf_per_sample[out_off] =
    sp20_compute_per_bar_aux_conf_k2(aux_logits + i*K)`.
  - CF slot write: `aux_conf_per_sample[cf_off] =
    aux_conf_per_sample[out_off]` (CF state shares the same env state
    so shares the same aux signal).
  - NULL-tolerant: defaults to 0.0f (K=2 uniform-prior sentinel).

GpuExperienceCollector:
  - +`aux_conf_per_sample: CudaSlice<f32>` field.
  - alloc with `total_output * cf_mult` for both on-policy + CF.
  - Threads as kernel arg of experience_env_step.
  - Clones into `GpuExperienceBatch.aux_conf` at end of
    collect_experiences_gpu (size `total = base_total × 2`).

GpuExperienceBatch: +`aux_conf: CudaSlice<f32>` field.

GpuReplayBuffer (crates/ml-dqn):
  - +`GpuBatchPtrs.aux_conf_ptr: u64`.
  - +ring storage `aux_conf: CudaSlice<f32>` [capacity].
  - +sample buffer `sample_aux_conf: CudaSlice<f32>` [max_batch_size].
  - +`trainer_aux_conf_ptr: u64` for direct path.
  - +`set_trainer_aux_conf_ptr` setter + `trainer_aux_conf_ptr`
    accessor (mirrors `set_isv_signals_ptr` pattern).
  - `insert_batch` adds 8th arg `aux_conf_in: &CudaSlice<f32>` —
    scatters via `scatter_insert_f32` (reuses the rewards/dones
    scatter kernel).
  - `sample_proportional` adds gather: direct-to-trainer if
    `trainer_aux_conf_ptr != 0`, else fallback into
    `sample_aux_conf`. Both paths populate `GpuBatchPtrs.aux_conf_ptr`.

Atomic caller updates (insert_batch arg from 7 to 8):
  - training_loop.rs:2522 (production)
  - gpu_residency.rs:77 (smoke)
  - performance.rs:144 (smoke)
  - training_stability.rs:154+201 (smoke ×2)
  - gpu_per_integration_test.rs:128 (integration)
  - sp15_phase1_oracle_tests.rs:2170+2189+2356 (oracle ×3)
  - 3 inline tests in gpu_replay_buffer.rs

Tests:
  - test_aux_conf_schema_round_trip (GPU): inserts 8 transitions with
    distinct aux_conf [0.0, 0.05, ..., 0.35]; samples 1; asserts the
    gathered slot value matches one of the inserted values
    (round-trip integrity invariant).
  - test_aux_conf_trainer_ptr_wiring_contract (CPU): asserts
    set_trainer_aux_conf_ptr / trainer_aux_conf_ptr setter+accessor
    behavior matches the SP15 Phase 3.5.5.b mirror pattern.

Verification:
  SQLX_OFFLINE=true cargo check -p ml --tests --features cuda    # green
  SQLX_OFFLINE=true cargo check -p ml-dqn --tests --features cuda # green
  SQLX_OFFLINE=true cargo test -p ml --lib sp20                  # 21/21 pass
  SQLX_OFFLINE=true cargo test -p ml-dqn test_aux_conf_trainer_ptr_wiring_contract  # CPU pass

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-10 08:56:19 +02:00

infra

…

plans

…

superpowers

docs(sp20): plan accuracy errata for Phase 2 (Tasks 2.0/2.1/2.2/2.3/2.4)

2026-05-10 00:30:33 +02:00

dqn-backward-nan-audit.md

fix(dqn): SP3 Mech 10 — ISV-driven h_s2 activation clamp + slot 49 diag

2026-04-30 17:26:25 +02:00

dqn-gpu-hot-path-audit.md