feat(sp15-p1.3.b-followup): per-env DD redesign — Path A env-0-canonical → Path B per-env tile + reduction

Closes the Phase 1.3.b deferred per-env redesign per
feedback_no_partial_refactor. Path A (env-0-canonical, commit 132609724)
made every downstream consumer read env-0's DD context; production envs
each have their own DD trajectory, so when env-3 was at 30% DD and
env-0 was at ATH, the model learning from env-3's transitions saw
dd_pct=0 and silently skipped the recovery shaping. Path B threads
each env's actual DD context through the reward shaping atomically:

(1) dd_state_kernel reshape — grid [n_envs, 1, 1], one thread per env,
    writes 6 scalars per env to a new per-env tile dd_state_per_env
    [n_envs * 6]. No more ISV scalar writes.
(2) NEW dd_state_reduce_kernel — single-block tree-reduce (no
    atomicAdd; BLOCK=256 with strided initial pass for n_envs up to
    32768 on H100). Mean-aggregates per-env tile → 6 scalar ISV slots
    [401..407) for HEALTH_DIAG diagnostic. Max-aggregates DD_PERSISTENCE
    → new slot DD_PERSISTENCE_MAX_INDEX=443 for plasticity injection
    trigger (one shared advantage-head ⇒ global firing ⇒ max-aggregate
    is the only correct rule). ISV_TOTAL_DIM 443→444; SP15_SLOT_END
    443→444; SP15_SLOT_COUNT 46→47.
(3) compute_sp15_final_reward_kernel migration — per-(i,t) per-env DD
    lookup via env_id = (idx % (N*L)) / L. Helpers sp15_dd_asymmetric_reward
    and sp15_dd_penalty migrated to take dd_pct + dd_current as scalar
    parameters (the kernel reads from the per-env tile, threads scalars
    in). On-policy + CF threads at the same (i,t) read the SAME tile
    entry (one DD trajectory per env, shared across slot kinds).
(4) plasticity_injection_kernel migration — persistence read switched
    from ISV[404] (mean) to ISV[443] (max). One set of advantage
    weights ⇒ ANY env exceeding the threshold should arm the gate.
(5) Per-env tile owned by GpuExperienceCollector (not the trainer) —
    the collector knows alloc_episodes (= n_envs); the trainer's
    batch_size is a different quantity. Reset to zero via the
    sp15_dd_state_per_env registry-arm dispatch.
(6) Per-step launch order: dd_state → dd_state_reduce →
    alpha_split_producer → final_reward, all on the same stream
    (CUDA serialises producer→consumer without explicit event sync).
(7) HEALTH_DIAG semantic shift (documented breaking change): slots
    401-406 now report cross-env mean, not env-0 value. For n_envs=1
    smoke configs the mean equals env-0's value (bit-stable migration).
(8) Layout fingerprint break: added markers DD_PERSISTENCE_MAX=443;
    ISV_TOTAL_DIM=444; DD_STATE_PER_ENV=sp15_phase_1_3_b_followup.
    Pre-followup checkpoints will not load (greenfield OK per spec Q1).
(9) 6 oracle tests migrated + 1 NEW behavioral test
    `dd_state_per_env_diverge_independently` — two-env config (env-0
    in recovery, env-1 deepening) verifies independent trajectories
    + mean-aggregate + max-aggregate semantics.

Phase 1.3.b-followup-B (separate split): dd_trajectory_decreasing_kernel
+ per_insert_pa migration is NOT in scope. The current Wave 4.3 reads
ISV[439] at insert-batch time (epoch end) — applied uniformly to ALL
inserted transitions (a known pre-existing limitation). Proper fix
requires per-(env, t) trajectory buffer [N*L] + env_id-aware lookup
in per_insert_pa via env_id = (j % (N*L)) / L. That's a different
contract change; splitting preserves no-partial-refactor within each
migration.

Atomic per feedback_no_partial_refactor: all 5 consumers of single-env-
canonical DD slots (final_reward kernel + plasticity kernel +
HEALTH_DIAG diagnostic + 6 oracle tests + new behavioral test) migrate
to per-env tile lookup in this commit; the new DD_PERSISTENCE_MAX
ISV slot lands with its sole consumer (plasticity).

Verified: SQLX_OFFLINE=true cargo check -p ml --features cuda clean;
cargo check -p ml --features cuda --tests clean;
CUDA_COMPUTE_CAP=86 cargo test -p ml --test sp15_phase1_oracle_tests
--features cuda -- --ignored: 17/17 oracle tests green (2 dd_state
incl. new per-env behavioral + 5 plasticity + 3 dd_trajectory + 6
final_reward + 1 per_sampler); cargo test -p ml --features cuda --lib:
947 pass / 12 fail HOLDS the 483cef454 baseline (no new regressions).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-07 09:35:58 +02:00
parent 483cef454c
commit 5b394f1035
14 changed files with 1004 additions and 227 deletions

View File

@@ -843,17 +843,41 @@ fn main() {
// kernel reads LobBar fields from synthetic markets and prod
// fxcache LOB (dev/prod parity per Q3).
"cost_net_sharpe_kernel.cu",
// SP15 Phase 1.3 (2026-05-06): drawdown reporting. Single-thread,
// single-block per-step state machine. Reads existing
// PS_PEAK_EQUITY (slot 7) and PS_PREV_EQUITY (slot 9) from the
// position state buffer (canonical equity tracking — NO new ISV
// slot, invariant verified during plan v2 critical review).
// Writes 6 ISV slots: DD_CURRENT (401), DD_MAX (402),
// DD_RECOVERY_BARS (403), DD_PERSISTENCE (404), CALMAR (405;
// floored max_dd for host composer = mean_pnl / value-here),
// DD_PCT (406). The 1e-4 calmar floor eliminates the saturation-
// at-100 artifact previously seen in the train-dd4xl HEALTH_DIAG.
// SP15 Phase 1.3 (2026-05-06) + 1.3.b-followup (2026-05-07):
// per-env drawdown state tracking. Per-env grid `[n_envs, 1, 1]`,
// one thread per env: each thread reads PS_PEAK_EQUITY (slot 7)
// and PS_PREV_EQUITY (slot 9) from its env's row of the position
// state buffer and writes 6 scalars to the per-env tile output
// `dd_state_per_env[env*6 + k]` for k ∈ {DD_CURRENT, DD_MAX,
// DD_RECOVERY_BARS, DD_PERSISTENCE, CALMAR (floored max_dd
// denominator), DD_PCT}. The 6 ISV scalar slots [401..407) are
// populated by the separate `dd_state_reduce_kernel` below
// (mean-aggregate across envs for HEALTH_DIAG diagnostic) plus
// the new `DD_PERSISTENCE_MAX_INDEX = 443` slot (max-aggregate
// for the plasticity-injection trigger). Path A → Path B
// migration: the original Phase 1.3.b kernel chose env-0 as the
// canonical DD observable and wrote ISV scalars directly; the
// followup-A redesign threads each env's actual DD context
// through the reward shaping per spec §6.3.
"dd_state_kernel.cu",
// SP15 Phase 1.3.b-followup (2026-05-07): DD state reduction.
// Single-block tree-reduce (no atomicAdd) of the per-env tile
// `dd_state_per_env[n_envs * 6]` into 6 scalar ISV slots
// [401..407) (mean-aggregate across envs — smooth across-env
// summary for HEALTH_DIAG diagnostic) plus the new
// DD_PERSISTENCE_MAX_INDEX = 443 slot (max-aggregate, consumed
// by `plasticity_injection_kernel` so the global advantage-head
// reset fires when ANY env's persistence exceeds the threshold:
// one set of advantage weights → global firing semantics →
// max-aggregate is the only correct rule). BLOCK=256 with the
// strided initial accumulation pattern handles n_envs up to
// production sizes (32768 envs on H100). Layout fingerprint
// marker `DD_STATE_PER_ENV=sp15_phase_1_3_b_followup` enforces
// synchrony between the producer's tile layout (in
// `dd_state_kernel.cu`) and this consumer's reads. Single
// kernel per cubin (mirrors the SP15 1.3 / 1.4 / 3.X
// single-source-to-cubin convention).
"dd_state_reduce_kernel.cu",
// SP15 Phase 1.4 + Wave 3a (2026-05-06): 4 constant-policy
// counterfactual baselines per spec §6.4. Single source file with
// 4 `extern "C" __global__` symbols (baseline_buyhold_kernel,