feat(sp15-p1.3.b-followup): per-env DD redesign — Path A env-0-canonical → Path B per-env tile + reduction
Closes the Phase 1.3.b deferred per-env redesign per feedback_no_partial_refactor. Path A (env-0-canonical, commit132609724) made every downstream consumer read env-0's DD context; production envs each have their own DD trajectory, so when env-3 was at 30% DD and env-0 was at ATH, the model learning from env-3's transitions saw dd_pct=0 and silently skipped the recovery shaping. Path B threads each env's actual DD context through the reward shaping atomically: (1) dd_state_kernel reshape — grid [n_envs, 1, 1], one thread per env, writes 6 scalars per env to a new per-env tile dd_state_per_env [n_envs * 6]. No more ISV scalar writes. (2) NEW dd_state_reduce_kernel — single-block tree-reduce (no atomicAdd; BLOCK=256 with strided initial pass for n_envs up to 32768 on H100). Mean-aggregates per-env tile → 6 scalar ISV slots [401..407) for HEALTH_DIAG diagnostic. Max-aggregates DD_PERSISTENCE → new slot DD_PERSISTENCE_MAX_INDEX=443 for plasticity injection trigger (one shared advantage-head ⇒ global firing ⇒ max-aggregate is the only correct rule). ISV_TOTAL_DIM 443→444; SP15_SLOT_END 443→444; SP15_SLOT_COUNT 46→47. (3) compute_sp15_final_reward_kernel migration — per-(i,t) per-env DD lookup via env_id = (idx % (N*L)) / L. Helpers sp15_dd_asymmetric_reward and sp15_dd_penalty migrated to take dd_pct + dd_current as scalar parameters (the kernel reads from the per-env tile, threads scalars in). On-policy + CF threads at the same (i,t) read the SAME tile entry (one DD trajectory per env, shared across slot kinds). (4) plasticity_injection_kernel migration — persistence read switched from ISV[404] (mean) to ISV[443] (max). One set of advantage weights ⇒ ANY env exceeding the threshold should arm the gate. (5) Per-env tile owned by GpuExperienceCollector (not the trainer) — the collector knows alloc_episodes (= n_envs); the trainer's batch_size is a different quantity. Reset to zero via the sp15_dd_state_per_env registry-arm dispatch. (6) Per-step launch order: dd_state → dd_state_reduce → alpha_split_producer → final_reward, all on the same stream (CUDA serialises producer→consumer without explicit event sync). (7) HEALTH_DIAG semantic shift (documented breaking change): slots 401-406 now report cross-env mean, not env-0 value. For n_envs=1 smoke configs the mean equals env-0's value (bit-stable migration). (8) Layout fingerprint break: added markers DD_PERSISTENCE_MAX=443; ISV_TOTAL_DIM=444; DD_STATE_PER_ENV=sp15_phase_1_3_b_followup. Pre-followup checkpoints will not load (greenfield OK per spec Q1). (9) 6 oracle tests migrated + 1 NEW behavioral test `dd_state_per_env_diverge_independently` — two-env config (env-0 in recovery, env-1 deepening) verifies independent trajectories + mean-aggregate + max-aggregate semantics. Phase 1.3.b-followup-B (separate split): dd_trajectory_decreasing_kernel + per_insert_pa migration is NOT in scope. The current Wave 4.3 reads ISV[439] at insert-batch time (epoch end) — applied uniformly to ALL inserted transitions (a known pre-existing limitation). Proper fix requires per-(env, t) trajectory buffer [N*L] + env_id-aware lookup in per_insert_pa via env_id = (j % (N*L)) / L. That's a different contract change; splitting preserves no-partial-refactor within each migration. Atomic per feedback_no_partial_refactor: all 5 consumers of single-env- canonical DD slots (final_reward kernel + plasticity kernel + HEALTH_DIAG diagnostic + 6 oracle tests + new behavioral test) migrate to per-env tile lookup in this commit; the new DD_PERSISTENCE_MAX ISV slot lands with its sole consumer (plasticity). Verified: SQLX_OFFLINE=true cargo check -p ml --features cuda clean; cargo check -p ml --features cuda --tests clean; CUDA_COMPUTE_CAP=86 cargo test -p ml --test sp15_phase1_oracle_tests --features cuda -- --ignored: 17/17 oracle tests green (2 dd_state incl. new per-env behavioral + 5 plasticity + 3 dd_trajectory + 6 final_reward + 1 per_sampler); cargo test -p ml --features cuda --lib: 947 pass / 12 fail HOLDS the483cef454baseline (no new regressions). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -843,17 +843,41 @@ fn main() {
|
||||
// kernel reads LobBar fields from synthetic markets and prod
|
||||
// fxcache LOB (dev/prod parity per Q3).
|
||||
"cost_net_sharpe_kernel.cu",
|
||||
// SP15 Phase 1.3 (2026-05-06): drawdown reporting. Single-thread,
|
||||
// single-block per-step state machine. Reads existing
|
||||
// PS_PEAK_EQUITY (slot 7) and PS_PREV_EQUITY (slot 9) from the
|
||||
// position state buffer (canonical equity tracking — NO new ISV
|
||||
// slot, invariant verified during plan v2 critical review).
|
||||
// Writes 6 ISV slots: DD_CURRENT (401), DD_MAX (402),
|
||||
// DD_RECOVERY_BARS (403), DD_PERSISTENCE (404), CALMAR (405;
|
||||
// floored max_dd for host composer = mean_pnl / value-here),
|
||||
// DD_PCT (406). The 1e-4 calmar floor eliminates the saturation-
|
||||
// at-100 artifact previously seen in the train-dd4xl HEALTH_DIAG.
|
||||
// SP15 Phase 1.3 (2026-05-06) + 1.3.b-followup (2026-05-07):
|
||||
// per-env drawdown state tracking. Per-env grid `[n_envs, 1, 1]`,
|
||||
// one thread per env: each thread reads PS_PEAK_EQUITY (slot 7)
|
||||
// and PS_PREV_EQUITY (slot 9) from its env's row of the position
|
||||
// state buffer and writes 6 scalars to the per-env tile output
|
||||
// `dd_state_per_env[env*6 + k]` for k ∈ {DD_CURRENT, DD_MAX,
|
||||
// DD_RECOVERY_BARS, DD_PERSISTENCE, CALMAR (floored max_dd
|
||||
// denominator), DD_PCT}. The 6 ISV scalar slots [401..407) are
|
||||
// populated by the separate `dd_state_reduce_kernel` below
|
||||
// (mean-aggregate across envs for HEALTH_DIAG diagnostic) plus
|
||||
// the new `DD_PERSISTENCE_MAX_INDEX = 443` slot (max-aggregate
|
||||
// for the plasticity-injection trigger). Path A → Path B
|
||||
// migration: the original Phase 1.3.b kernel chose env-0 as the
|
||||
// canonical DD observable and wrote ISV scalars directly; the
|
||||
// followup-A redesign threads each env's actual DD context
|
||||
// through the reward shaping per spec §6.3.
|
||||
"dd_state_kernel.cu",
|
||||
// SP15 Phase 1.3.b-followup (2026-05-07): DD state reduction.
|
||||
// Single-block tree-reduce (no atomicAdd) of the per-env tile
|
||||
// `dd_state_per_env[n_envs * 6]` into 6 scalar ISV slots
|
||||
// [401..407) (mean-aggregate across envs — smooth across-env
|
||||
// summary for HEALTH_DIAG diagnostic) plus the new
|
||||
// DD_PERSISTENCE_MAX_INDEX = 443 slot (max-aggregate, consumed
|
||||
// by `plasticity_injection_kernel` so the global advantage-head
|
||||
// reset fires when ANY env's persistence exceeds the threshold:
|
||||
// one set of advantage weights → global firing semantics →
|
||||
// max-aggregate is the only correct rule). BLOCK=256 with the
|
||||
// strided initial accumulation pattern handles n_envs up to
|
||||
// production sizes (32768 envs on H100). Layout fingerprint
|
||||
// marker `DD_STATE_PER_ENV=sp15_phase_1_3_b_followup` enforces
|
||||
// synchrony between the producer's tile layout (in
|
||||
// `dd_state_kernel.cu`) and this consumer's reads. Single
|
||||
// kernel per cubin (mirrors the SP15 1.3 / 1.4 / 3.X
|
||||
// single-source-to-cubin convention).
|
||||
"dd_state_reduce_kernel.cu",
|
||||
// SP15 Phase 1.4 + Wave 3a (2026-05-06): 4 constant-policy
|
||||
// counterfactual baselines per spec §6.4. Single source file with
|
||||
// 4 `extern "C" __global__` symbols (baseline_buyhold_kernel,
|
||||
|
||||
Reference in New Issue
Block a user