spec(dqn): review fixes — clarify hot-path scope, add Invariant 8 named dims

Self-review pass on the DQN v2 unified design spec (d13b53586). Fixes
applied inline:

Ambiguity fixes:
- §3 Invariant 3: explicit definition of "training step loop" (per-step
  code in fused_training::step_fused and captured graph children; NOT
  per-epoch / per-fold / per-checkpoint code).
- §2 Tier 1: "stable epochs" defined as epochs [warmup_end..run_end)
  with warmup_end from B.3's seed-phase decay or explicit flag.
- §4 A.2: migration mechanism clarified as fail-fast initially; helpers
  added only when a specific migration need arises (no speculative
  scaffolding).
- §4 A.3: N=5, K=6 stated as DEFAULTS with ≥ MINIMUMS per Invariant 6;
  resource budget flagged (~8-10 GPU-hours per validation pass).
- §4 B.3: "≥ 100K experiences" (minimum), warm-restart clarified as
  runtime condition not feature flag.
- §4 C.3: adaptive KL threshold mechanism made explicit (second ISV
  slot for threshold EMA, third for amplification multiplier).
- §4 C.6: cleanup timing explicit — old scaffolding removed in the SAME
  commit that migrates its last consumer (no deferred pass).
- §4 D.1: DQN-path-specific scope clarified; validation via grad-norm
  smoke + integration test.
- §4 D.6: plan_isv index naming made consistent with existing layout;
  coordinated state-layout migration commit called out.

New Invariant 8 — Named dimensions, not indices:
Every dimension, slot, offset, or semantic position in a multi-field
buffer has a named constant. Raw numeric indices (`[0]`, `[6]`, `[23]`)
appear ONLY in the definition site. Named constants specified for ISV
slots (existing), portfolio state ps[0..30), plan_isv[0..7), plan_params
[0..6), state-vector offsets, branch indices (BRANCH_DIR/MAG/ORD/URG),
action sub-indices (DIR_SHORT/HOLD/LONG/FLAT, MAG_QUARTER/HALF/FULL).
Enforcement: audit pass during A.1/A.2, lint via grep for raw-index
access outside definition modules.

Landing order revision:
- Dependency graph explicit (A.2 before new ISV slots, D.1 before
  D.2-4-8, state-layout changes in ONE coordinated commit).
- Spec decomposition note added — writing-plans may produce multiple
  sequential plans rather than one monolithic plan.

New doc tracked by pre-commit: docs/dqn-named-dims.md (Invariant 8).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs/superpowers/specs/2026-04-24-dqn-v2-unified-design.md

@@ -28,9 +28,9 @@ Each milestone has explicit exit criteria. Implementation does not declare "done
 
 ### Tier 1: Convergence discipline (primary, must pass first)
 
-- Multi-seed training (N=5 seeds) produces val-metric distribution with `std / mean < 0.15` on stable epochs (post-warmup).
-- Multi-fold walk-forward (K=6 folds) shows no fold-1 grad explosions, no Q saturation at `abs_half`, no cross-fold state-reset bugs.
-- Training halts self-terminates on any HEALTH_DIAG metric out-of-band (regression detection, A.4).
+- Multi-seed training (N ≥ 5 seeds) produces val-metric distribution with `std / mean < 0.15` on stable epochs. "Stable epochs" = epochs `[warmup_end .. run_end)` where `warmup_end` is determined by the seed-phase decay in B.3 (typically epochs 10–15 for a 60-epoch run) OR explicit flag `--warmup-end N` on the validation harness, whichever is later.
+- Multi-fold walk-forward (K ≥ 6 folds) shows no fold-1 grad explosions, no Q saturation at `abs_half`, no cross-fold state-reset bugs.
+- Training self-terminates on any HEALTH_DIAG metric out-of-band (regression detection, A.4).
 - No hot-path DtoH `memcpy` anywhere in the step loop (A.6 audit passes).
 
 ### Tier 2: Behavioural parity (val trades escape the Flat-trap)
@@ -51,7 +51,7 @@ Each milestone has explicit exit criteria. Implementation does not declare "done
 
 ## 3. Cross-cutting invariants (non-negotiable)
 
-Every feature added by this spec obeys all seven invariants. Violations are caught by pre-commit hooks and smoke tests.
+Every feature added by this spec obeys all eight invariants. Violations are caught by pre-commit hooks and smoke tests.
 
 ### Invariant 1 — ISV-driven adaptive bounds
 
@@ -67,6 +67,8 @@ Enforcement: every new module lands with a row in `docs/dqn-wire-up-audit.md` id
 
 ### Invariant 3 — GPU-only hot path
 
+**Definition of "training step loop":** every function called from `fused_training.rs::step_fused` or its descendants that executes once per training step (i.e. 178 times per epoch at current batch size), including kernel invocations inside the captured CUDA graph (`adam_grad_child`, `forward_child`, `aux_grad_child`, etc.) and any host-side orchestration between them. Per-epoch code (update_eval_v_range, update_grad_balance_targets if moved to cold path, etc.) is NOT the hot path. Per-fold and per-checkpoint code is never hot path.
+
 No `cudaMemcpy` / `memcpy_dtoh` / `memcpy_htod` calls inside the training step loop. Zero tolerance.
 
 The one allowed cross-boundary mechanism is **pinned + device-mapped memory**: allocated via `cuMemAllocHost + cuMemHostGetDevicePointer`. Kernel writes via the device pointer; host reads via the host pointer to the same physical page. No copy occurs. This is already the ISV pattern; it is promoted to the ONLY allowed per-step cross-boundary mechanism.
@@ -100,6 +102,24 @@ Every commit that adds or modifies a component updates:
 
 Pre-commit hook rejects component-adding commits that don't touch these docs.
 
+### Invariant 8 — Named dimensions, not indices
+
+Every dimension, slot, offset, or semantic position in a multi-field buffer has a **named constant** in its definition module. Production code references the name. Raw numeric indices (`[0]`, `[6]`, `[86]`) appear ONLY in the definition site — never in consumer code, kernels, or tests.
+
+Scope of this invariant:
+
+- **ISV slots:** names already exist (e.g. `V_CENTER_DIR_INDEX`). This invariant says ALL slot access uses the named constant. Spec's new slots (§6) each get a named constant.
+- **Portfolio state:** `ps[0]` through `ps[29]` → named constants in `state_layout.cuh`: `PS_VALUE`, `PS_POSITION`, `PS_CASH`, `PS_ENTRY_PRICE`, …, `PS_PLAN_TARGET_BARS` (23), etc. No kernel reads `ps[23]` directly.
+- **Plan ISV dims:** current `plan_isv[0..6)` → named: `PLAN_ISV_PROGRESS` (0), `PLAN_ISV_PNL_VS_TARGET` (1), `PLAN_ISV_PNL_VS_STOP` (2), `PLAN_ISV_ENTRY_CONVICTION` (3), `PLAN_ISV_CONVICTION_DRIFT` (4), `PLAN_ISV_REGIME_SHIFT` (5), plus D.6's new `PLAN_ISV_REMAINING_FRACTION` (6).
+- **Plan params:** `plan_params[0..6)` → named: `PLAN_PARAM_TARGET_BARS`, `PLAN_PARAM_PROFIT_TARGET`, `PLAN_PARAM_STOP_LOSS`, `PLAN_PARAM_SCALE_AGGRESSION`, `PLAN_PARAM_CONVICTION`, `PLAN_PARAM_ASYMMETRY`.
+- **State vector offsets:** `SL_MARKET_START`, `SL_OFI_START`, `SL_TLOB_START` (new, D.8), `SL_MTF_START`, `SL_PORTFOLIO_START`, `SL_PLAN_ISV_START` — all live in `state_layout.cuh`.
+- **Branch indices:** `BRANCH_DIR` (0), `BRANCH_MAG` (1), `BRANCH_ORD` (2), `BRANCH_URG` (3). Already partially named in Rust (via `[f32; 4]` arrays) but kernel code uses raw 0/1/2/3 — migrate.
+- **Action sub-indices:** `DIR_SHORT` (0), `DIR_HOLD` (1), `DIR_LONG` (2), `DIR_FLAT` (3); `MAG_QUARTER` (0), `MAG_HALF` (1), `MAG_FULL` (2).
+
+Rationale: raw indices are the other half of the "hardcoded tuned constant" problem — they encode semantic position as a number. A reader can't tell what `ps[23]` means without grepping. Named constants make the meaning local to the reader.
+
+Enforcement: audit pass during A.1 (State Reset Registry) and A.2 (ISV Contract) picks up and names every raw index. New constants added to `docs/dqn-named-dims.md` — a simple table of name → index → purpose — and lint via grep: any `*.rs` or `*.cu` file that accesses a slot by raw index outside the definition module flags a pre-commit warning.
+
 ---
 
 ## 4. Architecture
@@ -139,20 +159,21 @@ Design:
   - Consumer list (which modules read)
   - Enforced via module visibility + a single audit table in `docs/isv-slots.md`.
 - ISV slot `[0]` reserved as schema version (u32 bumped on any slot-layout change).
-- Checkpoint load: reads ISV[0] version, migrates or fails explicitly — no silent schema drift.
-- Reserve slots [36..48) for second-order ISV signals (Part D, diagnostic-on-ISV, plan temporal dynamics).
+- Checkpoint load: reads ISV[0] version. **Initial behaviour is fail-fast** — mismatch triggers explicit error with instructions. Migration helpers (slot-remap functions) added only when a specific migration need arises; not built speculatively.
+- ISV slot allocation laid out in full in §6. New ISV slots claimed in this spec fit within the `[37..72)` range; older allocations `[0..37)` unchanged.
 
 #### A.3 Multi-Seed / Multi-Fold Validation Harness
 
 Current: one seed, one fold triggered by `argo-train.sh`, metrics eyeballed. Signal-to-noise terrible.
 
-Design: `argo-train.sh --multi-seed N --folds K` spawns `N × K` training jobs from a single submission:
+Design: `argo-train.sh --multi-seed N --folds K` spawns `N × K` training jobs from a single submission. Defaults: `N=5`, `K=6`. Per Invariant 6 these are minimums; running higher is always allowed.
 
 - Per-epoch metric aggregation across seeds: mean ± 2σ, median, worst-case.
 - HEALTH_DIAG emits cross-seed stats at epoch boundary.
 - Exit criteria use **pessimistic** estimator (mean − 2σ) not mean.
 - `norm_stats_foldN.json` merged across seeds to produce `norm_stats_fold_aggregate.json`.
 - Job orchestration via Argo Workflow DAG (existing pattern in `infra/k8s/argo/train-template.yaml`).
+- Resource budget: with N=5, K=6, each job costing ~15-20min L40S time, the multi-seed harness costs ~8-10 GPU-hours per validation pass. Acceptable for pre-merge validation; not for every commit's smoke.
 
 #### A.4 Regression Detection
 
@@ -249,7 +270,7 @@ When the policy is trading at task-normal rate, novelty is 0 and the bonus is in
 
 Current: PER buffer starts empty, fills via Boltzmann-driven DQN trajectories. At t=0 those trajectories are near-random but magnitude-biased toward the scripted plan MLP output, which produces mostly-losing trades → CQL learns "trading is bad" from a biased sample.
 
-Design: before DQN training starts, seed the PER buffer with ~100K experiences drawn from a **portfolio of scripted policies**:
+Design: before DQN training starts, seed the PER buffer with **≥ 100K experiences** (exact count sized to fill at least one PER batch × `REPLAY_SEED_EPOCHS` worth of training steps; typical is 100K-200K) drawn from a **portfolio of scripted policies**:
 
 - 40% pure uniform random (all 108 factored actions equiprobable, respecting dir/mag constraints).
 - 20% momentum-1 / momentum-3 / momentum-5 (simple N-bar momentum entry, fixed magnitude).
@@ -258,7 +279,7 @@ Design: before DQN training starts, seed the PER buffer with ~100K experiences d
 
 Seed trajectories stored with lower initial PER priority than new DQN trajectories, displaced by DQN experiences over roughly 10–15 epochs. CQL alpha is coupled to the seed-phase decay — see C.5.
 
-Per `feedback_no_feature_flags.md`, this is NOT a feature flag. The seed is always run for a fresh training job. Warm-restart from a checkpoint skips the seed step.
+**Not a feature flag** — the seed step is unconditional for a fresh training job. The only exception is warm-restart from a checkpoint, where the replay buffer is restored from the checkpoint and the seed step is implicit in the checkpoint's history. This is a runtime condition (checkpoint present / absent), not a config toggle. Per `feedback_no_feature_flags.md`.
 
 #### B.4 Adaptive plan-MLP activation threshold
 
@@ -301,9 +322,9 @@ Enables "did micro_reward actually contribute this epoch or is it silently zero?
 
 Current: training and val state distributions can diverge structurally (plan_isv was OOD for months before we caught it). Only detectable post-hoc.
 
-Design: new ISV slot `STATE_KL_TRAIN_VAL_EMA` — running estimate of distributional divergence between train-state and val-state on a subset of state dimensions (portfolio + plan + ISV features). Computed at val backtest time by comparing per-dimension moments (mean, std).
+Design: new ISV slot `STATE_KL_TRAIN_VAL_EMA` — running estimate of distributional divergence between train-state and val-state on a subset of state dimensions (portfolio + plan + ISV features). Computed at val backtest time by comparing per-dimension moments (mean, std), aggregated via a KL-like summary over the monitored dimensions.
 
-When `KL > threshold` (also ISV-tracked, adaptive): training-time reward shaping (B.1, B.2) gets amplified automatically. Creates a feedback loop where distribution shift is self-correcting via training emphasis.
+**Adaptive threshold:** a second ISV slot `STATE_KL_THRESHOLD_EMA` tracks the EMA of `STATE_KL_TRAIN_VAL_EMA` itself. "KL > threshold" means KL is ≥ 2× its own EMA — i.e. "currently divergent relative to historical baseline", self-calibrating. When the threshold fires, a third ISV slot `STATE_KL_AMPLIFICATION` is bumped from 1.0 toward 2.0 (bounded), which multiplies B.1 and B.2's reward magnitudes. Decays back toward 1.0 as KL normalises. All three slots are scratchpad (slot [58] and [63..65) in §6).
 
 #### C.4 Temporal timing bonus
 
@@ -365,7 +386,7 @@ Benefits:
 - Fire-rate auditing is built-in (addresses `controller_activity` smoke's assertion that load-bearing controllers shouldn't fire in >50% of epochs).
 - Standardised diagnostics — the HEALTH_DIAG `controllers [anti_lr=true gamma=false clip=true …]` line becomes derived state, not manually-coded.
 
-Biggest-scope piece in the spec. Executed as a series of internal commits (one per existing controller migrated to the protocol) before a final commit that removes the old scaffolding.
+Biggest-scope piece in the spec. Executed as a series of internal commits — one per existing controller migrated to the protocol. Old scaffolding (the ad-hoc update functions being replaced) is removed in the SAME commit that migrates its last consumer to the new protocol, not in a deferred cleanup pass. Per `feedback_no_partial_refactor.md` — when a contract changes, every consumer migrates together.
 
 ### Part D — Temporal Architecture
 
@@ -375,11 +396,13 @@ Current: Mamba2 + Liquid-TC in trunk; plan_isv state machine; conviction drift;
 
 Current: forward works, backward stubbed.
 
-Design: properly wire Mamba2 backward — state-space gradients dx, dA, dB, dC through the recurrence. Substantial CUDA kernel work but load-bearing: every temporal improvement downstream assumes temporal learning actually propagates end-to-end.
+Design: properly wire Mamba2 backward for the **DQN path specifically** — state-space gradients dx, dA, dB, dC through the recurrence as consumed by the DQN trunk. Substantial CUDA kernel work but load-bearing: every temporal improvement downstream assumes temporal learning actually propagates end-to-end.
 
-Scope: fused backward kernel covering `mamba2_temporal_kernel.cu`. Validated via grad-norm smoke test (perturbation grad-check against finite difference).
+Scope: fused backward kernel covering `mamba2_temporal_kernel.cu` for the DQN-integration call sites only. The same kernel infrastructure can be reused for supervised Mamba2 training, but that is not in this spec's scope.
 
-This closes task #76 for the DQN path; TFT backward (also task #76) covered in Part E if TFT is adopted as auxiliary head.
+Validation: grad-norm smoke test (perturbation grad-check against finite difference) on a small fixture; integration test verifying that DQN training with Mamba2 backward produces non-zero gradients propagating into Mamba2 weights over several training steps.
+
+This closes task #76 for the DQN path; TFT backward (task #76's other half) is addressed in Part E only if/when TFT is adopted as an auxiliary head.
 
 #### D.2 Per-branch gamma
 
@@ -429,16 +452,20 @@ Smoother transitions preserve learned temporal signals across folds without comp
 
 #### D.6 Plan temporal dynamics enrichment
 
-Current `plan_isv[6]` dimensions: progress, pnl-vs-target, pnl-vs-stop, entry-conviction, conviction-drift, regime-shift.
+Current plan_isv has 6 dimensions `[0..6)`: progress, pnl-vs-target, pnl-vs-stop, entry-conviction, conviction-drift, regime-shift. That is `SL_PORTFOLIO_PLAN_DIM = 6` in `state_layout.cuh` and state positions `[86..92)` in the 96-dim state vector.
 
-Design: extend to `plan_isv[7]` — add a new dimension:
+Design: extend plan_isv to 7 dimensions `[0..7)` — add a new dimension at index 6:
 
 ```
-plan_isv[6] = (plan_target_bars − hold_time) / plan_target_bars
-            = remaining-fraction, clamped to [0, 1]
+plan_isv[6] = clamp((plan_target_bars − hold_time) / plan_target_bars, 0.0, 1.0)
+            = remaining-fraction
 ```
 
-Directly exposes temporal pressure to the policy ("my plan is 80% done, should I take the profit I have?"). Trivial to compute; state-layout version bump.
+Directly exposes temporal pressure to the policy ("my plan is 80% done, should I take the profit I have?"). Trivial to compute. Requires coordinated change:
+
+- `SL_PORTFOLIO_PLAN_DIM = 6 → 7` in `state_layout.cuh`.
+- State vector grows from 96 to 97 dimensions (the new plan_isv[6] slots into position [92]; everything downstream shifts by one OR we leave a pad slot and account for it). Decision during impl: prefer padded no-shift to minimise downstream reshuffling.
+- Schema version bump (A.2) paired with D.3 and D.8's similar layout impacts; all three land as one coordinated state-layout migration commit rather than three.
 
 #### D.7 Liquid Time-constant `liquid_mod` audit
 
@@ -518,18 +545,43 @@ Deliverable: `docs/ml-supervised-to-dqn-concept-audit.md`.
 
 ### Landing order (suggested, non-mandatory within spec)
 
-1. Part A.1, A.2, A.6 (state registry + ISV contract + hot-path audit — zero functional change but establishes the substrate).
-2. A.5 orphan audit (discovery — zero functional change).
-3. C.6 adaptive controller unification — big refactor but non-functional at the surface.
-4. C.1, C.2, C.3 (quantile atoms, reward attribution, KL divergence signal — surgical additions).
-5. B.1, B.2, B.3, B.4 (reward shaping + replay seed + adaptive plan threshold — behavioural change).
-6. D.1, D.2, D.3 (temporal infrastructure — Mamba2 backward, per-branch gamma, horizon-decomposed V).
-7. D.4, D.5, D.6, D.7, D.8 (temporal reward, soft transitions, plan dynamics, liquid audit, TLOB wire).
-8. E.1–E.8 (supervised concept adoption; each a sub-PR).
-9. A.3, A.4 (multi-seed harness + regression detection — validation substrate, no functional change to model).
-10. Final validation pass: multi-seed × multi-fold across Tier 1, 2, 3 exit criteria.
+Dependencies to respect:
 
-Not all must be sequential — many are independent and can parallelise across worktrees.
+- A.2 (ISV contract) before ANY feature claiming new ISV slots.
+- A.1 (reset registry) before D.5 (soft fold transitions — extends the registry).
+- D.1 (Mamba2 backward) before D.2, D.3, D.4, D.8 (anything that relies on temporal gradients propagating).
+- C.6 (adaptive controller unification) before D.2 (per-branch gamma), D.7 (liquid audit) — these use the controller protocol.
+- B.3 (replay seed) before C.5 (CQL α-seed coupling — needs the seed-phase signal).
+- State-layout changes (D.3 horizon-decomposed V, D.6 plan[6], D.8 TLOB features) land as ONE coordinated commit with a single schema-version bump, not three.
+
+Suggested order respecting dependencies:
+
+1. **Substrate (zero functional change):** A.1, A.2, A.5, A.6.
+2. **Refactor (non-functional):** C.6 adaptive controller unification.
+3. **Surgical atoms:** C.1 quantile atom support.
+4. **Core temporal:** D.1 Mamba2 backward — unlocks D.2–D.8.
+5. **Temporal features:** D.2 per-branch γ, D.5 soft fold transitions, D.7 liquid audit.
+6. **Coordinated state-layout migration:** D.3 + D.6 + D.8 in one commit.
+7. **Reward shaping:** B.1, B.2, B.4 + C.4, D.4 (all reward components together since they share ISV slots).
+8. **Replay warm-start:** B.3 + C.5 (coupled per C.5 design).
+9. **Diagnostics:** C.2 reward attribution, C.3 KL divergence signal.
+10. **Supervised concepts:** E.1–E.8 (each a sub-PR, in dependency order determined by E audit).
+11. **Validation substrate:** A.3 multi-seed harness, A.4 regression detection, A.4.1 nsys harness.
+12. **Final validation pass:** multi-seed × multi-fold across Tier 1, 2, 3 exit criteria.
+
+Many items in the same step are independent and parallelise across worktrees. The order is suggested; writing-plans may adjust based on concrete work estimation.
+
+### Spec decomposition for writing-plans
+
+This spec is large. The writing-plans skill may produce **multiple sequential implementation plans** rather than one monolithic plan — for example:
+
+- Plan 1: Substrate + refactor (items 1–3 above).
+- Plan 2: Temporal core (items 4–6).
+- Plan 3: Behavioural (items 7–9).
+- Plan 4: Supervised concept audit (item 10) — one mini-plan per E sub-item.
+- Plan 5: Validation substrate + final pass (items 11–12).
+
+Each plan ships independently; all plans are subordinate to this spec's invariants. Invariants enforced across plans via the pre-commit hooks (Invariant 7).
 
 ---
 
@@ -570,6 +622,7 @@ New `ISV_TOTAL_DIM` = 72 (from 37). Network-facing `ISV_NETWORK_DIM` remains 23
 - `docs/isv-slots.md` — ISV slot allocation registry with type/producer/consumers (A.2, Invariant 7).
 - `docs/dqn-gpu-hot-path-audit.md` — cross-boundary call classification (Invariant 3, 7).
 - `docs/ml-supervised-to-dqn-concept-audit.md` — Part E deliverable.
+- `docs/dqn-named-dims.md` — named-constant registry (Invariant 8).
 - `config/metric-bands.toml` — regression-detection bands (A.4, Invariant 7).
 
 Pre-commit hook rejects component-adding commits that don't touch the relevant audit docs.