# DQN v2 — Unified Integrated Policy System

**Date:** 2026-04-24
**Status:** Design (awaiting user review before writing-plans)
**Scope:** DQN model only — consolidates every lesson from the val-Flat-collapse investigation, the task #92 gradient-pathology triad, the ISV v-range unification work, the f64→f32 ABI-trap cleanup, and the long-running hard lessons about convergence, fold boundaries, orphan features, and hardcoded tuning.

---

## 1. Purpose

This spec is not a patch to a specific failure mode. It is a **consolidation of the Foxhunt DQN architecture into one properly integrated system**, synthesising the concepts we already built in supervised, the adaptive controllers we accreted over many sessions, the ISV signal-bus pattern that emerged from the Apr-23 bundle, and the convergence-discipline lessons from multiple fold-1 grad explosions.

Prior work bolted mechanisms on as problems surfaced. This spec replaces that with a single integrated design that:

- Obeys a small set of non-negotiable invariants (§3).
- Wires every supervised concept the project has built into its intended DQN consumer path (Parts D, E).
- Unifies the many adaptive controllers under one protocol (Part C.6).
- Makes convergence the primary guardrail, not an after-thought (Part A).
- Eliminates orphan code, hot-path DtoH copies, and hardcoded tuning constants as a design principle (§3).

The spec lands as one coordinated rollout. Internal commits during implementation are permitted (agents in worktrees, tree clean at every merge to main). Every commit respects the invariants.

---

## 2. Success criteria (tiered milestones)

Each milestone has explicit exit criteria. Implementation does not declare "done" until ALL tiers pass under the multi-seed / multi-fold harness (§4.A.3).

### Tier 1: Convergence discipline (primary, must pass first)

- Multi-seed training (N ≥ 5 seeds) produces val-metric distribution with `std / mean < 0.15` on stable epochs. "Stable epochs" = epochs `[warmup_end .. run_end)` where `warmup_end` is determined by the seed-phase decay in B.3 (typically epochs 10–15 for a 60-epoch run) OR explicit flag `--warmup-end N` on the validation harness, whichever is later.
- Multi-fold walk-forward (K ≥ 6 folds) shows no fold-1 grad explosions, no Q saturation at `abs_half`, no cross-fold state-reset bugs.
- Training self-terminates on any HEALTH_DIAG metric out-of-band (regression detection, A.4).
- No hot-path DtoH `memcpy` anywhere in the step loop (A.6 audit passes).

### Tier 2: Behavioural parity (val trades escape the Flat-trap)

- Val trade count within factor-of-2 of training trade count (approximately >= 1% of val bars positioned, vs. the current 0.01%).
- `plan_isv` `active_frac` in val > 0.2 on average across the window (currently 0.000 — plans structurally fail to activate).
- No systematic bias toward Hold or Flat direction argmax (entropy of argmax direction distribution > 0.8 of uniform).

### Tier 3: Profitability (val metric positive and stable)

- Annualised val Sharpe > 1.0 on the validation window.
- Val WinRate ≥ 52% on trades > 500/window.
- Profit Factor ≥ 1.1 stable across the multi-seed runs.

**No tier is skippable.** Tier 3 without Tier 1 means the result is not reproducible. Tier 2 without Tier 1 means the val-trade-count was a lucky seed. Tier 1 without Tier 2 means convergence is clean but the policy is uselessly conservative.

---

## 3. Cross-cutting invariants (non-negotiable)

Every feature added by this spec obeys all nine invariants. Violations are caught by pre-commit hooks and smoke tests.

### Invariant 1 — ISV-driven adaptive bounds

All adaptive clamps, thresholds, targets, and schedules live in the ISV signal bus and are read by consumers at runtime. No hardcoded multipliers, no tuned `clamp(0.25, 4.0)`, no `4 × median`, no `10 × q_gap`. Authority: `feedback_isv_for_adaptive_bounds.md`.

The single carve-out is numerical-safety hard limits: values that express dimensional or numerical-stability reality (e.g. `abs_half = 0.5 × (v_max − v_min)`, `1/limit` for a safety floor). These are documented as such at the constant-definition site.

### Invariant 2 — Wire-It-Up

Every module, feature, or kernel built is wired into its intended production consumer path in the same commit. Orphan code — compiles but no production path consumes it — is either wired or deleted. Authority: `feedback_wire_everything_up.md`.

Enforcement: every new module lands with a row in `docs/dqn-wire-up-audit.md` identifying its consumer. Pre-commit hook rejects commits that add a pub module without updating the audit doc.

### Invariant 3 — GPU-only hot path

**Definition of "training step loop":** every function called from `fused_training.rs::step_fused` or its descendants that executes once per training step (i.e. 178 times per epoch at current batch size), including kernel invocations inside the captured CUDA graph (`adam_grad_child`, `forward_child`, `aux_grad_child`, etc.) and any host-side orchestration between them. Per-epoch code (update_eval_v_range, update_grad_balance_targets if moved to cold path, etc.) is NOT the hot path. Per-fold and per-checkpoint code is never hot path.

No `cudaMemcpy` / `memcpy_dtoh` / `memcpy_htod` calls inside the training step loop. Zero tolerance.

The one allowed cross-boundary mechanism is **pinned + device-mapped memory**: allocated via `cuMemAllocHost + cuMemHostGetDevicePointer`. Kernel writes via the device pointer; host reads via the host pointer to the same physical page. No copy occurs. This is already the ISV pattern; it is promoted to the ONLY allowed per-step cross-boundary mechanism.

`memcpy` DtoH/HtoD is permitted only at cold path: checkpoint save/load, fold transitions, validation-window reshuffle at epoch boundaries. Every such call is annotated with `// cold-path: ok — <reason>` and audited in `docs/dqn-gpu-hot-path-audit.md`.

Authority: `feedback_gpu_cpu_roundtrip.md` (extended with zero-tolerance language in this spec).

### Invariant 4 — Pinned-only host memory

Any host-resident memory that backs a GPU data structure is pinned + device-mapped via `cuMemAllocHost_v2 + cuMemHostGetDevicePointer_v2`. Regular `malloc` for GPU-adjacent buffers is forbidden. UVM (`cudaMallocManaged`) is forbidden (page-fault stalls in the hot path).

### Invariant 5 — Diagnostic-first

Every adaptive controller, every reward component, every new state partition, every new ISV slot emits HEALTH_DIAG-compatible telemetry. If it's adaptive or consequential, it's diagnostic.

Enforcement: `controller_activity` and `reward_component_audit` smoke tests extended with coverage assertions. Missing diagnostics fail smoke.

### Invariant 6 — Convergence discipline (primary)

No feature is "done" until it passes multi-seed (N ≥ 5) × multi-fold (K ≥ 6) under the validation harness (§4.A.3). "It worked on one run" is not a proof.

### Invariant 7 — Wire-up + diagnostic audit per-commit

Every commit that adds or modifies a component updates:

- `docs/dqn-wire-up-audit.md` — consumer-path mapping per module.
- `docs/isv-slots.md` — ISV slot allocations with type, producer, consumers.
- `docs/dqn-gpu-hot-path-audit.md` — every cross-boundary call classified.
- `config/metric-bands.toml` — regression-detection bands for new metrics.

Pre-commit hook rejects component-adding commits that don't touch these docs.

### Invariant 8 — Named dimensions, not indices

Every dimension, slot, offset, or semantic position in a multi-field buffer has a **named constant** in its definition module. Production code references the name. Raw numeric indices (`[0]`, `[6]`, `[86]`) appear ONLY in the definition site — never in consumer code, kernels, or tests.

Scope of this invariant:

- **ISV slots:** names already exist (e.g. `V_CENTER_DIR_INDEX`). This invariant says ALL slot access uses the named constant. Spec's new slots (§6) each get a named constant.
- **Portfolio state:** `ps[0]` through `ps[29]` → named constants in `state_layout.cuh`: `PS_VALUE`, `PS_POSITION`, `PS_CASH`, `PS_ENTRY_PRICE`, …, `PS_PLAN_TARGET_BARS` (23), etc. No kernel reads `ps[23]` directly.
- **Plan ISV dims:** current `plan_isv[0..6)` → named: `PLAN_ISV_PROGRESS` (0), `PLAN_ISV_PNL_VS_TARGET` (1), `PLAN_ISV_PNL_VS_STOP` (2), `PLAN_ISV_ENTRY_CONVICTION` (3), `PLAN_ISV_CONVICTION_DRIFT` (4), `PLAN_ISV_REGIME_SHIFT` (5), plus D.6's new `PLAN_ISV_REMAINING_FRACTION` (6).
- **Plan params:** `plan_params[0..6)` → named: `PLAN_PARAM_TARGET_BARS`, `PLAN_PARAM_PROFIT_TARGET`, `PLAN_PARAM_STOP_LOSS`, `PLAN_PARAM_SCALE_AGGRESSION`, `PLAN_PARAM_CONVICTION`, `PLAN_PARAM_ASYMMETRY`.
- **State vector offsets:** `SL_MARKET_START`, `SL_OFI_START`, `SL_TLOB_START` (new, D.8), `SL_MTF_START`, `SL_PORTFOLIO_START`, `SL_PLAN_ISV_START` — all live in `state_layout.cuh`.
- **Branch indices:** `BRANCH_DIR` (0), `BRANCH_MAG` (1), `BRANCH_ORD` (2), `BRANCH_URG` (3). Already partially named in Rust (via `[f32; 4]` arrays) but kernel code uses raw 0/1/2/3 — migrate.
- **Action sub-indices:** `DIR_SHORT` (0), `DIR_HOLD` (1), `DIR_LONG` (2), `DIR_FLAT` (3); `MAG_QUARTER` (0), `MAG_HALF` (1), `MAG_FULL` (2).

Rationale: raw indices are the other half of the "hardcoded tuned constant" problem — they encode semantic position as a number. A reader can't tell what `ps[23]` means without grepping. Named constants make the meaning local to the reader.

Enforcement: audit pass during A.1 (State Reset Registry) and A.2 (ISV Contract) picks up and names every raw index. New constants added to `docs/dqn-named-dims.md` — a simple table of name → index → purpose — and lint via grep: any `*.rs` or `*.cu` file that accesses a slot by raw index outside the definition module flags a pre-commit warning.

### Invariant 9 — No deferred work, no stubs, no TODO/FIXME

Every commit lands complete. No stubs (functions that return placeholder values instead of doing the work), no `TODO`, `FIXME`, `XXX`, `HACK`, or `TBD` markers in production code, no "we'll fix this later" deferrals, no half-finished implementations.

**If it can't be finished in this commit, it doesn't start in this commit.** Either the feature is done (behaviourally correct, wired, tested, diagnostic-emitting) or the feature doesn't land. No partial state.

Why this matters for a GPU RL trainer specifically: stubs that silently return placeholder Q-values, TODO-marked reward components that return zero, FIXME'd gradient paths that skip computation — these **train the network on semantic emptiness**. The model "learns" that certain inputs don't matter because the broken path reinforces ignoring them. Hours of L40S GPU time are burned optimising against a training signal that's partly fake. The bug is often invisible in metrics because training "converges" to something — just not to the policy we wanted.

Authority: `feedback_no_stubs.md`, `feedback_no_todo_fixme.md`, `feedback_no_quickfixes.md`. This invariant elevates those three memory rules to first-class spec enforcement.

Enforcement: pre-commit hook greps added code for `TODO|FIXME|XXX|HACK|TBD|unimplemented!|todo!` — any match rejects the commit. Stub detection is partly linted by the existing hook (`grep -n "return 0.0\|return 1.0\|return 0.5"` with `// ok: <reason>` suppression); extended to cover new fallback patterns as they emerge.

**Applies to spec implementation commits only.** Legit open questions in the spec itself (§8) are fine — they are design-time decisions documented with their decision criteria, not implementation-time stubs.

---

## 4. Architecture

The spec is decomposed into five Parts. All Parts land in one coordinated rollout; internal commits during implementation are permitted (worktree isolation for parallel agents, sequential merges to main).

Parts A and B are load-bearing for everything else. Part A establishes the convergence substrate. Part B escapes the Flat-trap. Parts C, D, E are refinement, temporal unification, and supervised-concept reuse respectively.

### Part A — Foundation (convergence substrate)

#### A.1 State Reset Registry

Current state: some state resets at fold boundaries (`eval_v_range`, target network hard copy), some persists (learned weights, Adam momentum, ISV history), some is undefined. The Fold-1 grad explosion (task #31) and the fold-boundary gap (task #84) trace to this ambiguity.

Design: every piece of training-time state is classified into one of four categories via a typed `StateResetRegistry`:

| Category | Meaning | Example |
|---|---|---|
| `FoldReset` | Cleared at fold boundary | Kelly stats, plan_state, eval_q_*_ema, v-range ISV slots |
| `WindowReset` | Cleared at validation-window boundary | Portfolio state, hold_time, entry_price |
| `SoftReset(decay_bars)` | Anneals toward bootstrap over decay_bars at fold boundary | adaptive_gamma, ISV grad-balance targets |
| `TrainingPersist` | Survives across folds (learned weights) | Network params, target network, Adam momentum |
| `SchemaContract` | Version-stable across runs | ISV slot allocations themselves |

`reset_fold_state(fold_idx)` iterates the registry. Adding new state requires registering it with a category. Compile-time enforcement where possible via module visibility on a typed registry struct; runtime assertion in constructor.

See D.5 for the SoftReset motivation (some temporal state benefits from annealing, not hard cuts).

#### A.2 ISV Slot Contract + Schema Versioning

Current: 37 slots, each `pub const X_INDEX: usize = N`. No producer/consumer contract, no version, no migration path.

Design:

- Introduce `IsvSlot<T>` typed handles with:
  - Producer identity (which module writes)
  - Consumer list (which modules read)
  - Enforced via module visibility + a single audit table in `docs/isv-slots.md`.
- ISV slot `[0]` reserved as **layout fingerprint**: a `u64` structural hash computed by `const fn` over the list of `(slot_name, slot_index)` pairs at compile time. The hash changes automatically whenever any slot is added, removed, reordered, or renamed — no human action required, and no ordered "version space" exists.
- Checkpoint load: reads ISV[0] fingerprint. **Fail-fast only** — any mismatch errors out with "checkpoint layout does not match current code; retrain required". **No migration functions are ever added.** The error message does not mention or invite migration as an option. This is aligned with `feedback_no_legacy_aliases.md` and `feedback_no_partial_refactor.md`: when a contract changes, every consumer migrates together and old artefacts are invalidated.
- Why a fingerprint, not a version number: an integer "schema version" semantically implies a family of coexisting versions with upgrade paths between them. A structural hash has no ordered space and no natural pairing, so there is no natural place to write a `migrate_v1_to_v2` function. The forbidden backward-compat pattern becomes structurally unwritable, not merely discouraged.
- ISV slot allocation laid out in full in §6. New ISV slots claimed in this spec fit within the `[37..72)` range; older allocations `[0..37)` unchanged.

#### A.3 Multi-Seed / Multi-Fold Validation Harness

Current: one seed, one fold triggered by `argo-train.sh`, metrics eyeballed. Signal-to-noise terrible.

Design: `argo-train.sh --multi-seed N --folds K` spawns `N × K` training jobs from a single submission. Defaults: `N=5`, `K=6`. Per Invariant 6 these are minimums; running higher is always allowed.

- Per-epoch metric aggregation across seeds: mean ± 2σ, median, worst-case.
- HEALTH_DIAG emits cross-seed stats at epoch boundary.
- Exit criteria use **pessimistic** estimator (mean − 2σ) not mean.
- `norm_stats_foldN.json` merged across seeds to produce `norm_stats_fold_aggregate.json`.
- Job orchestration via Argo Workflow DAG (existing pattern in `infra/k8s/argo/train-template.yaml`).
- Resource budget: with N=5, K=6, each job costing ~15-20min L40S time, the multi-seed harness costs ~8-10 GPU-hours per validation pass. Acceptable for pre-merge validation; not for every commit's smoke.

#### A.4 Regression Detection

Current: a wildly wrong metric trains to completion; we catch it in logs.

Design: per-metric historical bands in `config/metric-bands.toml`:

```toml
[metric_bands.train_sharpe]
warn_low = -5.0
warn_high = 200.0
error_low = -50.0
error_high = 1000.0
```

HEALTH_DIAG emission checks every metric against its band. N consecutive out-of-band epochs → `WARN` log. 2N consecutive → `ERROR` + SIGTERM-self via `std::process::exit(1)`. Training workflow FAIL status; Argo notification fires.

The bands are populated from the last 3 known-good runs' metric distributions. Updated as part of the promotion workflow when a new baseline run succeeds.

##### A.4.1 nsys profile harness (sub-task)

Add `--profile` flag to `argo-train.sh`. When set:

- ci-builder image includes `nsys` (via apt install or pinned tarball).
- `train-template.yaml` wraps the training binary under `nsys profile --capture-range=cudaProfilerApi --output=profile-${SHORT_SHA}.nsys-rep`.
- .nsys-rep artifact pushed to `foxhunt-training-artifacts` MinIO bucket, path `profiles/${SHORT_SHA}/`.
- HEALTH_DIAG emits bucket path for operator download.

Baseline profiles captured per release. Regression detection extended: if per-epoch duration grows > 20% over baseline, WARN.

#### A.5 Orphan Audit (Wire-It-Up enforcement)

Scan every `pub` module in `crates/ml/src/` and classify:

| Classification | Action |
|---|---|
| `Wired` (consumed by prod path) | Log in audit doc |
| `Partial` (forward but not backward, or training-only) | Wire missing side via later Parts |
| `Orphan` (no production consumer) | Wire to intended consumer OR delete |
| `Ghost` (consumer path is stubbed) | Unstub + wire |

Known candidates requiring action:

- TLOB transformer: orphan to DQN trunk → D.8 wires.
- Liquid Time-constant `liquid_mod`: unknown effectiveness → D.7 audits.
- TFT backward: partial → D.1 (same infrastructure as Mamba2).
- xLSTM: orphan to DQN → E evaluates.
- KAN: orphan to DQN → E evaluates.

**Corrections discovered during A.5 audit (2026-04-24):**
- Mamba2 backward was believed stubbed but is in fact fully wired (kernel `mamba2_scan_projected_bwd` + call site `gpu_dqn_trainer.rs::mamba2_backward`). The "backward stubbed" language in earlier memory/docs was stale. Reclassified as Wired. Plan 2 D.1 scope narrows accordingly.
- KAN is wired inline in the DQN trunk (splines at weights 42–49, kernels `kan_gate_combine_kernel`/`kan_gate_backward_kernel`) distinct from the `ml-supervised` KAN crate facade (which is correctly OUT). Part E's KAN = OUT applies to the `ml-supervised` facade only; the DQN's own inline KAN is Wired.

Deliverable: `docs/dqn-wire-up-audit.md` with one row per pub module.

#### A.6 Hot-Path Purity Audit

Scan the training step loop (`fused_training.rs::step_fused`, `gpu_dqn_trainer.rs::*`, all cuda_pipeline kernels) for any `memcpy_dtoh` / `memcpy_htod` / `cudaMemcpy*` calls. Classify per Invariant 3:

- `OK-pinned` — already using zero-copy pattern.
- `MIGRATE` — uses `memcpy` but should be pinned; fix during spec impl.
- `COLD-PATH` — acceptable placement (checkpoint, fold transition, epoch boundary), annotated.

Deliverable: `docs/dqn-gpu-hot-path-audit.md`. Pre-commit hook flags any new unaudited cross-boundary call.

Known suspects to verify: `read_atom_utilization`, `branch_grad_scales()` accessor, PER priority batched updates, `kelly_stats_buf` synchronisation, any fold-boundary readbacks, IQL readiness readback, q-stats readback paths.

### Part B — Flat-trap Escape

#### B.1 ISV-driven continuous reward shaping

Current: Flat opp-cost ≤ 1e-6/bar — invisible. Q(Flat) ≈ 0 always wins argmax over CQL-pessimistic trade-action Q.

Design: scale the opp-cost by Q-magnitude via ISV[Q_DIR_ABS_REF]:

```
reward_flat = -shaping_scale × holding_cost × conviction × vol_proxy × isv[Q_DIR_ABS_REF]
```

When Q magnitudes are small (cold start), penalty is small. When Q converges into the ±20..±50 range, penalty scales proportionally so it remains meaningful relative to trade-action Q. Self-adaptive. No tuned scale multiplier; the existing `holding_cost_rate` config scalar is the only external parameter and it is now f32 (post-d64adc14f refactor).

#### B.2 ISV-driven trade-attempt bonus (novelty-weighted)

A separate reward term on Flat→Positioned transitions, representing exploration credit:

```
bonus = conviction × vol_proxy × novelty
novelty = max(0, 1 − isv[TRADE_ATTEMPT_RATE_EMA] / isv[TRADE_TARGET_RATE])
```

- `TRADE_ATTEMPT_RATE_EMA` (new ISV slot) — per-window Flat→Pos transition rate, updated via adaptive-rate EMA.
- `TRADE_TARGET_RATE` (new ISV slot) — frozen after first K epochs, captures the "normal rate" for this task. Not a tuned constant; measured from the training regime.

When the policy is trading at task-normal rate, novelty is 0 and the bonus is inert. When the policy under-trades (Flat-trap), novelty → 1 and the bonus fires. Self-disabling once the Flat-trap is escaped.

#### B.3 Replay Buffer Seeded Warm-Start

Current: PER buffer starts empty, fills via Boltzmann-driven DQN trajectories. At t=0 those trajectories are near-random but magnitude-biased toward the scripted plan MLP output, which produces mostly-losing trades → CQL learns "trading is bad" from a biased sample.

Design: before DQN training starts, seed the PER buffer with **≥ 100K experiences** (exact count sized to fill at least one PER batch × `REPLAY_SEED_EPOCHS` worth of training steps; typical is 100K-200K) drawn from a **portfolio of scripted policies**:

- 40% pure uniform random (all 108 factored actions equiprobable, respecting dir/mag constraints).
- 20% momentum-1 / momentum-3 / momentum-5 (simple N-bar momentum entry, fixed magnitude).
- 20% mean-reversion (short-horizon contrarian, quantile-based entry).
- 20% VWAP-deviation (volume-weighted trend crossover).

Seed trajectories stored with lower initial PER priority than new DQN trajectories, displaced by DQN experiences over roughly 10–15 epochs. CQL alpha is coupled to the seed-phase decay — see C.5.

**Not a feature flag** — the seed step is unconditional for a fresh training job. The only exception is warm-restart from a checkpoint, where the replay buffer is restored from the checkpoint and the seed step is implicit in the checkpoint's history. This is a runtime condition (checkpoint present / absent), not a config toggle. Per `feedback_no_feature_flags.md`.

#### B.4 Adaptive plan-MLP activation threshold

Current: `plan_state[0] > 0.5` hardcoded threshold. In the val-mkgwm run (2026-04-24), plan MLP outputs stayed sub-0.5 for the entire val window → plans never activated → `plan_isv` remained zero despite the val-side machinery being wired.

Design: new ISV slot `PLAN_PARAMS_0_EMA` tracks the EMA of `plan_params[0]` (target_bars MLP output) across training samples. Activation threshold is:

```
activation_threshold = 0.5 × isv[PLAN_PARAMS_0_EMA]
```

Self-scaling. If MLP outputs typically sit near 2.0, threshold is 1.0; if typically near 0.3, threshold is 0.15 and plans still activate. No hardcoded 0.5 constant remains.

Consumed by both training `experience_env_step` and val `backtest_plan_state_isv`.

### Part C — Refinement

#### C.1 Quantile-based atom support (replaces task #91)

Current: `half = max(10 × q_gap, 3 × std_ema, min_floor)`. The `10×` is a tuned constant; saturates `abs_half` when q_gap > 5.

Design: 8 new ISV slots (2 per branch) `Q_P05_EMA[dir,mag,ord,urg]`, `Q_P95_EMA[dir,mag,ord,urg]`. Updated from per-branch Q statistics via adaptive-rate EMA (matching the existing mean/std EMA pattern).

```
half = max(|q_p95 − v_center|, |v_center − q_p5|)
half = half.clamp(min_half_floor, abs_half)
```

Covers the observed Q distribution directly. No distributional assumptions. Tail-aware. No tuned multiplier.

#### C.2 Reward-component attribution

Current HEALTH_DIAG shows `grad_split_bwd` but no per-sample reward-component breakdown. Debugging which reward term is driving policy behaviour requires log-reading archaeology.

Design: new ISV slots `REWARD_COMPONENT_EMA_{popart, cf, trail, micro, opp_cost, bonus}` (6 slots) tracking per-component |reward| contribution via adaptive-rate EMA. HEALTH_DIAG emits `reward_split [...]` analog to `grad_split_bwd`.

Enables "did micro_reward actually contribute this epoch or is it silently zero?" at a glance.

#### C.3 State-distribution divergence signal

Current: training and val state distributions can diverge structurally (plan_isv was OOD for months before we caught it). Only detectable post-hoc.

Design: new ISV slot `STATE_KL_TRAIN_VAL_EMA` — running estimate of distributional divergence between train-state and val-state on a subset of state dimensions (portfolio + plan + ISV features). Computed at val backtest time by comparing per-dimension moments (mean, std), aggregated via a KL-like summary over the monitored dimensions.

**Adaptive threshold:** a second ISV slot `STATE_KL_THRESHOLD_EMA` tracks the EMA of `STATE_KL_TRAIN_VAL_EMA` itself. "KL > threshold" means KL is ≥ 2× its own EMA — i.e. "currently divergent relative to historical baseline", self-calibrating. When the threshold fires, a third ISV slot `STATE_KL_AMPLIFICATION` is bumped from 1.0 toward 2.0 (bounded), which multiplies B.1 and B.2's reward magnitudes. Decays back toward 1.0 as KL normalises. All three slots are scratchpad (slot [58] and [63..65) in §6).

#### C.4 Temporal timing bonus

Current: Mamba2 trunk has temporal memory; reward barely reaches 10 bars back. Architecture-reward misalignment.

Design: on trade exit, compute:

```
temporal_timing_bonus = shaping_scale × max(0, bars_early / hold_time) × |final_pnl| × conviction
```

`bars_early` = number of bars between trade entry and the peak unrealised P&L during the trade. Causal-safe (computed AT EXIT, uses no future info beyond the hold window we just completed).

Rewards the policy for "seeing it coming" — predictive timing that Mamba2 is architecturally capable of learning but not currently incentivised to.

#### C.5 CQL alpha schedule coupled to seed-phase decay

Current: `cql_alpha = 0.014` fixed.

In B.3, CQL pessimism learned from seeded experiences would counter the seed's purpose (we want CQL to learn from the seed-diverse Q distribution, not pull it pessimistic).

Design:

```
cql_alpha(t) = final_alpha × (1 − exp(−t / warmup_steps))
warmup_steps = tracked via ISV, proportional to the seed-phase decay in B.3
```

While seed trajectories dominate the buffer, `cql_alpha` is near zero (no pessimism). As seed trajectories displace, `cql_alpha` ramps to its final value.

#### C.6 Adaptive Mechanism Unification (major refactor)

Current: the DQN has many adaptive mechanisms (atoms, gamma, grad-balancer, Kelly cap, tau, cql_alpha, epsilon, conviction-floor, plan-threshold, and new ones added by this spec). Each was bolted on independently with its own update rule and ISV interaction. Most were CPU-computed and pushed to GPU consumers.

**Design — GPU computes, CPU reads (non-negotiable):**

All adaptive values and all schedules are computed by GPU kernels from ISV-visible inputs and written to ISV slots. CPU-side code has two roles only:

1. **CPU-born inputs to ISV** — values that fundamentally originate on CPU (epoch_idx, total_epochs, static config constants) are written to dedicated ISV slots once at the appropriate cadence (epoch boundary for epoch_idx, constructor for static configs). This is a pinned-memory zero-copy write.
2. **Read-only observation** — CPU reads ISV slots to emit HEALTH_DIAG diagnostics and track fire-rate statistics for the `controller_activity` smoke.

CPU never computes an adaptive value or schedule output. If the value is f(ISV_signals), a GPU kernel computes it.

**Rationale:**

- Unified adaptive machinery: every adaptive mechanism follows the same on-GPU pattern. No CPU-orchestrated special cases.
- Per-sample granularity available — adaptive mechanisms can be per-sample if useful (e.g., Expected SARSA τ already does this inline in c51_loss_kernel), whereas CPU controllers are bounded to per-epoch.
- Zero CPU→GPU config transfer in the hot path. CPU writes to ISV via the pinned pointer (no copy); GPU reads from ISV directly.
- ISV is the single source of truth for every adaptive value. CPU-side code is pure observability, enforcing Invariant 3's spirit beyond the hot path into epoch-boundary code as well.
- No conflation of "schedule" (epoch-deterministic) and "adaptive" (reactive to observed signals): both compute on GPU. The input signals differ; the compute location does not.

**The observer trait:**

```rust
pub(crate) trait AdaptiveMonitor {
    /// Read the GPU-computed current value from ISV.
    fn read(&self, isv: &IsvBus<'_>) -> f32;
    /// Emit diagnostic snapshot for HEALTH_DIAG.
    fn diagnose(&self, isv: &IsvBus<'_>) -> DiagSnapshot;
    /// Record one observation; drives fire-rate tracking across epoch boundaries.
    fn observe(&mut self, current: f32);
    fn fire_rate(&self) -> &FireRateStats;
    fn name(&self) -> &'static str;
}
```

No `update()`, no `write_output()`, no `Signal`/`Control` associated types. Pure observation.

**Mechanism → GPU kernel mapping:**

| Mechanism | Nature | Kernel behaviour | CPU monitor? |
|---|---|---|---|
| atoms | Reactive to Q-distribution | reads ISV v-range; writes atom positions to device buffer + summary to ISV | yes |
| gamma | Reactive to Q-stats util | reads ISV Q-stats util; writes γ to ISV | yes |
| kelly_cap | Reactive to trade P&L distribution | reads ISV Kelly stats; writes cap to ISV | yes |
| tau (Polyak EMA) | Schedule + health-reactive | reads ISV epoch_idx + total_epochs + health; writes τ_eff to ISV | yes |
| epsilon | Schedule + volatility-reactive | reads ISV epoch_idx + total_epochs + volatility; writes ε to ISV | yes |
| grad_balancer | Reactive to per-branch grad norms | existing `grad_balance_isv_update` kernel (already GPU-driven) | yes (new) |
| cql_alpha | Static config | CPU writes to ISV at constructor; no kernel | no |
| conviction_floor | Static config | CPU writes to ISV at constructor; no kernel | no |
| plan_threshold | Static config | CPU writes to ISV at constructor; no kernel (Plan 3 §B.4 makes reactive) | no |

Expected SARSA τ in `c51_loss_kernel` is already per-sample GPU-computed from ISV — no change needed; diagnostic surface already wired via `last_sarsa_tau_factor()`. Exemplar of the pattern.

**New ISV slots for computed outputs + CPU-born inputs:**

Allocated in dedicated ranges (§6 appendix). The fingerprint seed updates automatically as slots are added; fingerprint position shifts to the new tail.

- `EPOCH_IDX_INDEX`, `TOTAL_EPOCHS_INDEX` — CPU-written at epoch boundary and constructor respectively.
- `EPSILON_EFF_INDEX`, `TAU_EFF_INDEX`, `GAMMA_EFF_INDEX`, `KELLY_CAP_EFF_INDEX` — GPU-written by the respective kernels.
- `CQL_ALPHA_INDEX`, `CONVICTION_FLOOR_INDEX`, `PLAN_THRESHOLD_INDEX` — CPU-written at constructor for the three static configs.

Rationale for CPU→ISV static writes (rather than Rust constants): consumers read via ISV for consistency with the adaptive mechanisms. A reader cannot tell whether a value is static or GPU-updated; both use the same slot-read pattern. This eliminates a class of "wrong lookup" bugs and gives static values a uniform diagnostic surface.

**Benefits preserved from the original C.6 design:**

- Adding a new adaptive mechanism follows a protocol — write a GPU kernel + a CPU monitor.
- Fire-rate auditing built in via `controller_activity` smoke.
- Standardised diagnostics — HEALTH_DIAG's `controllers [...]` line derived from monitor state.

**Execution:** a series of internal commits — spec+plan revision first, then revert any prior CPU-compute implementation, then per-kernel implementation + CPU monitor pairs. Old CPU-side scaffolding is removed in the SAME commit that lands its GPU-kernel replacement, per `feedback_no_partial_refactor.md`.

### Part D — Temporal Architecture

Current: Mamba2 + Liquid-TC in trunk; plan_isv state machine; conviction drift; per-bar reward with sparse trade-completion. The architecture is deep-temporal; the reward signal is shallow-temporal. Misalignment.

#### D.1 Complete Mamba2 backward pipeline (task #76 subset)

Current: forward works, backward stubbed.

Design: properly wire Mamba2 backward for the **DQN path specifically** — state-space gradients dx, dA, dB, dC through the recurrence as consumed by the DQN trunk. Substantial CUDA kernel work but load-bearing: every temporal improvement downstream assumes temporal learning actually propagates end-to-end.

Scope: fused backward kernel covering `mamba2_temporal_kernel.cu` for the DQN-integration call sites only. The same kernel infrastructure can be reused for supervised Mamba2 training, but that is not in this spec's scope.

Validation: grad-norm smoke test (perturbation grad-check against finite difference) on a small fixture; integration test verifying that DQN training with Mamba2 backward produces non-zero gradients propagating into Mamba2 weights over several training steps.

This closes task #76 for the DQN path; TFT backward (task #76's other half) is addressed in Part E only if/when TFT is adopted as an auxiliary head.

#### D.2 Per-branch gamma

Current: single scalar gamma across all 4 factored branches.

Design: 4 ISV slots `GAMMA_{DIR, MAG, ORD, URG}`, per-branch gamma updated by an `AdaptiveController` impl. Each branch's C51 projection uses its own gamma. Direction can run gamma=0.99 (trend horizon); urgency can run gamma=0.80 (execution horizon). Driven by per-branch Q-gap spread and atom utilization, not tuned.

Consumed by: `block_bellman_project_f` in c51_loss_kernel.cu (reads per-branch gamma), `iql_compute_per_sample_support`, etc.

#### D.3 Horizon-decomposed value function

Current: IQL `V(s)` is scalar.

Design: decompose `V(s) = V_short(s) + V_long(s)`:

- V_short: 10-bar rolling P&L expectation (execution quality).
- V_long: 100-bar P&L expectation (trend capture).
- Two IQL heads, both expectile-regressed.
- Sum is the Q-target; decomposition is diagnostic + allows horizon-weighted action selection.

Novel: lets the TD target explicitly attribute "this action helps short-term or long-term". Debuggable — we see if policy is over-indexing on one horizon.

State layout impact: adds one dimension to the IQL value output. Schema version bump (A.2) + state_layout coordinated change.

#### D.4 Generalised temporal reward coupling

Extends C.4 with two more components:

- **Persistence credit:** trade held through noise before taking profit → bonus. Discourages whipsaw exits.
- **Regime-shift penalty:** entered trade assuming regime X, regime flipped mid-trade → penalty proportional to how early the exit should have happened.
- **Temporal conviction consistency:** policy's conviction signal was stable across N bars before entry → bonus for well-reasoned entry.

All ISV-scaled and conviction-gated. New ISV slots:
`TEMPORAL_REWARD_EMA_{persist, regime_shift, consistency}` — producer: env_step; consumer: HEALTH_DIAG.

#### D.5 Soft fold-boundary transitions

Current (per A.1): reset registry with `FoldReset` (hard) and `TrainingPersist` (unchanged).

Design: registry extended with `SoftReset(decay_bars: u32)` variant. Soft-reset state anneals toward its bootstrap value over the first `decay_bars` of the new fold. Applies to:

- `adaptive_gamma` — shouldn't jump from 0.95 back to 0.90 at fold boundary.
- Grad-balance ISV targets — re-seed anneals, don't hard-reset.
- Plan-params EMA — partial persist so MLP output distribution is known at fold start.

Smoother transitions preserve learned temporal signals across folds without compromising fold-independence of weight-related state.

#### D.6 Plan temporal dynamics enrichment

Current plan_isv has 6 dimensions `[0..6)`: progress, pnl-vs-target, pnl-vs-stop, entry-conviction, conviction-drift, regime-shift. That is `SL_PORTFOLIO_PLAN_DIM = 6` in `state_layout.cuh` and state positions `[86..92)` in the 96-dim state vector.

Design: extend plan_isv to 7 dimensions `[0..7)` — add a new dimension at index 6:

```
plan_isv[6] = clamp((plan_target_bars − hold_time) / plan_target_bars, 0.0, 1.0)
            = remaining-fraction
```

Directly exposes temporal pressure to the policy ("my plan is 80% done, should I take the profit I have?"). Trivial to compute. Requires coordinated change:

- `SL_PORTFOLIO_PLAN_DIM = 6 → 7` in `state_layout.cuh`.
- State vector grows from 96 to 97 dimensions (the new plan_isv[6] slots into position [92]; everything downstream shifts by one OR we leave a pad slot and account for it). Decision during impl: prefer padded no-shift to minimise downstream reshuffling.
- Schema version bump (A.2) paired with D.3 and D.8's similar layout impacts; all three land as one coordinated state-layout migration commit rather than three.

#### D.7 Liquid Time-constant `liquid_mod` audit

Current: `liquid_mod_buf` appears in HEALTH_DIAG and is passed to c51 kernels. Its measurable effect on Q-update is unclear.

Design: audit-only subtask —

1. Confirm `liquid_mod` actually modulates c51 gradients proportionally (trace through `c51_grad_kernel`).
2. Measure its fire rate — does it ever deviate meaningfully from 1.0?
3. Verify its controller is ISV-driven (per Invariant 1).

Outcome: either wire properly (via C.6's `AdaptiveController` impl) or — if verified as no-op in practice — mark for deletion per `feedback_wire_everything_up.md`.

#### D.8 TLOB-DQN temporal integration

**REVISED (2026-04-24 pivot):** Port TLOB's attention using the DQN's existing cuBLAS/autograd infrastructure. **No ONNX, no pretraining, no supervised side-stream.** Random-init, trainable end-to-end from the DQN's reward signal — uniform with Mamba2, IQL, atoms/γ/τ/ε kernels, all of which already follow this pattern.

**Rationale (`pearl_tlob_no_pretraining.md`):**

1. The DQN crate already contains every primitive TLOB needs: `gpu_attention.rs` (self-attention layer in the trunk), `batched_forward.rs` / `batched_backward.rs` (batched SGEMM forward/backward), `shared_cublas_handle.rs` (cuBLASLt handles), `cublas_algo_deterministic.rs` (deterministic algorithm selection), `cuda_autograd` (Rust autograd over cuBLAS ops), `GpuLinear` (trainable linear layer). TLOB's Q/K/V + attention + output projections are a composition of these primitives, not a new kernel family.
2. The ONNX pathway in `ml-supervised::tlob::transformer` is dead (Plan 2 Task 6C audit): the ONNX runtime was stripped, `load_onnx_model` is a stub, no pretrained checkpoint exists, the `TLOBTrainableAdapter` is an untrained harness. Resurrecting pretraining costs more than starting from the cuBLAS primitives.
3. For RL with bounded single-instrument data, reward-signal end-to-end training learns the relevant attention pattern directly. Supervised pretraining on a proxy objective (future-return prediction) introduces target misalignment.
4. Alignment with every other DQN temporal encoder: Mamba2 (validated by Plan 2 Task 2 grad-check), IQL, KAN. TLOB was the only outlier assuming pretraining.

**Design:** TLOB as **complementary temporal encoder** composed from existing cuBLAS primitives, inserted into the DQN trunk alongside Mamba2.

```
MBP-10 event sequence → TLOB (cuBLAS Q/K/V + softmax + out-proj) → tlob_features[D_tlob]
MBP-10 raw bars       → classical OFI                              → ofi_features[D_ofi]
bar sequences         → Mamba2 state-space                         → mamba2_features[D_mamba2]

trunk_input = [market ‖ tlob_features ‖ ofi_features ‖ mtf ‖ portfolio ‖ plan_isv]
```

Two complementary learned temporal modalities: TLOB attention (event-level), Mamba2 state-space (bar-level). Plus classical OFI (VPIN, Kyle's λ).

**Wiring (minimum-viable + trainable):**

- TLOB layer constructed like any other trunk layer at `GpuDqnTrainer::new`: allocate cuBLAS-backed `GpuLinear` for Q/K/V projections + output projection, Xavier-init weights, register in Adam param groups, register in backward graph.
- Forward: called at state-assembly time (cold path, per-epoch-boundary OR per-bar depending on whether TLOB reads the full event window or bar-aggregated events). MVP: per-bar, reading the current bar's MBP-10 event sequence as a fixed-length context.
- Backward: gradients flow through TLOB weights via the existing `cuda_autograd` mechanism. Same pattern as Mamba2's backward.
- **No freeze toggle.** TLOB trains with the DQN unconditionally per `feedback_no_feature_flags.md`.
- New ISV slot `TLOB_REGIME_FOCUS_EMA` — attention-weight summary (mean max-attention per sample) exposed as a diagnostic. CPU monitor reads; no CPU compute.

**State-layout impact:** adds `SL_TLOB_DIM` dimensions (pick 16 or whatever matches the TLOB output projection width). Coordinated with D.6 and D.3 — state vector grows, layout fingerprint auto-recomputes from updated seed bytes.

**Scope boundaries for the implementation task:**

- MVP: Q/K/V + scaled-dot-product attention + output projection. Single-head or multi-head — pick during impl based on param-budget. No positional encoding in MVP (events are a short window; order implicit).
- Full TLOB paper architecture (multi-layer stack, residual+LN, etc.) is a follow-up if MVP's attention pattern is too weak.
- TLOB input: MBP-10 events over the current bar. Exact seq-len picked during impl (16-64 depending on MBP-10 event density).

**Prerequisites (already satisfied, unlike the original pretrained-ONNX design):**

- `gpu_attention.rs` exists.
- `GpuLinear` trainable layer exists.
- `cuda_autograd` over cuBLAS exists.
- MBP-10 data is already in the DQN data pipeline.

Nothing blocks the port. What was documented as "BLOCKED on prerequisites" in the 6C audit was blocked only on the OLD design (pretrained ONNX). The cuBLAS-native design has all prerequisites satisfied.

### Part E — Supervised→DQN Concept Adoption

Concrete decisions (per Invariant 9, every E item either lands or does not — no "evaluate later"):

| Concept | Current DQN use | Decision | Action |
|---|---|---|---|
| TFT Variable Selection Network (VSN) | Partial (`vsn_mag`, `vsn_dir` masks) | **IN** | E.1 — extend existing masks to full TFT VSN over all state feature groups |
| Gated Residual Network (GRN) | Audit result TBD during A.5 | **IN if absent** | E.2 — if A.5 finds GRN not used in trunk, adopt as trunk building block; if present, mark existing impl as canonical in the audit |
| TFT multi-quantile heads | Partial (IQN CVaR only) | **IN** | E.3 — extend IQN branch to explicit multi-quantile decomposition (5/25/50/75/95 quantiles) |
| TLOB attention encoder | Orphan to DQN | **IN** | D.8 wires to DQN trunk |
| Mamba2 forward | Wired | — | already wired |
| Mamba2 backward | Partial | **IN** | D.1 completes |
| Liquid Time-constant | Unclear | **Audit-then-IN-or-DELETE** | D.7 decides; no third path |
| xLSTM | Orphan | **OUT** | Redundant with Mamba2 (state-space) + TLOB (attention); adding a third temporal encoder violates YAGNI. Leaving the supervised-crate impl as-is; no DQN wiring. |
| KAN | Orphan | **OUT** | Function-approximation is not a current bottleneck; Q-head dueling + C51 atoms already give sufficient expressivity. Leaving the supervised-crate impl as-is. |
| Encoder-decoder separation | Single trunk | **IN** | E.4 — D.3 (horizon-decomposed V) is a form of this; E.4 extends by separating the shared trunk into "state encoder" + per-branch "value decoders" with explicit interface |
| Attention-weight interpretability | Partial | **IN** | E.5 — ISV-expose attention weights from Mamba2, TLOB, and trunk heads; new diagnostic slots `ATTN_*_FOCUS_EMA` |
| Multi-task auxiliary heads | None | **IN** | E.6 — add auxiliary regression heads to trunk: next-bar return prediction + 5-bar regime classification. Trained via joint loss; gradients flow to trunk only (heads themselves are lightweight) |

**Explicit OUT items (xLSTM, KAN):** the supervised-crate implementations remain in place for ml-supervised use. They are NOT orphans to this spec's scope because they serve their supervised consumers. The Wire-It-Up invariant applies to consumer paths per crate; xLSTM and KAN having zero DQN consumers is the correct state.

E.1, E.2, E.3, E.4, E.5, E.6 are all in scope for this spec. Not "evaluate" — concrete engineering. Each lands wired, diagnostic, and fully finished per Invariant 9.

Deliverable: `docs/ml-supervised-to-dqn-concept-audit.md`.

---

## 5. Implementation methodology

### Internal commits + worktrees

- Spec implementation is expected to span many internal commits. Permitted.
- Agents work in git worktrees (`git worktree add`) per `feedback_no_concurrent_agents_shared_tree.md` when parallelism is needed on non-overlapping files.
- Sequential single-tree work for agents touching shared files.
- Every commit merged to main compiles and passes smoke tests. No "broken for 3 commits while we fix forward".

### Cleanup contract

- Every commit landed on main preserves all nine invariants.
- Worktrees are removed after merge. No abandoned worktrees.
- Orphan modules discovered during implementation are resolved in the same commit that discovers them (wired or deleted).

### Landing order (suggested, non-mandatory within spec)

Dependencies to respect:

- A.2 (ISV contract) before ANY feature claiming new ISV slots.
- A.1 (reset registry) before D.5 (soft fold transitions — extends the registry).
- D.1 (Mamba2 backward) before D.2, D.3, D.4, D.8 (anything that relies on temporal gradients propagating).
- C.6 (adaptive controller unification) before D.2 (per-branch gamma), D.7 (liquid audit) — these use the controller protocol.
- B.3 (replay seed) before C.5 (CQL α-seed coupling — needs the seed-phase signal).
- State-layout changes (D.3 horizon-decomposed V, D.6 plan[6], D.8 TLOB features) land as ONE coordinated commit. The layout fingerprint (A.2) updates automatically because slot constants change — no manual bump; the single-commit discipline is about keeping all consumers in sync, not about a version stamp.

Suggested order respecting dependencies:

1. **Substrate (zero functional change):** A.1, A.2, A.5, A.6.
2. **Refactor (non-functional):** C.6 adaptive controller unification.
3. **Surgical atoms:** C.1 quantile atom support.
4. **Core temporal:** D.1 Mamba2 backward — unlocks D.2–D.8.
5. **Temporal features:** D.2 per-branch γ, D.5 soft fold transitions, D.7 liquid audit.
6. **Coordinated state-layout migration:** D.3 + D.6 + D.8 in one commit.
7. **Reward shaping:** B.1, B.2, B.4 + C.4, D.4 (all reward components together since they share ISV slots).
8. **Replay warm-start:** B.3 + C.5 (coupled per C.5 design).
9. **Diagnostics:** C.2 reward attribution, C.3 KL divergence signal.
10. **Supervised concepts:** E.1–E.8 (each a sub-PR, in dependency order determined by E audit).
11. **Validation substrate:** A.3 multi-seed harness, A.4 regression detection, A.4.1 nsys harness.
12. **Final validation pass:** multi-seed × multi-fold across Tier 1, 2, 3 exit criteria.

Many items in the same step are independent and parallelise across worktrees. The order is suggested; writing-plans may adjust based on concrete work estimation.

### Spec decomposition for writing-plans

This spec is large. The writing-plans skill may produce **multiple sequential implementation plans** rather than one monolithic plan — for example:

- Plan 1: Substrate + refactor (items 1–3 above).
- Plan 2: Temporal core (items 4–6).
- Plan 3: Behavioural (items 7–9).
- Plan 4: Supervised concept audit (item 10) — one mini-plan per E sub-item.
- Plan 5: Validation substrate + final pass (items 11–12).

Each plan ships independently; all plans are subordinate to this spec's invariants. Invariants enforced across plans via the pre-commit hooks (Invariant 7).

---

## 6. Appendix: ISV slot allocation (post-spec)

Current: 37 slots (indices [0..37)). This spec adds:

| Index range | Purpose | Source |
|---|---|---|
| [0] | Schema version | A.2 |
| [1..12) | (existing) | — |
| [12] | learning_health | (existing) |
| [13..22) | (existing Q-scale, sharpe_ema) | — |
| [22] | SHARPE_EMA | (existing) |
| [23..31) | Per-branch v-range | (existing) |
| [31..35) | Per-branch grad-norm targets | (existing) |
| [35] | Grad scale limit | (existing) |
| [36] | IQL branch_scales floor | (existing) |
| [37..41) | Per-branch gamma (D.2) | D.2 |
| [41..45) | Per-branch Q_P05_EMA (C.1) | C.1 |
| [45..49) | Per-branch Q_P95_EMA (C.1) | C.1 |
| [49] | PLAN_PARAMS_0_EMA (B.4) | B.4 |
| [50] | TRADE_ATTEMPT_RATE_EMA (B.2) | B.2 |
| [51] | TRADE_TARGET_RATE (B.2) | B.2 |
| [52..58) | REWARD_COMPONENT_EMA_* (C.2) | C.2 |
| [58] | STATE_KL_TRAIN_VAL_EMA (C.3) | C.3 |
| [59..62) | TEMPORAL_REWARD_EMA_* (D.4) | D.4 |
| [62] | TLOB_REGIME_FOCUS_EMA (D.8) | D.8 |
| [63..72) | Reserved for Part E additions | — |

New `ISV_TOTAL_DIM` = 72 (from 37). Network-facing `ISV_NETWORK_DIM` remains 23 — the new slots are scratchpad / adaptive-control state not fed into the attention path.

---

## 7. Appendix: new documents tracked by pre-commit

- `docs/dqn-wire-up-audit.md` — component → consumer path mapping (Invariant 2, 7).
- `docs/isv-slots.md` — ISV slot allocation registry with type/producer/consumers (A.2, Invariant 7).
- `docs/dqn-gpu-hot-path-audit.md` — cross-boundary call classification (Invariant 3, 7).
- `docs/ml-supervised-to-dqn-concept-audit.md` — Part E deliverable.
- `docs/dqn-named-dims.md` — named-constant registry (Invariant 8).
- `config/metric-bands.toml` — regression-detection bands (A.4, Invariant 7).

Pre-commit hook rejects component-adding commits that don't touch the relevant audit docs.

---

## 8. Decisions deferred to implementation (with decision criteria)

Per Invariant 9, this spec has no "evaluate later" work items. The items below are **bounded decisions** — the design path is fixed, only the parameter point is chosen at implementation time with an explicit decision criterion written into the spec.

- **D.8 TLOB training mode** — decision criterion: benchmark a single training step with TLOB end-to-end vs TLOB frozen. If end-to-end cost per step ≤ 5ms incremental over baseline, use end-to-end. If > 5ms, use frozen-with-side-stream (retrain TLOB via supervised job every N epochs on future-return prediction). No third option. Decision made in the D.8 commit message based on measured benchmark data.

- **C.6 controller migration order** — decision: migrate in order of increasing ISV surface area (number of slots the controller reads/writes). Concretely: `atoms` → `gamma` → `Kelly cap` → `cql_alpha` → `tau` → `epsilon` → `conviction_floor` → `plan_threshold` → `balancer`. This order is now IN the spec; not deferred.

- **A.3 multi-seed resource plan** — decision: default to L40S pool. If `N=5, K=6` total budget exceeds 12 GPU-hours (measured on the first multi-seed run), migrate to H100 pool (2-3× faster). The switch is automated via `argo-train.sh` based on measured budget. No human intervention.

- **A.4 metric bands initial population** — decision: populate from the most recent 3 successful training runs' epoch-by-epoch metric distributions. Band = `[mean − 3σ, mean + 3σ]` per metric. Re-fit after every promotion of a new baseline run.

All four decisions are concrete enough that implementation agents execute them without further design input. The decision criterion is data-driven (D.8), order-based (C.6), budget-based (A.3), or statistical (A.4) — no discretionary "we'll see" choices.

---

## 9. Non-goals

Explicit things this spec does NOT address:

- PPO trainer path (separate model).
- ml-supervised training itself (only its borrowable concepts).
- Deployment / paper trading harness.
- Data-pipeline changes (MBP-10 ingest, fxcache format).
- TFT as a separate predictive model (only TFT concepts borrowed into DQN via E.1, E.2, E.3).

---

**End of spec.**
