# Adaptive Risk Management — Design (Layered Stack)

**Goal**: close the train-eval distribution-shift gap surfaced by the 2026-05-30 adaptive-controller-floor refactor (commit `b1ef6664a`). The fix preserves the +$18M baseline (G9 PASS, +$15.81M) but fold 1 walk-forward eval came in marginally worse (-$616k vs -$500k OLD) despite massively better train pnl. **Eval JSONL revealed the load-bearing missing layers**: position commitment, action selection risk-awareness, and inventory exposure are all under-adapted relative to what the controller-floor architecture made possible.

**Why now**: the eval `win_rate=0.21 × R-multiple=1.8` Kelly fraction is negative — agent should not trade but does (1024 batches). Train win_rate 0.55 vs eval 0.21 = severe distribution shift. The controller-floor refactor made signal *interpretation* adaptive; commitment, action-selection, and reward shaping for inventory remained hardcoded.

**Out of scope**: changes to the perception encoder, Mamba2/CfC architecture, advantage normalization (Phase 4.5), the 12 adaptive controllers from b1ef6664a (those are now correct). Decision policy (the agent still chooses among the 11 discrete actions). Regime detection / meta-policy switching (deferred to follow-up per research caveat).

---

## Diagnosis — what the JSONL says

Per fold 1 eval (`/feature-cache/alpha-rl-runs/b1ef6664a/fold1/eval_summary.json`):

| Metric | Value | Interpretation |
|---|---|---|
| total_pnl_usd | -$616k | losing |
| avg_win | $13,785 | wins are large |
| avg_loss | $7,631 | R-multiple 1.8 (good) |
| win_rate | 0.21 (vs train 0.55) | severe directional overfit |
| max_drawdown_pct | 18.7% | controlled but real |
| profit_factor | 0.83 | losing aggregate |

**Kelly at observed edge**: f* = (0.21 × 1.8 − 0.79)/1.8 = **−0.23** → don't trade. Agent trades anyway. **Position commitment never adapts to edge.**

**Internal evidence from in-run JSONL** (fold 1 step 16k, qpa = +0.68 = high agent confidence): **the IQN distribution would tell us the pessimistic Q (lower quantile) is significantly negative even when expected Q is positive**. The agent uses `argmax(E_τ[Q])` (mean over quantiles) — over-weights upside under noisy data. Risk-averse action selection (lower quantile τ) would naturally filter these losing trades.

**Bigger picture**: the system has no adaptive layer between "agent's decision" and "actual market commitment." Five distinct layers are absent or hardcoded:

| Layer | What it does | Status pre-spec |
|---|---|---|
| Hard constraints (max position, DD cap, etc.) | Override agent on safety violation | Mostly absent |
| Risk-averse action selection | Bias decisions toward downside-aware Q | Uses E[Q] — risk-neutral |
| Inventory exposure penalty | Discourage one-sided drift | Absent in reward shaping |
| Position commitment / sizing | Scale lots by observed edge | Fixed at base lots |
| Regime detection | Distribution-shift handling | Absent |

This spec addresses the first four. Regime detection is deferred per research evidence ("research-active, not production-validated").

---

## Scope — the layered risk stack

Apply `feedback_adaptive_not_tuned` consistently across the commitment layer:

1. **Layer 1 — Hard CMDP-style constraints (~120 LOC)**: 5 deterministic risk gates that override agent actions
2. **Layer 2 — IQN risk-averse action selection (~30 LOC)**: use lower-quantile τ from existing IQN distribution
3. **Layer 3 — Inventory penalty in reward shaping (~60 LOC)**: discourage one-sided net-position drift
4. **Layer 4 — Kelly-fraction position sizing (~80 LOC)**: scale committed lots by observed edge
5. **TrailTighten/Loosen wiring (~30 LOC)**: close dead-action gap per [[pearl_dead_trail_stop_actions_a7_a8]]

Single atomic commit per `feedback_no_partial_refactor`. Same pattern as the controller-floor refactor.

**The crucial reordering vs an earlier draft of this spec**: Kelly was originally the primary fix. Research (FineFT 2026, distributional-RL surveys) plus internal analysis showed **Layer 2 (IQN risk-averse τ) is the higher-leverage minimum-cost change** because we already compute the full IQN quantile distribution but throw away everything except the expectation. Kelly is now Layer 4 — fine-tuning after pessimistic action selection has filtered the trade set.

---

## Approaches considered

### Approach A — Kelly fraction only
Original minimum-viable design. Single new controller computes f* from observed win_rate × R-multiple, multiplies lot size. ~80 LOC.

**Verdict**: Insufficient. Kelly is the *commitment* layer; the *selection* layer (which trades to even consider) is more impactful. The 21% eval win_rate means the agent shouldn't even be entering most trades, regardless of size.

### Approach B — Kelly + session DD circuit breaker
Add a session-pnl-tracking ISV slot + override-to-Hold logic. Two-layer (Kelly per-trade + session-wide cap).

**Verdict**: Closer but still missing the action-selection layer. Session DD CB triggers AFTER damage is done.

### Approach C — B + confidence-to-size scaling
Add Q-distribution magnitude as continuous size multiplier alongside Kelly.

**Verdict**: Three multiplicative layers (Kelly × confidence × base) become hard to reason about. Confidence is implicit in Layer 2's quantile selection.

### Approach D — Add TrailTighten/Loosen wiring to any of A/B/C
Close the 10%-policy-mass-wasted gap by activating dead actions.

**Verdict**: Orthogonal to risk management. Worth doing but small impact.

### Approach E — Layered risk stack (RECOMMENDED)

Replace the Kelly-first design with five distinct layers, each addressing a different risk concern:

| Layer | Concern | ISV-driven? | Production validation |
|---|---|---|---|
| 1. Hard CMDP gates | Catastrophic-loss safety (session DD, max position, max inventory) | ✓ | Highest (standard in live HFT) |
| 2. IQN risk-averse τ | Downside-aware action selection | ✓ (adaptive τ) | Research, but our IQN gives it for free |
| 3. Inventory penalty | Adverse selection, one-sided drift | ✓ (adaptive β) | Highest (Avellaneda-Stoikov is canonical HFT) |
| 4. Kelly sizing | Edge-aware commitment | ✓ | Lower (research stage in RL context) |
| (D) TrailTighten/Loosen | Close dead-action gap | ✓ (factor slots) | N/A — wiring fix |

**Why E**: research evidence (FineFT KDD 2026, plus broader distributional-RL literature) supports the layered approach. Each layer addresses a distinct failure mode visible in the JSONL. Combined, they form the "deployment-grade risk stack" pattern.

**Total scope**: ~320 LOC across 5 new + 5 modified files. Same atomic-commit discipline as the prior refactor.

---

## Detailed design (Approach E)

### Layer 1 — Hard CMDP constraints

Deterministic safety gates that override agent actions in `actions_to_market_targets.cu`. NOT learned by the agent. Each constraint is an independent ISV slot, AND'd together — any single trigger forces Hold (or partial-flatten).

**New ISV slots** (allocated starting at `RL_SLOTS_END = 662`):

| Slot | Name | Purpose | Bootstrap |
|---|---|---|---|
| 662 | `RL_SESSION_PNL_USD_INDEX` | Cumulative pnl since session boundary | 0.0 |
| 663 | `RL_SESSION_DD_LIMIT_USD_INDEX` | Drawdown cap (USD, negative) | -3500.0 (= 10% × $35k) |
| 664 | `RL_SESSION_DD_TRIGGERED_INDEX` | Sticky flag, set when DD breached | 0.0 |
| 665 | `RL_MAX_OPEN_UNITS_INDEX` | Max concurrent open positions per batch | 4.0 (= current `MAX_UNITS`) |
| 666 | `RL_CONSEC_LOSS_LIMIT_INDEX` | Max consecutive losses before cool-down | 10.0 |
| 667 | `RL_CONSEC_LOSS_COUNT_INDEX` | Running consecutive-loss count | 0.0 |
| 668 | `RL_COOLDOWN_REMAINING_STEPS_INDEX` | Cool-down period after limit hit | 0.0 |
| 669 | `RL_COOLDOWN_DURATION_INDEX` | Default cool-down length (steps) | 500.0 |
| 670 | `RL_NET_INVENTORY_LIMIT_USD_INDEX` | Cap on net one-sided exposure | 10000.0 |

**Kernel: `rl_cmdp_constraints_check.cu`** (~100 LOC, single-thread single-block, runs once per step before `actions_to_market_targets`)

```c
// Layer 1: hard CMDP gates. Output: writes booleans to ISV slots that
// actions_to_market_targets reads as overrides.
//
// Five independent constraints, ALL must be satisfied (= no override) for
// the agent's chosen action to be executed. Any violation → force Hold.

extern "C" __global__ void rl_cmdp_constraints_check(
    float* __restrict__ isv,
    const float* __restrict__ realized_pnl_per_batch,   // [b_size]
    const float* __restrict__ net_position_per_batch,   // [b_size] (signed contracts)
    const u8*    __restrict__ unit_active_per_batch,    // [b_size × MAX_UNITS]
    const int*   __restrict__ trade_outcome_per_batch,  // [b_size] −1/0/+1
    int b_size
) {
    if (threadIdx.x != 0 || blockIdx.x != 0) return;

    // 1. Session pnl accumulation + DD check.
    float step_pnl = 0.0f;
    for (int b = 0; b < b_size; b++) step_pnl += realized_pnl_per_batch[b];
    const float session_pnl = isv[RL_SESSION_PNL_USD_INDEX] + step_pnl;
    isv[RL_SESSION_PNL_USD_INDEX] = session_pnl;
    if (session_pnl < isv[RL_SESSION_DD_LIMIT_USD_INDEX]) {
        isv[RL_SESSION_DD_TRIGGERED_INDEX] = 1.0f;
    }

    // 2. Cool-down decrement.
    float cooldown = isv[RL_COOLDOWN_REMAINING_STEPS_INDEX];
    if (cooldown > 0.0f) {
        isv[RL_COOLDOWN_REMAINING_STEPS_INDEX] = cooldown - 1.0f;
    }

    // 3. Consecutive loss tracking. Outcomes: -1 = loss, 0 = open/hold, +1 = win.
    float consec = isv[RL_CONSEC_LOSS_COUNT_INDEX];
    for (int b = 0; b < b_size; b++) {
        const int outcome = trade_outcome_per_batch[b];
        if (outcome == -1) consec += 1.0f;
        else if (outcome == 1) consec = 0.0f;
        // outcome == 0 (no change): consec unchanged
    }
    isv[RL_CONSEC_LOSS_COUNT_INDEX] = consec;
    if (consec >= isv[RL_CONSEC_LOSS_LIMIT_INDEX]) {
        isv[RL_COOLDOWN_REMAINING_STEPS_INDEX] = isv[RL_COOLDOWN_DURATION_INDEX];
        isv[RL_CONSEC_LOSS_COUNT_INDEX] = 0.0f;  // reset after triggering
    }
}
```

Override logic in `actions_to_market_targets.cu`:

```c
const bool dd_triggered  = isv[RL_SESSION_DD_TRIGGERED_INDEX] >= 0.5f;
const bool in_cooldown   = isv[RL_COOLDOWN_REMAINING_STEPS_INDEX] > 0.0f;
const bool override_hold = dd_triggered || in_cooldown;

if (override_hold) {
    actions[b] = ACTION_HOLD;
    target_lots[b] = 0;
    return;  // skip rest of action translation
}

// Max-open-units check (per-batch, dynamic):
int active_units = 0;
for (int u = 0; u < MAX_UNITS; u++) {
    if (unit_active[b * MAX_UNITS + u]) active_units++;
}
if (active_units >= (int)isv[RL_MAX_OPEN_UNITS_INDEX] &&
    is_opening_action(actions[b])) {
    actions[b] = ACTION_HOLD;  // refuse new opens, allow closes
}

// Inventory cap check:
const float net_pos_usd = compute_net_position_usd(b);
if (fabsf(net_pos_usd) > isv[RL_NET_INVENTORY_LIMIT_USD_INDEX] &&
    increases_one_sided_exposure(actions[b], net_pos_usd)) {
    actions[b] = ACTION_HOLD;
}
```

**Session boundary**: trainer resets `RL_SESSION_PNL_USD_INDEX`, `RL_SESSION_DD_TRIGGERED_INDEX`, `RL_CONSEC_LOSS_COUNT_INDEX` to 0 on each fold boundary (start of train phase, start of eval phase).

### Layer 2 — IQN risk-averse action selection

The genuinely-novel insight from research: **we already compute the IQN quantile distribution**. The standard action selector picks `argmax(E_τ[Q(s, a, τ)])` — discards the distribution shape. Instead, pick `argmax(Q(s, a, τ_action))` with adaptive `τ_action ∈ (0, 0.5]` — lower τ = more pessimistic = downside-aware decisions.

**New ISV slots**:

| Slot | Name | Purpose | Bootstrap |
|---|---|---|---|
| 671 | `RL_IQN_ACTION_TAU_INDEX` | Quantile used for action selection ∈ (0, 1] | 0.5 (= median, near current E[Q] behavior) |
| 672 | `RL_IQN_ACTION_TAU_MIN_INDEX` | Lower bound | 0.1 |
| 673 | `RL_IQN_ACTION_TAU_DD_SENSITIVITY_INDEX` | How fast τ drops with drawdown | 5.0 |

**Adaptive controller `rl_iqn_action_tau_controller.cu`** (~50 LOC):

```c
// Adapt action-selection quantile based on recent drawdown.
// Normal trading: τ = 0.5 (median ≈ expected-Q-like behavior).
// In drawdown: τ → MIN (pessimistic — only take trades where even
// the worst-case Q is positive).
//
// Recent drawdown signal: (session_pnl - max_session_pnl) / |starting_capital|
// Bounded in [0, 1]: 0 = at peak, 1 = total wipeout.
// τ_action = max(τ_min, 0.5 - sensitivity × drawdown_frac)

extern "C" __global__ void rl_iqn_action_tau_controller(
    float* __restrict__ isv
) {
    if (threadIdx.x != 0 || blockIdx.x != 0) return;

    const float session_pnl = isv[RL_SESSION_PNL_USD_INDEX];
    const float max_pnl     = isv[RL_SESSION_MAX_PNL_USD_INDEX];  // tracked separately
    const float capital     = isv[RL_STARTING_CAPITAL_USD_INDEX];

    const float drawdown_usd  = fmaxf(0.0f, max_pnl - session_pnl);
    const float drawdown_frac = drawdown_usd / fmaxf(capital, 1.0f);

    const float tau_min     = isv[RL_IQN_ACTION_TAU_MIN_INDEX];
    const float sensitivity = isv[RL_IQN_ACTION_TAU_DD_SENSITIVITY_INDEX];
    const float tau_target  = fmaxf(tau_min, 0.5f - sensitivity * drawdown_frac);
    isv[RL_IQN_ACTION_TAU_INDEX] = tau_target;
}
```

**Modification to existing action selection kernel** (e.g., `rl_ensemble_action_value.cu` or wherever the argmax over Q happens):

```c
// BEFORE:
const float exp_q = compute_expected_q(s, a);  // mean over sampled τ
// argmax over actions on exp_q

// AFTER:
const float tau_a = isv[RL_IQN_ACTION_TAU_INDEX];  // adaptive, default 0.5
const float q_at_tau = compute_q_at_quantile(s, a, tau_a);
// argmax over actions on q_at_tau
```

The IQN kernel already computes `Q(s, a, τ)` for sampled `τ`. Selecting a specific quantile is a single-line indexing change.

**Critical interaction with C51 ensemble**: the agent uses C51+IQN ensemble. The ensemble blend (α) is already ISV-driven. We apply the risk-averse τ to the IQN side only, then blend as usual. The C51 side continues to use expected value because C51 doesn't naturally expose quantiles.

### Layer 3 — Inventory penalty (Avellaneda-Stoikov)

Add a reward shaping term proportional to `|net_position_per_batch|`. Discourages systematic one-sided exposure that bleeds via adverse selection.

**New ISV slots**:

| Slot | Name | Purpose | Bootstrap |
|---|---|---|---|
| 674 | `RL_INVENTORY_PENALTY_BETA_INDEX` | Penalty coefficient per |net_position| unit | 0.0 (sentinel — adaptive on first observation) |
| 675 | `RL_INVENTORY_VARIANCE_EMA_INDEX` | Observed inventory variance for β scaling | 0.0 sentinel |

**Modification to `rl_fused_reward_pipeline.cu`** (~30 LOC):

```c
// After raw reward computation, before clamps:
const float net_pos = net_position_per_batch[b];  // signed contracts
const float beta    = isv[RL_INVENTORY_PENALTY_BETA_INDEX];
const float penalty = beta * fabsf(net_pos);  // always reduces reward
shaped_reward = raw_reward - penalty;
```

**Adaptive controller `rl_inventory_beta_controller.cu`** (~50 LOC) — derives β from observed inventory variance:

```c
// β should scale so penalty is meaningful when inventory drifts but
// negligible during balanced trading.
// Target: penalty ≈ 1% of typical absolute reward when inventory at ~2σ.
//
// β = 0.01 × E[|reward|] / (2 × sqrt(var(inventory)))
//
// Sentinel-zero: hold β = 0 until variance EMA accumulates real signal.

const float reward_mag = isv[RL_REWARD_MAGNITUDE_EMA_INDEX];  // existing slot 614
const float inv_var    = isv[RL_INVENTORY_VARIANCE_EMA_INDEX];
if (inv_var <= 0.0f || reward_mag <= 0.0f) return;

const float inv_std = sqrtf(inv_var);
const float beta_target = 0.01f * reward_mag / (2.0f * inv_std);
// Wiener blend with floor 0.4 (existing pearl pattern)
isv[RL_INVENTORY_PENALTY_BETA_INDEX] = wiener_blend(
    isv[RL_INVENTORY_PENALTY_BETA_INDEX],
    beta_target,
    isv[RL_WIENER_ALPHA_FLOOR_INDEX]
);
```

### Layer 4 — Kelly-fraction position sizing

The original spec's contribution, demoted to fine-tuning layer. Operates *after* Layer 2 has filtered actions and Layer 1 has applied safety gates. Sizes the trades that survive.

**New ISV slots**:

| Slot | Name | Purpose | Bootstrap |
|---|---|---|---|
| 676 | `RL_KELLY_FRACTION_INDEX` | Position-size multiplier ∈ [0, 1] | 1.0 (full size pre-warmup) |
| 677 | `RL_WIN_RATE_EMA_INDEX` | Observed win_rate EMA | 0.5 (neutral sentinel) |
| 678 | `RL_AVG_WIN_USD_EMA_INDEX` | Observed avg_win | 0.0 sentinel |
| 679 | `RL_AVG_LOSS_USD_EMA_INDEX` | Observed avg_loss | 0.0 sentinel |
| 680 | `RL_KELLY_SAFETY_FRAC_INDEX` | Half-Kelly multiplier (Thorp) | 0.5 |
| 681 | `RL_KELLY_MIN_TRADES_FOR_RELEASE_INDEX` | Warmup gate | 1000.0 |

**Kernel `rl_kelly_fraction_controller.cu`** (~80 LOC) — same design as original spec, repeated here for completeness:

```c
extern "C" __global__ void rl_kelly_fraction_controller(float* __restrict__ isv) {
    if (threadIdx.x != 0 || blockIdx.x != 0) return;

    // Warmup: hold at bootstrap until enough closed trades.
    if (isv[RL_CUMULATIVE_DONES_INDEX] < isv[RL_KELLY_MIN_TRADES_FOR_RELEASE_INDEX]) {
        isv[RL_KELLY_FRACTION_INDEX] = 1.0f;
        return;
    }

    const float p = isv[RL_WIN_RATE_EMA_INDEX];
    const float avg_win  = isv[RL_AVG_WIN_USD_EMA_INDEX];
    const float avg_loss = isv[RL_AVG_LOSS_USD_EMA_INDEX];
    if (avg_loss <= 0.0f) { isv[RL_KELLY_FRACTION_INDEX] = 1.0f; return; }

    const float b = avg_win / avg_loss;
    const float f_kelly = (p * b - (1.0f - p)) / b;
    const float safety  = isv[RL_KELLY_SAFETY_FRAC_INDEX];
    float       f       = f_kelly * safety;
    f = fmaxf(0.0f, fminf(f, 1.0f));
    isv[RL_KELLY_FRACTION_INDEX] = f;
}
```

**Modification in `actions_to_market_targets.cu`** (~3 LOC):

```c
// After Layer 1 overrides have been applied (DD CB, cooldown, etc.):
const float kelly = isv[RL_KELLY_FRACTION_INDEX];
target_lots[b] = (int)ceilf((float)target_lots[b] * kelly);
// If kelly = 0, target_lots = 0 = no commitment.
```

### Layer D — TrailTighten/Loosen wiring

Resolve the 10%-policy-mass-wasted gap per [[pearl_dead_trail_stop_actions_a7_a8]]:

**New ISV slots**:

| Slot | Name | Purpose | Bootstrap |
|---|---|---|---|
| 682 | `RL_TRAIL_TIGHTEN_FACTOR_INDEX` | Multiplier on trail distance for a7 | 0.9 (tighten by 10%) |
| 683 | `RL_TRAIL_LOOSEN_FACTOR_INDEX` | Multiplier on trail distance for a8 | 1.1 (loosen by 10%) |

**Modification to `actions_to_market_targets.cu`** wires a7/a8 to actually mutate `unit_trail_distance_d`:

```c
case ACTION_TRAIL_TIGHTEN: {
    const float factor = isv[RL_TRAIL_TIGHTEN_FACTOR_INDEX];
    for (int u = 0; u < MAX_UNITS; u++) {
        if (unit_active_d[b * MAX_UNITS + u]) {
            unit_trail_distance_d[b * MAX_UNITS + u] *= factor;
        }
    }
    break;
}
case ACTION_TRAIL_LOOSEN: {
    const float factor = isv[RL_TRAIL_LOOSEN_FACTOR_INDEX];
    for (int u = 0; u < MAX_UNITS; u++) {
        if (unit_active_d[b * MAX_UNITS + u]) {
            unit_trail_distance_d[b * MAX_UNITS + u] *= factor;
        }
    }
    break;
}
```

---

## Slot allocation summary

22 new ISV slots, `RL_SLOTS_END` 662 → 684.

| Range | Layer | Purpose |
|---|---|---|
| 662–670 | 1 (CMDP gates) | Session pnl, DD limit, max open, consec loss, cooldown, inventory cap (9 slots) |
| 671–673 | 2 (IQN τ) | Action τ, τ_min, DD sensitivity (3 slots) |
| 674–675 | 3 (inventory penalty) | β + inventory variance EMA (2 slots) |
| 676–681 | 4 (Kelly) | Kelly fraction + EMAs + safety + warmup (6 slots) |
| 682–683 | D (trail) | Tighten/loosen factors (2 slots) |

Memory impact: 88 bytes additional (negligible).

---

## Per-step pipeline order

The per-step trainer pipeline must execute Layers in this order:

1. EMA producers (existing — kl_pi_ema, q_divergence_ema, etc., PLUS new: win_rate_ema, avg_win_ema, avg_loss_ema, inventory_variance_ema)
2. Welford variance updates (existing kernel from b1ef6664a)
3. Adaptive controllers (existing 12 — fused kernel)
4. **Layer 1 — `rl_cmdp_constraints_check`** (new)
5. **Layer 2 — `rl_iqn_action_tau_controller`** (new — adapts τ from drawdown)
6. **Layer 3 — `rl_inventory_beta_controller`** (new — adapts β)
7. **Layer 4 — `rl_kelly_fraction_controller`** (new)
8. **Layer 5 — TrailTighten/Loosen ISV slots** (no per-step kernel; ISV slots read during action translation)
9. Action selection — IQN ensemble uses adaptive τ for argmax
10. `actions_to_market_targets` — applies Layer 1 overrides + Layer 4 Kelly scaling + Layer D trail mutations
11. Reward computation — `rl_fused_reward_pipeline` applies Layer 3 inventory penalty
12. PPO/DQN updates as usual

---

## Files

### New

- `crates/ml-alpha/cuda/rl_cmdp_constraints_check.cu` (~100 LOC)
- `crates/ml-alpha/cuda/rl_iqn_action_tau_controller.cu` (~50 LOC)
- `crates/ml-alpha/cuda/rl_inventory_beta_controller.cu` (~50 LOC)
- `crates/ml-alpha/cuda/rl_kelly_fraction_controller.cu` (~80 LOC)
- `crates/ml-alpha/cuda/rl_win_rate_ema_update.cu` (~40 LOC if not derivable)
- `crates/ml-alpha/cuda/rl_avg_win_loss_ema_update.cu` (~50 LOC)
- `crates/ml-alpha/cuda/rl_inventory_variance_update.cu` (~40 LOC)
- `crates/ml-alpha/tests/risk_stack_invariants.rs` (~10 GPU-oracle gates)

### Modified

- `crates/ml-alpha/src/rl/isv_slots.rs` — +22 slots, `RL_SLOTS_END` 662 → 684
- `crates/ml-alpha/build.rs` — register new cubins
- `crates/ml-alpha/cuda/actions_to_market_targets.cu` — Layer 1 overrides + Layer 4 Kelly scaling + Layer D trail mutations
- `crates/ml-alpha/cuda/rl_fused_reward_pipeline.cu` — Layer 3 inventory penalty term
- `crates/ml-alpha/cuda/rl_ensemble_action_value.cu` (or wherever IQN action selection happens) — Layer 2 quantile-τ argmax
- `crates/ml-alpha/cuda/rl_fused_controllers.cu` — add new branches for τ, β, Kelly (matching the fused-kernel pattern from b1ef6664a)
- `crates/ml-alpha/src/trainer/integrated.rs` — load new cubins, wire per-step launches, bootstrap entries, session reset on fold boundary

### Not touched

- 12 adaptive controllers from b1ef6664a (orthogonal layer)
- Welford-variance kernel infrastructure (reused)
- Phase 4.5 advantage normalization
- Perception encoder, Mamba2, CfC heads
- Decision policy gradient (the agent still picks discrete actions)

---

## Validation gates

### Local unit-level (RTX 3050 Ti)

**G1 — CMDP DD trigger**: Write `session_pnl = -4000` (below limit -3500) → assert triggered=1, agent actions overridden to Hold.

**G2 — CMDP cooldown**: Trigger 10 consecutive losses → assert cooldown counter set to 500, agent actions overridden until counter reaches 0.

**G3 — CMDP max-open**: Pre-set 4 active units, agent picks opening action → assert action forced to Hold.

**G4 — CMDP inventory cap**: Set `net_position = $11k` (above $10k limit), agent picks opening-same-side action → assert Hold.

**G5 — IQN τ adapts to drawdown**: Set `session_pnl = -1500` (≈4% drawdown of $35k capital) → assert `τ_action ≈ max(0.1, 0.5 - 5 × 0.043) ≈ 0.285` (mid-pessimistic).

**G6 — IQN τ at no-drawdown**: Set `session_pnl = max_pnl` → assert `τ_action = 0.5`.

**G7 — Inventory penalty β adapts**: Set `reward_mag_ema = 0.1`, `inv_variance_ema = 4.0` (σ=2) → assert `β = 0.01 × 0.1 / (2 × 2) = 0.00025`. Negligible at balanced inventory, real at high.

**G8 — Kelly math at known edge**: Feed `win_rate=0.55, avg_win=1.8, avg_loss=1.0, trades=2000` → expect `f = 0.5 × ((0.55×1.8 - 0.45)/1.8) = 0.5 × 0.30 = 0.15`.

**G9 — Kelly returns 0 at negative edge**: Feed `win_rate=0.21, avg_win=1.8, avg_loss=1.0, trades=2000` → expect `f = 0` (Kelly negative → clamped).

**G10 — TrailTighten reduces distance**: Pre-set `unit_trail_distance = 100`, agent picks a7 → assert distance becomes 90.

**G11 — integrated_trainer_smoke** still passes end-to-end.

**G12 — 1k local smoke at b=128**: completed_clean, no NaN, all 5 layers' ISV slots populated and tracking sensibly.

### Cluster

**G13 — Walk-forward fold 1 re-run** (same params as today's 62x7p, new SHA). Combined layers should close the eval gap.

Targets (each is a soft pass; combination of 3+ passes = G13 PASS):
- eval pnl > -$200k (better than today's -$616k, **PRIMARY**)
- profit_factor > 0.90 (vs 0.83)
- max_drawdown_pct < 15% (vs 18.7%)
- Kelly fraction: ends < 1.0 if observed edge negative (shows Layer 4 active)
- IQN τ: ends < 0.5 if any drawdown occurred (shows Layer 2 adaptive)
- DD breaker: fires when expected (or never if pnl stays positive)
- Inventory penalty β: > 0 throughout (shows Layer 3 adapted off bootstrap)

**G14 — ksll2 regression test**: same params as today's rwk9t. Final pnl ≥ +$13M (within 20% of +$15.81M). Confirms layered stack doesn't regress large-data regime.

---

## Open questions

1. **Starting capital ($35k) per [[project_ml_alpha_starting_capital]]** — should we add `RL_STARTING_CAPITAL_USD_INDEX` as an ISV slot? Yes, makes session DD cap and Layer 2 drawdown ratio both adaptive to capital changes. Add to Layer 1.

2. **Layer 2 sensitivity (5.0)** — at session DD 10%, τ drops from 0.5 to 0. At 4% drawdown, τ ≈ 0.3. Reasonable? Tunable via ISV. May need empirical sweep.

3. **Inventory variance EMA — measured over what window?** Need to choose Wiener-α. Default `RL_WIENER_ALPHA_FLOOR_INDEX = 0.4` from existing pearl. Adaptive could come later.

4. **Layer 3 β bootstrap (0.0 = sentinel)** — agent gets ZERO inventory penalty until variance EMA accumulates. Acceptable cold-start.

5. **Should `RL_MAX_OPEN_UNITS_INDEX` = 4 (current `MAX_UNITS`) be redundant?** The action space already enforces this via unit_active tracking. Yes, redundant but cheap; treats it as belt-and-braces.

6. **Approach E vs the original B+D — anything we lose?** The `rl_session_drawdown_check.cu` kernel from the prior design is now folded into the broader `rl_cmdp_constraints_check.cu`. Same functionality, more layered.

7. **Multi-seed validation?** A single fold-1 result is noisy (1024 trades, high variance). Worth running 3 seeds to confirm. Defer to post-validation cluster sweep if G13 single-seed result is marginal.

8. **Regime detection (Approach E+1)?** FineFT (KDD 2026) demonstrates risk-aware ensemble routing on regime changes. Heavyweight implementation (online clustering + meta-controller). Defer to follow-up spec if Approach E doesn't close the eval gap.

---

## Risks and mitigations

- **Layer 2 τ-quantile path may not be exposed by current IQN kernel signature** → audit `rl_ensemble_action_value.cu` first; if `Q(s, a, τ)` isn't readily indexable, may need kernel restructure (small change).

- **Kelly EMA estimates unreliable in early training** → warmup gate (1000 trades) holds Kelly at bootstrap. Same pattern as b1ef6664a's reward_floor cumulative-dones gate.

- **Five layers compounding may make agent ultra-conservative — eval might be 0 with no trades** → that's actually fine for G13 (eval pnl > -$200k bar). But for G14 (ksll2 +$13M), need to verify layers don't over-restrict on full-data regime. Mitigation: warmup gates default to "behave like the prior code" until enough signal accumulates.

- **Session boundary resets must be wired by trainer** → must call ISV resets on train→eval transition. Test G11 catches this. Plan task explicitly enumerates the reset points.

- **Approach D wiring changes action distribution dynamics** → trainable but worth a regression check on entropy. 1k smoke catches it.

- **CMDP "force Hold" creates discontinuity in policy gradient** → standard technique in CMDP-RL literature, well-understood. Override doesn't update gradient through the constraint, only through the Hold action's natural Q.

---

## Next step after approval

Write the plan: `docs/superpowers/plans/2026-05-30-adaptive-risk-management-plan.md`. Task-level decomposition:

1. ISV slot allocation (22 slots)
2. Layer 1 (CMDP constraints kernel + actions_to_market_targets overrides)
3. Layer 2 (IQN τ controller + action-selection kernel modification)
4. Layer 3 (inventory penalty in reward + β controller)
5. Layer 4 (Kelly controller + actions_to_market_targets sizing)
6. Layer D (TrailTighten/Loosen wiring)
7. New EMA producers (win_rate, avg_win, avg_loss, inventory variance)
8. Fused-kernel integration of new controllers
9. Trainer integration (per-step launches + bootstrap + session reset)
10. GPU-oracle invariant tests (G1-G11)
11. 1k local smoke + sanitizer (G12)
12. Atomic commit
13. Cluster G13 (fold 1 re-run) + G14 (ksll2 regression)

Expected scope: ~320 LOC across 8 new + 7 modified files, ~13 tasks, single atomic commit. Same `feedback_no_partial_refactor` discipline.
