# IQL H100 Hardening Design

## Goal

Fix three production concerns before H100 baseline: readiness activates too fast on large batches, per-sample C51 support drops samples silently, staleness decay is ineffective at scale. All fixes self-calibrating, zero new hyperparameters.

## Fix 1: V(s) Inter-Batch Drift Readiness

Replace CV-based readiness (`(cv_initial - cv_current) / cv_initial`) with two-timescale EMA drift detection on V(s) mean.

### Kernel changes (`iql_adv_sigma_ema_update`)

The kernel currently computes advantage sigma EMA + CV readiness. Replace the readiness computation:

```
// Compute v_mean from v_out_buf (grid-stride loop in same single-thread kernel)
v_mean = sum(v_out_buf[0..B]) / B

// Two-timescale EMA
v_ema_fast = 0.9 * v_ema_fast + 0.1 * v_mean
v_ema_slow = 0.99 * v_ema_slow + 0.01 * v_mean

// Relative drift
drift = |v_ema_fast - v_ema_slow| / max(|v_ema_slow|, 1e-4)

// Self-calibrating scale: capture first drift after 10-step warmup
if (step_counter < 10) { readiness = 0; step_counter++; }
else if (drift_scale < 1e-8) { drift_scale = drift; readiness = 0; }
else { readiness = 1 - clamp(drift / drift_scale, 0, 1); }
```

### Buffer changes

Expand `readiness_buf` from `[2]` to `[6]`:
- `[0]` = readiness scalar
- `[1]` = drift_scale (self-calibrated)
- `[2]` = v_ema_fast
- `[3]` = v_ema_slow
- `[4]` = step_counter (as float, cast to int in kernel)
- `[5]` = reserved

### Kernel signature change

Add `const float* __restrict__ v_out_buf` and `int batch_size` parameters to `iql_adv_sigma_ema_update` so it can compute v_mean internally.

### Dead code

The CV-based readiness in the current kernel (`cv = sigma_ema / |mean|`) is replaced entirely. The `readiness_buf[1]` which stored `initial_cv` is repurposed for `drift_scale`.

## Fix 2: Frugal p5 Quantile Floor for C51 Support

### New kernel: `iql_support_floor`

Single-thread kernel (grid=1, block=1) that runs after `iql_compute_per_sample_support`:

```
// Step 1: Frugal p5 estimator update
// Read half_widths from per_sample_support (recoverable as (v_max - v_min) / 2)
for b in 0..batch_size:
    hw = (per_sample_support[b*3+1] - per_sample_support[b*3+0]) / 2
    if hw < p5_estimate:
        p5_estimate -= p5_estimate * 0.05  // proportional step down
    else:
        p5_estimate += p5_estimate * 0.05 * (0.05 / 0.95)  // asymmetric for p5

// Step 2: Floor delta_z for all samples
floor_hw = p5_estimate * 0.1
floor_dz = (2 * floor_hw) / (num_atoms - 1)
for b in 0..batch_size:
    if per_sample_support[b*3+2] < floor_dz:
        // Recompute v_min/v_max/delta_z with floored half_width
        center = (per_sample_support[b*3+0] + per_sample_support[b*3+1]) / 2
        per_sample_support[b*3+0] = center - floor_hw
        per_sample_support[b*3+1] = center + floor_hw
        per_sample_support[b*3+2] = floor_dz
```

### Buffer changes

- New: `p5_ema_buf: CudaSlice<f32>` size 1, initialized to 1.0 (safe default)
- New kernel function loaded from cubin: `iql_support_floor`

### Integration

Called from `submit_aux_ops` immediately after `compute_per_sample_support`, before `compute_branch_scales`.

## Fix 3: Buffer-Relative Staleness

### Kernel change (`iql_modulate_td_errors`)

Replace:
```cuda
float decay = expf(-staleness_lambda * (float)age / fmaxf(staleness_tau, 1.0f));
```
With:
```cuda
float age_norm = (float)age / fmaxf((float)capacity, 1.0f);
float decay = expf(-3.0f * age_norm);
```

Remove `staleness_lambda` and `staleness_tau` parameters from kernel signature. Remove `float beta` parameter (the advantage temperature is already baked into the advantage weights by `iql_compute_advantage_weights`).

Wait — `beta` is used for computing K (dynamic clamp). Keep `beta`. Only remove `staleness_lambda` and `staleness_tau`.

### Rust changes

- Remove `staleness_lambda` and `staleness_tau` from `GpuIqlConfig`
- Remove corresponding `.arg()` calls in `modulate_td_errors()` launcher
- Remove from `GpuIqlConfig::default()`
- Remove from `FusedTrainingCtx::new` where they're set to `0.001` and `10000.0`

### Dead code removed

- `GpuIqlConfig::staleness_lambda` field
- `GpuIqlConfig::staleness_tau` field
- Both default values
- Both `.arg()` calls in the Rust launcher

## Files Modified

| File | Changes |
|------|---------|
| `iql_value_kernel.cu` | Rewrite `iql_adv_sigma_ema_update` (drift readiness), new `iql_support_floor` kernel, simplify `iql_modulate_td_errors` |
| `gpu_iql_trainer.rs` | Expand `readiness_buf` to [6], add `p5_ema_buf`, load `iql_support_floor` kernel, update launcher signatures, remove staleness config |
| `fused_training.rs` | Call `support_floor` after `compute_per_sample_support`, remove staleness args from IQL config init |

## Testing

1. `test_no_nan_after_graph_capture` — basic smoke
2. `test_production_training_stability` — 10 epochs, check Best Sharpe
3. `test_walk_forward_no_overfitting_50_epochs` — 50 epochs × 3 folds, verify peak epoch moves later and Sharpe holds
