# Homeostatic Regularization — Design Spec

**Goal:** Unified self-regulating framework that maintains healthy training dynamics by adapting penalty strengths based on deviation from homeostatic targets. Replaces ad-hoc fixed lambdas (Q-mean drift=0.01, branch independence=0.01, temporal consistency=0.005) with a single principled mechanism borrowed from biological neural homeostasis.

**Problem:** Multiple training observables can drift or collapse (Q-mean, atom utilization, branch correlation, trade frequency, Q-gap). Each has its own fixed lambda that was hand-tuned. When one observable drifts faster than expected (Q-mean reached +4.89 by epoch 10 with lambda=0.01), the fixed penalty is too weak. When it's stable, the penalty wastes gradient budget. No coordinated response — each penalty operates independently.

**Architecture:** One CUDA kernel `homeostatic_regularizer` reads N observable scalars from pinned device-mapped memory, compares to N adaptive targets, produces N penalty gradients scaled by deviation². Targets self-calibrate from the first 5 epochs' healthy range. Penalties coordinate — if multiple observables drift simultaneously, total penalty budget is capped to prevent gradient domination.

---

## The Mechanism

Biological neurons maintain homeostasis through intrinsic plasticity: fire too much → excitability decreases. Fire too little → excitability increases. The system converges to a target firing rate without external tuning.

Applied to DQN training:

```
For each observable k (Q-mean, atom_util, branch_corr, trade_freq, Q-gap):
    error_k = (observed_k - target_k) / max(|target_k|, eps)   // normalized error
    penalty_k = lambda_base * error_k^2 * sign(error_k)        // quadratic, signed
    
    // Adaptive target: EMA of healthy range (epochs 1-5)
    if epoch <= 5:
        target_k = EMA(observed_k, alpha=0.3)    // calibrate
    // else: target frozen, penalties active
    
    // Budget cap: total penalty gradient <= 10% of C51 gradient
    total_penalty = sum(|penalty_k|)
    if total_penalty > budget:
        scale = budget / total_penalty
        penalty_k *= scale
```

## Observables and Targets

| Observable | Target | Why | Source |
|-----------|--------|-----|--------|
| Q-mean | 0.0 | Prevents bootstrapping drift | q_mean_scratch_pinned |
| Atom utilization | 0.85 | Keeps C51 resolution healthy | q_stats_buf[6] |
| Max branch correlation | 0.1 | Forces orthogonal branch features | branch_indep_penalty_buf |
| Trade frequency | calibrated | Prevents churn (too high) or inactivity (too low) | GPU epoch summary |
| Q-gap | calibrated | Prevents collapse (too low) or explosion (too high) | per_branch_q_gaps |
| Q-variance | calibrated | Prevents overconfidence (too low) or chaos (too high) | q_stats_buf[4] |

## Implementation

### CUDA Kernel

```cuda
extern "C" __global__ void homeostatic_regularizer(
    const float* __restrict__ observables,   /* [N_OBS] current values (pinned) */
    const float* __restrict__ targets,       /* [N_OBS] homeostatic targets (pinned) */
    float* __restrict__ penalties,           /* [N_OBS] output penalty gradients */
    float* __restrict__ total_penalty,       /* [1] sum of |penalty_k| for budget cap */
    int n_obs,
    float lambda_base,                       /* 0.01 */
    float budget_fraction                    /* 0.10 = max 10% of C51 gradient */
) {
    // Single-block kernel (n_obs <= 8)
    int k = threadIdx.x;
    if (k >= n_obs) return;
    
    float obs = observables[k];
    float tgt = targets[k];
    float denom = fmaxf(fabsf(tgt), 1e-6f);
    float error = (obs - tgt) / denom;
    
    // Quadratic signed penalty
    float pen = lambda_base * error * fabsf(error);
    penalties[k] = pen;
    
    // Atomic sum for budget cap
    atomicAdd(total_penalty, fabsf(pen));
}
```

### Rust Integration

```rust
struct HomeostaticState {
    observables_pinned: *mut f32,    // [N_OBS] pinned device-mapped
    observables_dev_ptr: u64,
    targets_pinned: *mut f32,        // [N_OBS] pinned device-mapped  
    targets_dev_ptr: u64,
    penalties_buf: CudaSlice<f32>,   // [N_OBS]
    total_penalty_buf: CudaSlice<f32>, // [1]
    kernel: CudaFunction,
    calibration_epoch: usize,        // epoch at which targets freeze (5)
    n_obs: usize,                    // number of observables (6)
}
```

### Observable Update (every training step)

```rust
fn update_observables(&self) {
    unsafe {
        let obs = self.observables_pinned;
        *obs.add(0) = *self.q_mean_scratch_pinned;      // Q-mean
        *obs.add(1) = self.utilization_ema;               // atom util
        *obs.add(2) = 0.0; // branch corr (read from penalty buf after compute)
        *obs.add(3) = self.epoch_trade_count as f32;     // trade freq
        *obs.add(4) = self.last_q_gap;                   // Q-gap
        *obs.add(5) = self.last_q_var;                   // Q-variance
    }
}
```

### Target Calibration (epochs 1-5)

```rust
fn calibrate_targets(&self, epoch: usize) {
    if epoch > self.calibration_epoch { return; }
    let alpha = 0.3_f32;
    unsafe {
        for k in 0..self.n_obs {
            let obs = *self.observables_pinned.add(k);
            let old = *self.targets_pinned.add(k);
            *self.targets_pinned.add(k) = (1.0 - alpha) * old + alpha * obs;
        }
    }
    // Override Q-mean target to always be 0.0 (no drift is always the goal)
    unsafe { *self.targets_pinned.add(0) = 0.0; }
}
```

### Penalty Application

The penalties feed into the C51 gradient (same path as the current Q-mean drift penalty). The `homeostatic_regularizer` kernel runs after Q-stats, and the penalty for observable 0 (Q-mean) replaces the current hardcoded drift penalty in `c51_grad_kernel.cu`.

For branch correlation (observable 2) and Q-gap (observable 4), the penalties feed into the respective auxiliary loss accumulator buffers.

## Interaction with Existing Components

- **Replaces**: Q-mean drift penalty (G9d), fixed branch independence lambda (G6), fixed temporal consistency lambda (G10)
- **Coordinates with**: gamma annealing (G4 uses atom_util, homeostatic also targets atom_util — the two mechanisms reinforce each other)
- **Enhances**: cost curriculum (G2) — the sigmoid center (currently hardcoded epoch 10) should be driven by model readiness: `center = epoch where training_Sharpe_ema first exceeds 0.3`. If the model finds edges quickly, costs ramp sooner. If it struggles, costs stay low longer. This replaces the hardcoded epoch 10 with an adaptive trigger.

## Success Criteria

| Metric | Current (fixed lambda) | Target (homeostatic) |
|--------|----------------------|---------------------|
| Q-mean at epoch 10 | +4.89 (drifting) | < ±1.0 (contained) |
| Atom util at epoch 10 | 87% (healthy) | > 85% (maintained) |
| Training Sharpe oscillation | ±0.3 | < ±0.15 (damped) |
| Number of hand-tuned lambdas | 5+ | 1 (lambda_base) |

## Risks

| Risk | Mitigation |
|------|-----------|
| Calibration period too short | 5 epochs × 88s = 7 min. Sufficient for healthy range. Extend to 8 if needed. |
| Budget cap too aggressive | 10% of C51 gradient. Increase to 20% if penalties never fire. |
| Targets miscalibrated from bad early epochs | Q-mean target hardcoded to 0.0. Others have sensible fallbacks. |
| Penalty coordination masks individual issues | Log each penalty_k separately for diagnosis. |
