foxhunt/AGENT_225_GRADIENT_PRIORITY_2_FIXES.md

# Agent 225: Priority 2 Gradient Tracking Fixes

**Date**: 2025-10-15
**Agent**: 225
**Task**: Apply Agent 219's Priority 2 gradient extraction and optimizer integration fixes
**Status**: ✅ COMPLETE
**Files Modified**: `ml/src/mamba/mod.rs`

---

## Summary

Agent 225 successfully applied Priority 2 fixes from Agent 219's Quick Fix Guide to enable proper gradient flow in the MAMBA-2 SSM training pipeline. These fixes extract gradients after the backward pass and populate the optimizer's gradient HashMap for parameter updates.

**Key Changes**:
1. Modified `backward_pass()` to extract gradients from SSM parameters (A, B, C, delta matrices)
2. Verified `optimizer_step()` consumes gradients using layer-specific keys
3. Added trace logging for debugging gradient extraction and parameter updates
4. Compilation verified with `cargo check -p ml` (passed with minor warnings only)

---

## Priority 2 Fixes Applied

### Fix #4: Extract Gradients After Backward Pass

**Location**: `ml/src/mamba/mod.rs`, lines 1237-1302 (backward_pass function)

**Problem**: After calling `loss.backward()`, gradients were computed but never extracted from the SSM parameter tensors, leaving `self.gradients` HashMap empty for the optimizer.

**Solution**: Added gradient extraction loop that:
- Iterates through all SSM layers (`self.state.ssm_states`)
- Extracts gradients using `tensor.grad()?` method for each parameter (A, B, C, delta)
- Stores gradients in `self.gradients` HashMap with layer-specific keys: `"A_0"`, `"B_0"`, `"C_0"`, `"delta_0"`, etc.
- Adds trace logging for debugging gradient extraction

**Code Changes**:

```rust
fn backward_pass(
    &mut self,
    loss: &Tensor,
    _input: &Tensor,
    _target: &Tensor,
) -> Result<(), MLError> {
    // Compute gradients using automatic differentiation
    loss.backward()?;  // Changed from: let _grad = loss.backward()?;

    // PRIORITY 2 FIX (Agent 225): Extract gradients from SSM parameters after backward()
    trace!("[Agent 225] Extracting gradients from SSM parameters");
    self.gradients.clear();
    for (layer_idx, ssm_state) in self.state.ssm_states.iter().enumerate() {
        if let Some(A_grad) = ssm_state.A.grad()? {
            self.gradients.insert(format!("A_{}", layer_idx), A_grad);
            trace!("[Agent 225] Extracted A gradient for layer {}", layer_idx);
        }
        if let Some(B_grad) = ssm_state.B.grad()? {
            self.gradients.insert(format!("B_{}", layer_idx), B_grad);
            trace!("[Agent 225] Extracted B gradient for layer {}", layer_idx);
        }
        if let Some(C_grad) = ssm_state.C.grad()? {
            self.gradients.insert(format!("C_{}", layer_idx), C_grad);
            trace!("[Agent 225] Extracted C gradient for layer {}", layer_idx);
        }
        if let Some(delta_grad) = ssm_state.delta.grad()? {
            self.gradients.insert(format!("delta_{}", layer_idx), delta_grad);
            trace!("[Agent 225] Extracted delta gradient for layer {}", layer_idx);
        }
    }

    self.clip_gradients(self.config.grad_clip)?;

    // Gradient stability check (updated to use layer-specific keys)
    for layer_idx in 0..self.state.ssm_states.len() {
        for param_name in &["A", "B", "C", "delta"] {
            let key = format!("{}_{}", param_name, layer_idx);
            if let Some(grad) = self.gradients.get(&key) {
                let grad_norm = grad.sqr()?.sum_all()?.to_vec0::<f64>()?.sqrt();
                if grad_norm.is_nan() || grad_norm.is_infinite() {
                    warn!("[Agent 225] Unstable gradient detected in {}: {}", key, grad_norm);
                    return Err(MLError::NumericalError(format!(
                        "Unstable gradient in {}: {}",
                        key, grad_norm
                    )));
                }
            }
        }
    }

    Ok(())
}
```

---

### Fix #6: Update Optimizer to Use Layer-Specific Keys

**Location**: `ml/src/mamba/mod.rs`, lines 1346-1445 (optimizer_step function)

**Status**: ✅ Already implemented (verified, no changes needed)

**Implementation**: The optimizer already uses layer-specific gradient keys with the pattern:

```rust
fn optimizer_step(&mut self) -> Result<(), MLError> {
    // ... Adam hyperparameters setup ...

    // PRIORITY 2 FIX: Use layer-specific gradient keys
    let num_layers = self.state.ssm_states.len();
    for layer_idx in 0..num_layers {
        // Collect layer-specific gradients
        let a_grad = self.gradients.get(&format!("A_{}", layer_idx)).cloned();
        let b_grad = self.gradients.get(&format!("B_{}", layer_idx)).cloned();
        let c_grad = self.gradients.get(&format!("C_{}", layer_idx)).cloned();
        let delta_grad = self.gradients.get(&format!("delta_{}", layer_idx)).cloned();

        // Apply Adam updates to each parameter
        if let Some(ref A_grad) = a_grad {
            trace!("[Agent 225] Updating A matrix for layer {}", layer_idx);
            // ... Adam update logic ...
        }
        // ... similar for B, C, delta matrices ...
    }

    Ok(())
}
```

**Key Pattern**:
- Gradient keys use format: `format!("A_{}", layer_idx)`, `format!("B_{}", layer_idx)`, etc.
- Gradients are collected INSIDE the layer loop for proper per-layer parameter updates
- This enables multi-layer MAMBA-2 architectures with independent parameter learning

---

## Technical Details

### Gradient Flow Architecture

```
Training Batch → Forward Pass → Loss Computation
                                      ↓
                               loss.backward()  ← Compute gradients via autodiff
                                      ↓
                        Extract gradients from parameters
                                      ↓
                    Store in HashMap<String, Tensor>
                    ("A_0" → A_grad_layer_0, "B_0" → B_grad_layer_0, ...)
                                      ↓
                            Gradient Clipping
                                      ↓
                          Gradient Stability Check
                                      ↓
                            optimizer_step()
                                      ↓
                    Retrieve gradients per layer
                    (layer 0: get "A_0", "B_0", "C_0", "delta_0")
                                      ↓
                        Apply Adam Updates
                        (m_t, v_t, parameter updates)
                                      ↓
                    Update SSM parameters in-place
```

### SSM Parameter Gradients

Each MAMBA-2 layer has 4 learnable parameter matrices:

1. **A (State Transition Matrix)**: `(d_state, d_state)` - Controls hidden state evolution
2. **B (Input Matrix)**: `(d_state, d_inner)` - Projects input into state space
3. **C (Output Matrix)**: `(d_inner, d_state)` - Projects state to output
4. **delta (Discretization)**: `(d_model,)` - Time-step scaling factor

For a 2-layer MAMBA-2 model, the gradient HashMap contains:
- `"A_0"`, `"A_1"` - State transition gradients per layer
- `"B_0"`, `"B_1"` - Input projection gradients per layer
- `"C_0"`, `"C_1"` - Output projection gradients per layer
- `"delta_0"`, `"delta_1"` - Discretization gradients per layer

### Gradient Extraction Pattern

```rust
// For each SSM layer
for (layer_idx, ssm_state) in self.state.ssm_states.iter().enumerate() {
    // Extract gradient from Tensor (if computed during backward pass)
    if let Some(A_grad) = ssm_state.A.grad()? {
        // Store with unique key: "A_0", "A_1", etc.
        self.gradients.insert(format!("A_{}", layer_idx), A_grad);
    }
}
```

**Why Layer-Specific Keys?**
- Enables multi-layer architectures (MAMBA-2 can have N layers)
- Each layer learns independently during training
- Prevents gradient conflicts between layers
- Supports heterogeneous learning rates per layer (future enhancement)

---

## Verification

### Compilation Check

```bash
cargo check -p ml
```

**Result**: ✅ PASSED

```
    Checking ml v0.1.0 (/home/jgrusewski/Work/foxhunt/ml)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.68s
```

**Warnings** (non-critical):
- Unused imports (candle_nn components not used in current scope)
- Unnecessary qualifications (Mamba2Config::default() can be simplified)

These warnings do not affect functionality and can be cleaned up in a separate pass.

---

## Dependencies

### Agent 224 Prerequisites (Priority 1)

Agent 225's work depends on Agent 224 completing Priority 1 fixes:

✅ **Fix #1: Remove Gradient Detach** (Agent 224 completed)
- Location: `ml/src/mamba/mod.rs`, line ~1010-1017
- Changed: `let input = input.detach();` → `let input = input;`
- Impact: Gradients now flow through input tensor during backward pass

Without Agent 224's fix, gradients would be severed at the input layer, making gradient extraction in Priority 2 useless.

---

## Impact on Training

### Before Priority 2 Fixes

```rust
// Gradients computed but never extracted
let _grad = loss.backward()?;

// self.gradients HashMap remains empty
self.clip_gradients(self.config.grad_clip)?;

// optimizer_step() gets EMPTY HashMap
// No parameter updates occur
// Training stalls (loss doesn't decrease)
```

### After Priority 2 Fixes

```rust
// Gradients computed
loss.backward()?;

// Gradients extracted and stored
self.gradients = {
    "A_0": tensor([...]),   // Layer 0 state transition gradient
    "B_0": tensor([...]),   // Layer 0 input projection gradient
    "C_0": tensor([...]),   // Layer 0 output projection gradient
    "delta_0": tensor([...]),  // Layer 0 discretization gradient
    // ... additional layers ...
}

// optimizer_step() consumes gradients
// Adam updates applied to all SSM parameters
// Training progresses (loss decreases)
```

**Expected Training Behavior**:
- Gradients properly flow from loss to optimizer
- SSM parameters update on each training step
- Loss decreases over epochs (convergence)
- Model learns temporal dependencies in data

---

## Testing

### Manual Verification

Run MAMBA-2 training test:

```bash
cargo test -p ml --test e2e_mamba2_training -- --nocapture
```

**Expected Output**:
- ✅ Gradients computed (backward pass successful)
- ✅ Gradients extracted (self.gradients HashMap populated)
- ✅ Parameters updating (Adam optimizer applies updates)
- ✅ Loss decreasing (training convergence)

**Trace Logging** (with `RUST_LOG=trace`):
```
TRACE [Agent 225] Extracting gradients from SSM parameters
TRACE [Agent 225] Extracted A gradient for layer 0
TRACE [Agent 225] Extracted B gradient for layer 0
TRACE [Agent 225] Extracted C gradient for layer 0
TRACE [Agent 225] Extracted delta gradient for layer 0
TRACE [Agent 225] Updating A matrix for layer 0
TRACE [Agent 225] Updating B matrix for layer 0
...
```

### Integration Test Expectations

The E2E training test should now:
1. Load synthetic training data (sequence prediction task)
2. Initialize MAMBA-2 model with gradient tracking enabled
3. Run forward pass and compute loss
4. Run backward pass (gradients computed via autodiff)
5. Extract gradients from SSM parameters ← **Agent 225's work**
6. Clip gradients and check stability
7. Apply optimizer updates using extracted gradients ← **Agent 225's work**
8. Verify loss decreases over training epochs

**Success Criteria**:
- Loss decreases by >10% after 100 training steps
- No gradient explosions (all gradients < 1e6)
- No NaN/Inf values in parameters or gradients
- Parameter norms increase (learning is occurring)

---

## Next Steps

### Agent 226 (Priority 3)

Agent 226 will apply remaining fixes from Agent 219's guide:

**Fix #2: Enable SSM Gradient Tracking** (lines 259-286)
- Add `.requires_grad(true)?` to A, B, C, delta initialization
- Ensures Candle tracks gradients during forward pass

**Fix #3: Store VarMap** (lines 358 & 377)
- Add `var_map: candle_nn::VarMap` field to Mamba2SSM struct
- Store VarMap in constructor for later gradient extraction from Linear layers

**Fix #5: Direct F64 Loss Extraction** (line 1168)
- Change `loss.to_scalar::<f32>()? as f64` → `loss.to_scalar::<f64>()?`
- Eliminates precision loss during loss value extraction

### Full Training Pipeline

Once all 6 fixes are applied (Agents 224-226):

```bash
# Run full MAMBA-2 training test
cargo test -p ml --test e2e_mamba2_training -- --nocapture

# Expected output:
# ✅ Gradient tracking enabled
# ✅ Gradients computed during backward pass
# ✅ Gradients extracted to optimizer
# ✅ Parameters updating with Adam
# ✅ Loss decreasing over epochs
# ✅ MAMBA-2 training pipeline operational
```

---

## References

- **Agent 219 Quick Fix Guide**: `/home/jgrusewski/Work/foxhunt/AGENT_219_QUICK_FIX_GUIDE.md`
- **Modified File**: `/home/jgrusewski/Work/foxhunt/ml/src/mamba/mod.rs`
- **MAMBA-2 Paper**: "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" (Gu & Dao, 2023)
- **Candle Framework**: https://github.com/huggingface/candle

---

## Agent Timeline

| Agent | Priority | Task | Status |
|-------|----------|------|--------|
| 224 | 1 | Remove gradient detach (Fix #1) | ✅ COMPLETE |
| **225** | **2** | **Extract gradients, populate optimizer (Fix #4, #6)** | ✅ **COMPLETE** |
| 226 | 3 | Enable SSM gradient tracking, store VarMap, F64 loss (Fix #2, #3, #5) | ⏳ PENDING |

**Total Estimated Time**: 2 hours (all fixes)
- Agent 224 (Priority 1): 30 minutes ✅
- **Agent 225 (Priority 2): 1 hour** ✅
- Agent 226 (Priority 3): 30 minutes ⏳

---

**Agent 225 Complete** ✅
**Compilation Status**: PASSED (cargo check -p ml)
**Next Agent**: 226 (Priority 3 fixes)