foxhunt/WAVE_8_6_TEST_UPDATES.md

# Wave 8.6: Test File Updates for Proper Weight Initialization

**Objective**: Replace `VarBuilder::zeros()` with `VarBuilder::from_varmap()` in all TFT test files

**Rationale**: `VarBuilder::zeros()` creates all-zero weights, bypassing proper Xavier Uniform initialization

---

## Summary of Changes Needed

### Files to Update: 4
- `ml/src/tft/gated_residual.rs` - 8 tests
- `ml/src/tft/temporal_attention.rs` - 2 tests
- `ml/src/tft/variable_selection.rs` - 5 tests
- `ml/src/tft/quantile_outputs.rs` - 6 tests

### Total Tests: 21

---

## Change Pattern

### Add Import
```rust
use std::sync::Arc;  // Add if not present
use candle_nn::{VarBuilder, VarMap};  // Update if only VarBuilder imported
```

### Replace Pattern
```rust
// ❌ BEFORE
let vs = VarBuilder::zeros(DType::F32, &device);

// ✅ AFTER
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

---

## Detailed Changes by File

### 1. ml/src/tft/gated_residual.rs (8 tests)

**Line 225** - `test_grn_creation`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 236** - `test_grn_forward_same_dims`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 252** - `test_grn_forward_different_dims`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 268** - `test_grn_forward_with_context`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 287** - `test_grn_forward_3d`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 303** - `test_glu_creation`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 313** - `test_glu_forward`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 328** - `test_grn_stack`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

---

### 2. ml/src/tft/temporal_attention.rs (2 tests)

**Line 382** - `test_temporal_self_attention_creation`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 423** - `test_interpretable_multi_head_attention`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

---

### 3. ml/src/tft/variable_selection.rs (5 tests)

**Line 193** - `test_variable_selection_network_creation`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 204** - `test_variable_selection_forward`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 222** - `test_attention_weights`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 240** - `test_variable_selection_3d`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 261** - `test_zero_attention_fallback`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

---

### 4. ml/src/tft/quantile_outputs.rs (6 tests)

**Line 260** - `test_quantile_output_creation`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 273** - `test_quantile_output_forward`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 293** - `test_quantile_output_monotonicity`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 311** - `test_quantile_loss`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 329** - `test_median_quantile`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

**Line 361** - `test_asymmetric_loss`
```rust
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);
```

---

## Automated Update Script

```bash
#!/bin/bash
# Script to update VarBuilder initialization in test files

FILES=(
    "ml/src/tft/gated_residual.rs"
    "ml/src/tft/temporal_attention.rs"
    "ml/src/tft/variable_selection.rs"
    "ml/src/tft/quantile_outputs.rs"
)

for file in "${FILES[@]}"; do
    echo "Updating $file..."

    # Add import if not present
    if ! grep -q "use std::sync::Arc;" "$file"; then
        sed -i '1i use std::sync::Arc;' "$file"
    fi

    # Replace VarBuilder::zeros pattern
    sed -i 's/let vs = VarBuilder::zeros(DType::F32, \&device);/let varmap = Arc::new(VarMap::new());\n    let vs = VarBuilder::from_varmap(\&varmap, DType::F32, \&device);/g' "$file"

    echo "✓ Updated $file"
done

echo ""
echo "All files updated. Run tests to verify:"
echo "cargo test -p ml --lib tft"
```

---

## Verification After Updates

### 1. Run TFT Tests
```bash
cargo test -p ml --lib tft --no-fail-fast
```

**Expected**: All tests should pass with non-zero outputs

### 2. Run Weight Initialization Tests
```bash
cargo test --test test_grn_weight_initialization -p ml
```

**Expected**: All 9 tests should pass

### 3. Run Verification Example
```bash
cargo run --example verify_grn_weight_init -p ml
```

**Expected Output**:
```
✓ PASS: Weights are properly initialized (non-zero variance)
✓ PASS: Different inputs produce different outputs
✓ PASS: Context has measurable effect
```

---

## Why This Matters

### Problem with VarBuilder::zeros()
1. Creates all-zero weight matrices
2. Bypasses Xavier Uniform initialization
3. Tests only verify shape/dimension handling
4. Does NOT validate actual weight initialization behavior

### Benefits of VarBuilder::from_varmap()
1. ✅ Proper Xavier Uniform initialization
2. ✅ Non-zero, normally distributed weights
3. ✅ Maintains gradient flow during training
4. ✅ Matches production code behavior

---

## Impact Assessment

### Test Behavior Before Fix
- ❌ All outputs are zero (or near-zero due to bias terms)
- ❌ Different inputs produce same outputs
- ❌ Context has no effect
- ✅ Shape/dimension tests pass (misleading success)

### Test Behavior After Fix
- ✅ Outputs have non-zero variance
- ✅ Different inputs produce different outputs
- ✅ Context integration works correctly
- ✅ Tests validate actual initialization behavior

---

## References

- **Main Report**: `WAVE_8_6_GRN_WEIGHT_INITIALIZATION.md`
- **Quick Reference**: `WAVE_8_6_QUICK_REFERENCE.md`
- **Test Suite**: `ml/tests/test_grn_weight_initialization.rs`
- **Example**: `ml/examples/verify_grn_weight_init.rs`

---

**Status**: Ready for implementation
**Priority**: Medium (tests need updating, production code is correct)
**Estimated Time**: 15 minutes (manual) or 5 minutes (automated script)