foxhunt/docs/archive/waves/WAVE_8_6_TEST_UPDATES.md

Wave 8.6: Test File Updates for Proper Weight Initialization

Objective: Replace VarBuilder::zeros() with VarBuilder::from_varmap() in all TFT test files

Rationale: VarBuilder::zeros() creates all-zero weights, bypassing proper Xavier Uniform initialization

---

Summary of Changes Needed

Files to Update: 4

• ml/src/tft/gated_residual.rs - 8 tests
• ml/src/tft/temporal_attention.rs - 2 tests
• ml/src/tft/variable_selection.rs - 5 tests
• ml/src/tft/quantile_outputs.rs - 6 tests

Total Tests: 21

---

Change Pattern

Add Import

use std::sync::Arc;  // Add if not present
use candle_nn::{VarBuilder, VarMap};  // Update if only VarBuilder imported

Replace Pattern

// ❌ BEFORE
let vs = VarBuilder::zeros(DType::F32, &device);

// ✅ AFTER
let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

---

Detailed Changes by File

1. ml/src/tft/gated_residual.rs (8 tests)

Line 225 - test_grn_creation

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 236 - test_grn_forward_same_dims

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 252 - test_grn_forward_different_dims

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 268 - test_grn_forward_with_context

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 287 - test_grn_forward_3d

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 303 - test_glu_creation

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 313 - test_glu_forward

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 328 - test_grn_stack

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

---

2. ml/src/tft/temporal_attention.rs (2 tests)

Line 382 - test_temporal_self_attention_creation

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 423 - test_interpretable_multi_head_attention

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

---

3. ml/src/tft/variable_selection.rs (5 tests)

Line 193 - test_variable_selection_network_creation

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 204 - test_variable_selection_forward

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 222 - test_attention_weights

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 240 - test_variable_selection_3d

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 261 - test_zero_attention_fallback

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

---

4. ml/src/tft/quantile_outputs.rs (6 tests)

Line 260 - test_quantile_output_creation

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 273 - test_quantile_output_forward

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 293 - test_quantile_output_monotonicity

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 311 - test_quantile_loss

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 329 - test_median_quantile

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

Line 361 - test_asymmetric_loss

let varmap = Arc::new(VarMap::new());
let vs = VarBuilder::from_varmap(&varmap, DType::F32, &device);

---

Automated Update Script

#!/bin/bash
# Script to update VarBuilder initialization in test files

FILES=(
    "ml/src/tft/gated_residual.rs"
    "ml/src/tft/temporal_attention.rs"
    "ml/src/tft/variable_selection.rs"
    "ml/src/tft/quantile_outputs.rs"
)

for file in "${FILES[@]}"; do
    echo "Updating $file..."

    # Add import if not present
    if ! grep -q "use std::sync::Arc;" "$file"; then
        sed -i '1i use std::sync::Arc;' "$file"
    fi

    # Replace VarBuilder::zeros pattern
    sed -i 's/let vs = VarBuilder::zeros(DType::F32, \&device);/let varmap = Arc::new(VarMap::new());\n    let vs = VarBuilder::from_varmap(\&varmap, DType::F32, \&device);/g' "$file"

    echo "✓ Updated $file"
done

echo ""
echo "All files updated. Run tests to verify:"
echo "cargo test -p ml --lib tft"

---

Verification After Updates

1. Run TFT Tests

cargo test -p ml --lib tft --no-fail-fast

Expected: All tests should pass with non-zero outputs

2. Run Weight Initialization Tests

cargo test --test test_grn_weight_initialization -p ml

Expected: All 9 tests should pass

3. Run Verification Example

cargo run --example verify_grn_weight_init -p ml

Expected Output:

✓ PASS: Weights are properly initialized (non-zero variance)
✓ PASS: Different inputs produce different outputs
✓ PASS: Context has measurable effect

---

Why This Matters

Problem with VarBuilder::zeros()

1. Creates all-zero weight matrices
2. Bypasses Xavier Uniform initialization
3. Tests only verify shape/dimension handling
4. Does NOT validate actual weight initialization behavior

Benefits of VarBuilder::from_varmap()

1. ✅ Proper Xavier Uniform initialization
2. ✅ Non-zero, normally distributed weights
3. ✅ Maintains gradient flow during training
4. ✅ Matches production code behavior

---

Impact Assessment

Test Behavior Before Fix

• ❌ All outputs are zero (or near-zero due to bias terms)
• ❌ Different inputs produce same outputs
• ❌ Context has no effect
• ✅ Shape/dimension tests pass (misleading success)

Test Behavior After Fix

• ✅ Outputs have non-zero variance
• ✅ Different inputs produce different outputs
• ✅ Context integration works correctly
• ✅ Tests validate actual initialization behavior

---

References

• Main Report: WAVE_8_6_GRN_WEIGHT_INITIALIZATION.md
• Quick Reference: WAVE_8_6_QUICK_REFERENCE.md
• Test Suite: ml/tests/test_grn_weight_initialization.rs
• Example: ml/examples/verify_grn_weight_init.rs

---

Status: Ready for implementation Priority: Medium (tests need updating, production code is correct) Estimated Time: 15 minutes (manual) or 5 minutes (automated script)