foxhunt/AGENT_FWD06_INTEGRATION_GUIDE.md

FWD-06: Integration Guide for INT8 TFT Forward Pass

Status: ✅ READY FOR INTEGRATION Date: 2025-10-21 Prerequisites: All FWD-01 through FWD-05 components complete

---

Quick Start

The complete INT8 forward pass implementation is ready in: /home/jgrusewski/Work/foxhunt/ml/src/tft/quantized_tft_forward.rs

To integrate into production:

1. Copy all methods from quantized_tft_forward.rs
2. Paste them inside the impl QuantizedTemporalFusionTransformer block in quantized_tft.rs (before the closing } at line 1075)
3. Replace the existing placeholder forward() method (lines 395-417)
4. Run tests: cargo test --test tft_int8_forward_integration_test

---

Step-by-Step Integration

Step 1: Backup Current Implementation

cp ml/src/tft/quantized_tft.rs ml/src/tft/quantized_tft.rs.backup

Step 2: Locate Integration Point

Open ml/src/tft/quantized_tft.rs and find:

• Line 55: impl QuantizedTemporalFusionTransformer {
• Line 1075: Closing } of impl block

Step 3: Remove Placeholder Methods

Delete or comment out these methods (if they exist):

// OLD - Lines ~395-417
pub fn forward(
    &self,
    static_features: &Tensor,
    _historical_features: &Tensor,
    _future_features: &Tensor,
) -> Result<Tensor, MLError> {
    let batch_size = static_features.dims()[0];
    let dummy = Tensor::zeros(
        &[batch_size, self.config.prediction_horizon, self.config.num_quantiles],
        DType::F32,
        &self.device,
    )?;
    Ok(dummy)
}

Step 4: Add New Methods

Copy all methods from quantized_tft_forward.rs and paste them inside the impl block:

Methods to add (in this order):

1. validate_inputs (~85 lines)

/// Validate input tensor dimensions and device placement
fn validate_inputs(
    &self,
    static_features: &Tensor,
    historical_features: &Tensor,
    future_features: &Tensor,
) -> Result<(), MLError> {
    // ... (see quantized_tft_forward.rs)
}

2. forward_static_vsn (~15 lines)

/// Step 1: Static Variable Selection Network (placeholder)
fn forward_static_vsn(&self, static_features: &Tensor) -> Result<Tensor, MLError> {
    // ... (see quantized_tft_forward.rs)
}

3. forward_historical_encoder (~40 lines)

/// Step 2: Historical Encoder (LSTM with INT8 weights)
fn forward_historical_encoder(&self, historical_features: &Tensor) -> Result<Tensor, MLError> {
    // ... (see quantized_tft_forward.rs)
}

4. forward_lstm_layer (~80 lines)

/// Forward pass through a single LSTM layer with quantized weights
fn forward_lstm_layer(
    &self,
    input: &Tensor,
    h_prev: &Tensor,
    c_prev: &Tensor,
    layer_weights: &HashMap<String, QuantizedTensor>,
) -> Result<(Tensor, Tensor, Tensor), MLError> {
    // ... (see quantized_tft_forward.rs)
}

5. forward_future_decoder (~10 lines)

/// Step 3: Future Decoder (simplified - uses same LSTM architecture)
fn forward_future_decoder(&self, future_features: &Tensor) -> Result<Tensor, MLError> {
    // ... (see quantized_tft_forward.rs)
}

6. combine_contexts (~30 lines)

/// Step 5: Combine encodings (static, attention output, future)
fn combine_contexts(
    &self,
    static_encoding: &Tensor,
    attention_output: &Tensor,
    future_encoding: &Tensor,
) -> Result<Tensor, MLError> {
    // ... (see quantized_tft_forward.rs)
}

7. forward_quantile_output (~15 lines)

/// Step 6: Quantile Output Layer (placeholder)
fn forward_quantile_output(&self, combined: &Tensor) -> Result<Tensor, MLError> {
    // ... (see quantized_tft_forward.rs)
}

8. forward_integrated (rename to forward) (~40 lines)

/// Complete end-to-end INT8 forward pass
pub fn forward(
    &self,
    static_features: &Tensor,
    historical_features: &Tensor,
    future_features: &Tensor,
) -> Result<Tensor, MLError> {
    // ... (see quantized_tft_forward.rs)
}

Step 5: Verify Compilation

cargo check -p ml --lib

Expected output:

Checking ml v1.0.0 (/home/jgrusewski/Work/foxhunt/ml)
Finished `dev` profile [unoptimized + debuginfo] target(s) in X.XXs

Step 6: Run Tests

cargo test --test tft_int8_forward_integration_test

Expected output:

running 5 tests
test test_quantized_tft_forward_pass_integration ... ok
test test_quantized_tft_input_validation ... ok
test test_quantized_tft_batch_consistency ... ok
test test_quantized_tft_device_consistency ... ok
test test_quantized_tft_memory_usage ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

---

File Structure After Integration

ml/src/tft/
├── quantized_tft.rs           # Main implementation (1075+ lines)
│   ├── struct QuantizedTemporalFusionTransformer
│   ├── impl QuantizedTemporalFusionTransformer
│   │   ├── new()
│   │   ├── new_with_device()
│   │   ├── initialize_attention_weights()
│   │   ├── forward_temporal_attention()  [EXISTING from FWD-04]
│   │   ├── create_causal_mask()
│   │   ├── initialize_grn()
│   │   ├── validate_inputs()             [NEW from FWD-06]
│   │   ├── forward_static_vsn()          [NEW from FWD-06]
│   │   ├── forward_historical_encoder()  [NEW from FWD-06]
│   │   ├── forward_lstm_layer()          [NEW from FWD-06]
│   │   ├── forward_future_decoder()      [NEW from FWD-06]
│   │   ├── combine_contexts()            [NEW from FWD-06]
│   │   ├── forward_quantile_output()     [NEW from FWD-06]
│   │   ├── forward()                     [REPLACED from FWD-06]
│   │   └── memory_usage_bytes()
│   └── [End impl block]
└── quantized_tft_forward.rs   # Reference implementation (can be deleted after integration)

---

Integration Checklist

Pre-Integration

• Backup current quantized_tft.rs
• Review quantized_tft_forward.rs implementation
• Confirm forward_temporal_attention() exists (from FWD-04)
• Confirm lstm_weights field exists in struct

During Integration

• Remove old placeholder forward() method
• Copy validate_inputs() method
• Copy forward_static_vsn() method
• Copy forward_historical_encoder() method
• Copy forward_lstm_layer() method
• Copy forward_future_decoder() method
• Copy combine_contexts() method
• Copy forward_quantile_output() method
• Copy forward_integrated() as forward()
• Verify all methods inside impl block
• Check no duplicate method names

Post-Integration

• Run cargo check -p ml --lib (should pass)
• Run cargo test --test tft_int8_forward_integration_test (5 tests should pass)
• Run cargo clippy -p ml --lib (should have no errors)
• Review compiler warnings (if any)
• Update documentation (if needed)
• Delete quantized_tft_forward.rs (optional)

---

Troubleshooting

Error: self parameter not allowed

Cause: Methods placed outside impl block

Fix: Ensure all methods are inside the impl QuantizedTemporalFusionTransformer { ... } block

Error: Duplicate method names

Cause: Old placeholder method not removed

Fix: Delete old forward() method before adding new one

Error: Missing field lstm_weights

Cause: Struct definition doesn't include required fields

Fix: Ensure struct has:

pub struct QuantizedTemporalFusionTransformer {
    pub config: TFTConfig,
    quantizer: Quantizer,
    device: Device,
    varmap: Arc<VarMap>,

    // Quantized attention weights (Q/K/V projections)
    q_weights: Option<QuantizedTensor>,
    k_weights: Option<QuantizedTensor>,
    v_weights: Option<QuantizedTensor>,
    o_weights: Option<QuantizedTensor>,

    // Quantized LSTM weights for historical encoder
    lstm_weights: Vec<HashMap<String, QuantizedTensor>>,  // REQUIRED

    // Quantized GRN for post-LSTM processing
    grn: Option<QuantizedGatedResidualNetwork>,
}

Test Failures

Test: test_quantized_tft_forward_pass_integration Failure: Shape mismatch Fix: Check forward_quantile_output() returns [batch, horizon, num_quantiles]

Test: test_quantized_tft_input_validation Failure: Not rejecting invalid inputs Fix: Ensure validate_inputs() has proper dimension checks

---

Performance Validation

After integration, run these benchmarks:

1. Latency Test

cargo bench --bench tft_int8_latency

Expected: <5ms per batch

2. Memory Test

cargo test test_quantized_tft_memory_usage -- --nocapture

Expected: 125MB (100-150MB range)

3. Accuracy Test (when FP32 baseline available)

cargo test test_quantized_vs_fp32_accuracy

Expected: <5% degradation

---

Next Steps After Integration

Immediate (FWD-07)

1. Run all integration tests
2. Benchmark latency (<5ms)
3. Measure memory usage (125MB)
4. Profile for bottlenecks

Short-term (FWD-08)

1. Replace VSN placeholder with actual QuantizedVariableSelectionNetwork
2. Replace quantile output placeholder with QuantizedGRN + linear layer
3. Add separate decoder weights (currently shared with encoder)
4. Implement checkpoint loading for INT8 weights

Long-term (Wave 12+)

1. Per-channel quantization (better accuracy)
2. Calibration-based quantization (vs symmetric)
3. INT4 quantization (further memory reduction)
4. CUDA kernel optimization for INT8 ops

---

References

• Implementation: /home/jgrusewski/Work/foxhunt/ml/src/tft/quantized_tft_forward.rs
• Tests: /home/jgrusewski/Work/foxhunt/ml/tests/tft_int8_forward_integration_test.rs
• Documentation: /home/jgrusewski/Work/foxhunt/AGENT_FWD06_INTEGRATION_COMPLETE.md
• Quick Summary: /home/jgrusewski/Work/foxhunt/AGENT_FWD06_QUICK_SUMMARY.md
• This Guide: /home/jgrusewski/Work/foxhunt/AGENT_FWD06_INTEGRATION_GUIDE.md

---

Summary

Integration Complexity: Medium (copy-paste with careful placement) Estimated Time: 15-30 minutes Risk Level: Low (extensive testing provided) Rollback Plan: Restore from backup file

Status: ✅ READY FOR INTEGRATION

Once integrated, the QuantizedTFT will have a complete end-to-end INT8 forward pass with:

• ✅ Input validation
• ✅ Static VSN (placeholder)
• ✅ Historical LSTM encoder (INT8)
• ✅ Future LSTM decoder (INT8)
• ✅ Temporal attention (INT8)
• ✅ Context combination
• ✅ Quantile output (placeholder)
• ✅ Error handling
• ✅ Device consistency
• ✅ 5 integration tests