fix(ml): Fix quantized attention mask handling
- Fixed matmul shape mismatch by removing unnecessary .t() transpose operations - Fixed causal mask broadcasting to match scores shape [batch, num_heads, seq_len, seq_len] - Refactored 3D matmul to use 2D reshape for compatibility with Candle - Added test_attention_with_mask test to validate mask behavior - Fixed weight projection logic in compute_projections_slow - Added .contiguous() calls after transpose operations for memory layout - Added test_attention_gradients test for STE gradient flow validation Resolves device/shape mismatch errors in attention mask application.
This commit is contained in:
@@ -9,7 +9,7 @@ use crate::memory_optimization::quantization::{
|
||||
QuantizationConfig, QuantizationType, QuantizedTensor, Quantizer,
|
||||
};
|
||||
use crate::MLError;
|
||||
use candle_core::{DType, Device, Tensor};
|
||||
use candle_core::{Device, Tensor};
|
||||
use candle_nn::VarBuilder;
|
||||
use std::collections::HashMap;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user