fix(ml): Fix quantized attention mask handling

- Fixed matmul shape mismatch by removing unnecessary .t() transpose operations
- Fixed causal mask broadcasting to match scores shape [batch, num_heads, seq_len, seq_len]
- Refactored 3D matmul to use 2D reshape for compatibility with Candle
- Added test_attention_with_mask test to validate mask behavior
- Fixed weight projection logic in compute_projections_slow
- Added .contiguous() calls after transpose operations for memory layout
- Added test_attention_gradients test for STE gradient flow validation

Resolves device/shape mismatch errors in attention mask application.
This commit is contained in:
jgrusewski
2025-10-23 11:55:16 +02:00
parent 73b9ca0659
commit 257b794361

View File

@@ -9,7 +9,7 @@ use crate::memory_optimization::quantization::{
QuantizationConfig, QuantizationType, QuantizedTensor, Quantizer,
};
use crate::MLError;
use candle_core::{DType, Device, Tensor};
use candle_core::{Device, Tensor};
use candle_nn::VarBuilder;
use std::collections::HashMap;