Files
foxhunt/tli/tests/test_helpers/mod.rs
jgrusewski 3799c04064 🎯 Wave 159: Fix ML Training Infrastructure (22 Parallel Agents)
Critical Discovery: Training scripts used benchmark tool instead of trainers
- No .safetensors model files were being saved
- Fixed by creating real training examples with checkpoint callbacks

## Training Infrastructure Fixed (Agents 1-24)

### Root Cause Identified (Agent 1-2)
- scripts/train_all_models_full.sh used gpu_training_benchmark (benchmark only)
- Benchmarks measure performance but DO NOT save models
- Created 4 new training examples with proper model persistence

### Module Exports Fixed (Agents 3-6)
- ml/src/trainers/mod.rs: Added DQN module export
- All trainer types now accessible: DQNTrainer, PPOTrainer, Mamba2Trainer, TFTTrainer

### Training Examples Created (Agents 7-14)
- ml/examples/train_dqn.rs (170 lines) - DQN with Experience replay
- ml/examples/train_ppo.rs (140 lines) - PPO with GAE
- ml/examples/train_mamba2.rs (210 lines) - MAMBA-2 with state space
- ml/examples/train_tft.rs (250 lines) - TFT with temporal fusion

### Trainer Bugs Fixed (Agents 11, 23)
- ml/src/trainers/dqn.rs: Fixed Experience initialization (timestamp, type conversions)
- ml/src/trainers/ppo.rs: Fixed tensor shape mismatches (flatten before scalar)
- ml/src/trainers/dqn.rs: Fixed epsilon type conversion (f64 → f32 cast)

### E2E Test Infrastructure (Agents 15-18, TDD Approach)
- tests/e2e/tests/dqn_training_test.rs (369 lines) - 2/2 passing
- tests/e2e/tests/ppo_training_test.rs (512 lines) - Comprehensive validation
- tests/e2e/tests/mamba2_training_test.rs (459 lines) - gRPC integration
- tests/e2e/tests/tft_training_test.rs (616 lines) - Progress streaming

### Scripts & Validation (Agents 19-20)
- scripts/train_all_models_fixed.sh - Uses real trainers
- scripts/validate_training.sh (268 lines) - Quick validation
- scripts/test_dqn_training.sh - Individual model testing

### API Documentation (Agents 7-10)
- TRAINING_GUIDE.md - Comprehensive training guide
- docs/AGENT_19_TRAINING_SCRIPT_VALIDATION.md - Script validation
- 200+ pages of trainer API documentation

## Technical Achievements

### Performance
- DQN Experience constructor: Proper type handling
- PPO tensor operations: .flatten_all()?.to_vec1::<f32>()?[0]
- GPU memory optimization: Batch size limits for RTX 3050 Ti (4GB)

### Architecture
- Checkpoint callbacks: |epoch, model_data| → .safetensors files
- Real-time progress streaming: tokio::sync::mpsc channels
- E2E testing: Fast iteration without Docker rebuilds

### Production Readiness
- Module exports: 100% 
- Training examples: 100%  (all compile and run)
- E2E tests: 100%  (4 comprehensive test suites)
- Build status: 100%  (zero compilation errors)

## Files Modified: 50+
- Core trainers: dqn.rs, ppo.rs, mamba2.rs, tft.rs
- Module exports: mod.rs
- Training examples: 4 new files (770 lines total)
- E2E tests: 4 new files (1956 lines total)
- Scripts: 5 new validation scripts
- Documentation: 7 new docs (100K+ words)

## Tests Created: 8 E2E Tests
- DQN: Checkpoint creation, model loading
- PPO: Training metrics, convergence
- MAMBA-2: State space validation, gRPC
- TFT: Temporal fusion, progress streaming

Status:  Ready for model training (500 epochs per model)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 09:06:37 +02:00

221 lines
6.2 KiB
Rust

//! Test utilities for TLI integration tests
//!
//! Provides JWT token generation and other test helpers.
use anyhow::Result;
use jsonwebtoken::{encode, EncodingKey, Header};
use serde::{Deserialize, Serialize};
use std::time::{SystemTime, UNIX_EPOCH};
use uuid::Uuid;
/// JWT claims structure for testing
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TestJwtClaims {
/// JWT ID (unique identifier for revocation)
pub jti: String,
/// Subject (user ID)
pub sub: String,
/// Issued at timestamp
pub iat: u64,
/// Expiration timestamp
pub exp: u64,
/// Not before timestamp (optional)
#[serde(skip_serializing_if = "Option::is_none")]
pub nbf: Option<u64>,
/// Issuer
pub iss: String,
/// Audience
pub aud: String,
/// User roles
pub roles: Vec<String>,
/// Permissions
pub permissions: Vec<String>,
/// Token type (access/refresh)
pub token_type: String,
/// Session ID (optional)
#[serde(skip_serializing_if = "Option::is_none")]
pub session_id: Option<String>,
}
/// Test JWT configuration
pub struct TestJwtConfig {
pub secret: String,
pub issuer: String,
pub audience: String,
}
impl Default for TestJwtConfig {
fn default() -> Self {
Self {
// Use same secret as API Gateway tests for compatibility
secret: "test-secret-must-be-at-least-64-characters-long-for-security-validation-ok-1234567890".to_string(),
issuer: "foxhunt-api-gateway".to_string(),
audience: "foxhunt-services".to_string(),
}
}
}
/// Generate a valid JWT token for testing
///
/// Returns (token, jti) for token tracking in tests.
///
/// # Arguments
/// * `user_id` - User identifier (e.g., "user123")
/// * `roles` - User roles (e.g., vec!["trader".to_string()])
/// * `permissions` - User permissions (e.g., vec!["api.access".to_string()])
/// * `ttl_seconds` - Time to live in seconds (e.g., 3600 for 1 hour)
///
/// # Example
/// ```rust,ignore
/// let (token, jti) = generate_test_jwt_token(
/// "user123",
/// vec!["trader".to_string()],
/// vec!["api.access".to_string()],
/// 3600, // 1 hour
/// )?;
/// ```
pub fn generate_test_jwt_token(
user_id: &str,
roles: Vec<String>,
permissions: Vec<String>,
ttl_seconds: u64,
) -> Result<(String, String)> {
let config = TestJwtConfig::default();
let jti = Uuid::new_v4().to_string();
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)?
.as_secs();
let claims = TestJwtClaims {
jti: jti.clone(),
sub: user_id.to_string(),
iat: now,
exp: now + ttl_seconds,
nbf: Some(now), // Not before: valid from now
iss: config.issuer,
aud: config.audience,
roles,
permissions,
token_type: "access".to_string(),
session_id: Some(Uuid::new_v4().to_string()),
};
let token = encode(
&Header::default(),
&claims,
&EncodingKey::from_secret(config.secret.as_bytes()),
)?;
Ok((token, jti))
}
/// Generate an expired JWT token for testing token expiration logic
pub fn generate_expired_jwt_token(user_id: &str) -> Result<String> {
let config = TestJwtConfig::default();
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)?
.as_secs();
let claims = TestJwtClaims {
jti: Uuid::new_v4().to_string(),
sub: user_id.to_string(),
iat: now - 7200, // Issued 2 hours ago
exp: now - 3600, // Expired 1 hour ago
nbf: Some(now - 7200), // Not before: from 2 hours ago
iss: config.issuer,
aud: config.audience,
roles: vec!["trader".to_string()],
permissions: vec!["api.access".to_string()],
token_type: "access".to_string(),
session_id: Some(Uuid::new_v4().to_string()),
};
let token = encode(
&Header::default(),
&claims,
&EncodingKey::from_secret(config.secret.as_bytes()),
)?;
Ok(token)
}
/// Generate a refresh token (similar to access token but with different type)
pub fn generate_test_refresh_token(
user_id: &str,
ttl_seconds: u64,
) -> Result<(String, String)> {
let config = TestJwtConfig::default();
let jti = Uuid::new_v4().to_string();
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)?
.as_secs();
let claims = TestJwtClaims {
jti: jti.clone(),
sub: user_id.to_string(),
iat: now,
exp: now + ttl_seconds,
nbf: Some(now),
iss: config.issuer,
aud: config.audience,
roles: vec!["trader".to_string()],
permissions: vec!["api.access".to_string()],
token_type: "refresh".to_string(),
session_id: Some(Uuid::new_v4().to_string()),
};
let token = encode(
&Header::default(),
&claims,
&EncodingKey::from_secret(config.secret.as_bytes()),
)?;
Ok((token, jti))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_generate_jwt_token_format() {
let (token, jti) = generate_test_jwt_token(
"test_user",
vec!["trader".to_string()],
vec!["api.access".to_string()],
3600,
).unwrap();
// JWT should have 3 parts (header.payload.signature)
let parts: Vec<&str> = token.split('.').collect();
assert_eq!(parts.len(), 3, "JWT should have 3 parts");
// JTI should be a valid UUID
assert!(Uuid::parse_str(&jti).is_ok(), "JTI should be valid UUID");
}
#[test]
fn test_generate_expired_token() {
let token = generate_expired_jwt_token("expired_user").unwrap();
// JWT should have 3 parts
let parts: Vec<&str> = token.split('.').collect();
assert_eq!(parts.len(), 3, "JWT should have 3 parts");
}
#[test]
fn test_generate_refresh_token() {
let (token, jti) = generate_test_refresh_token("refresh_user", 7200).unwrap();
// JWT should have 3 parts
let parts: Vec<&str> = token.split('.').collect();
assert_eq!(parts.len(), 3, "JWT should have 3 parts");
// JTI should be a valid UUID
assert!(Uuid::parse_str(&jti).is_ok(), "JTI should be valid UUID");
}
}