foxhunt/SECURITY_FIXES_DESIGN.md

Security Fixes Design - Agent 122

Date: 2025-10-14 Agent: Agent 122 Priority: HIGH (Critical security issues blocking production deployment)

---

Executive Summary

This document outlines the implementation design for three critical security vulnerabilities identified in Agent 108's security audit:

1. SEC-001: Missing checkpoint cryptographic signatures
2. SEC-002: No model poisoning detection
3. SEC-003: Insufficient prediction sanity checks

Implementation Timeline: 2 weeks Files Modified: ~8 files New Files Created: ~4 files Test Coverage: 100% for security features

---

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                   Security Layer Architecture                │
└─────────────────────────────────────────────────────────────┘

┌─────────────────┐      ┌──────────────────┐      ┌─────────────────┐
│  Checkpoint     │      │   Prediction     │      │    Ensemble     │
│  Signature      │──────│   Validator      │──────│    Anomaly      │
│  System         │      │                  │      │    Detector     │
└────────┬────────┘      └────────┬─────────┘      └────────┬────────┘
         │                        │                         │
         ▼                        ▼                         ▼
┌─────────────────┐      ┌──────────────────┐      ┌─────────────────┐
│  Vault          │      │  Statistical     │      │  Temporal       │
│  Key Storage    │      │  Bounds Check    │      │  Pattern        │
│  (HMAC keys)    │      │  (Z-score)       │      │  Analysis       │
└─────────────────┘      └──────────────────┘      └─────────────────┘
         │                        │                         │
         └────────────────────────┴─────────────────────────┘
                                  │
                                  ▼
                    ┌──────────────────────────┐
                    │   Security Audit Logger   │
                    │  (ml_security_events)     │
                    └──────────────────────────┘

---

1. Checkpoint Signature System (SEC-001)

Design Goals

• Authenticity: Verify checkpoint origin using HMAC-SHA256 signatures
• Integrity: Detect tampering via signature validation
• Key Rotation: Support quarterly key rotation with versioned keys
• Performance: <100μs signature overhead per checkpoint operation

Implementation Components

1.1 Extended CheckpointMetadata

pub struct CheckpointMetadata {
    // ... existing fields ...

    // Security fields (new)
    pub signature: Option<String>,           // HMAC-SHA256 signature (hex-encoded)
    pub signature_algorithm: String,         // "HMAC-SHA256"
    pub signing_key_id: String,              // Key rotation support (e.g., "2024-Q4")
    pub signed_at: Option<DateTime<Utc>>,    // Signature timestamp
}

1.2 CheckpointSigner

File: ml/src/checkpoint/signer.rs (NEW)

pub struct CheckpointSigner {
    vault_client: Arc<VaultClient>,
    key_cache: RwLock<HashMap<String, Vec<u8>>>,  // key_id -> secret_key
    cache_ttl: Duration,  // 5 minutes
}

impl CheckpointSigner {
    /// Sign checkpoint data with HMAC-SHA256
    pub async fn sign_checkpoint(
        &self,
        data: &[u8],
        model_type: ModelType,
    ) -> Result<SignatureInfo, MLError>;

    /// Verify checkpoint signature
    pub async fn verify_signature(
        &self,
        data: &[u8],
        signature: &str,
        key_id: &str,
    ) -> Result<(), MLError>;

    /// Rotate signing keys (quarterly)
    pub async fn rotate_keys(&self) -> Result<(), MLError>;
}

1.3 Vault Key Storage

Key Path Structure:

secret/foxhunt/ml/checkpoint-signing/
  ├── DQN/
  │   ├── 2024-Q4  (32-byte HMAC key)
  │   └── 2025-Q1  (next rotation)
  ├── PPO/
  ├── MAMBA-2/
  └── TFT/

Key Generation:

# Generate 256-bit HMAC key
openssl rand -hex 32 > /tmp/hmac_key.txt

# Store in Vault
vault kv put secret/foxhunt/ml/checkpoint-signing/DQN/2024-Q4 \
  key_value=@/tmp/hmac_key.txt

Integration Points

1. CheckpointManager::save_checkpoint(): Sign before saving
2. CheckpointManager::load_checkpoint(): Verify after loading
3. ValidationManager::validate_checkpoint(): Add signature validation step

Performance Impact

• Signing: ~50μs (HMAC-SHA256 computation)
• Verification: ~50μs
• Key cache hit: <1μs (no Vault call)
• Key cache miss: ~10ms (Vault HTTP request)

---

2. Prediction Validator (SEC-002)

Design Goals

• Outlier Detection: Identify predictions >3 standard deviations from mean
• Range Validation: Enforce [-1.0, 1.0] bounds
• Rate Limiting: Detect >10% extreme predictions as poisoning signal
• Performance: <10μs validation overhead per prediction

Implementation Components

2.1 PredictionValidator

File: ml/src/security/prediction_validator.rs (NEW)

pub struct PredictionValidator {
    // Historical statistics (rolling window)
    prediction_stats: RwLock<PredictionStatistics>,

    // Configuration
    z_score_threshold: f64,           // Default: 3.0 (99.7% of normal distribution)
    confidence_threshold: f64,         // Default: 0.5
    extreme_prediction_threshold: f64, // Default: 0.9
    max_extreme_rate: f64,             // Default: 0.1 (10%)

    // Temporal tracking
    extreme_prediction_window: VecDeque<Instant>,
    window_duration: Duration,         // Default: 60 seconds
}

pub struct PredictionStatistics {
    mean: f64,
    std_dev: f64,
    sample_count: u64,
    last_updated: Instant,
}

pub struct ValidatedPrediction {
    pub value: f64,
    pub is_outlier: bool,
    pub z_score: f64,
    pub should_override: bool,  // Use ensemble fallback
    pub validation_flags: Vec<ValidationFlag>,
}

pub enum ValidationFlag {
    OutOfBounds { value: f64 },
    Outlier { z_score: f64 },
    LowConfidence { confidence: f64 },
    ExtremeRateLimit { rate: f64 },
}

2.2 Validation Logic

impl PredictionValidator {
    /// Validate prediction with multi-layer checks
    pub fn validate(
        &self,
        prediction: f64,
        confidence: f64,
        model_id: &str,
    ) -> Result<ValidatedPrediction, MLError> {
        let mut flags = Vec::new();

        // Layer 1: Range check
        if prediction < -1.0 || prediction > 1.0 {
            flags.push(ValidationFlag::OutOfBounds { value: prediction });
            return Err(MLError::ValidationError { ... });
        }

        // Layer 2: Z-score outlier detection
        let stats = self.prediction_stats.read().unwrap();
        let z_score = (prediction - stats.mean) / stats.std_dev;

        if z_score.abs() > self.z_score_threshold {
            flags.push(ValidationFlag::Outlier { z_score });
            // Log security event
            self.log_security_event(SecurityEvent::OutlierDetected { ... });
        }

        // Layer 3: Confidence sanity check
        if confidence < self.confidence_threshold {
            flags.push(ValidationFlag::LowConfidence { confidence });
        }

        // Layer 4: Rate limit extreme predictions
        if prediction.abs() > self.extreme_prediction_threshold {
            let extreme_rate = self.calculate_extreme_rate();
            if extreme_rate > self.max_extreme_rate {
                flags.push(ValidationFlag::ExtremeRateLimit { rate: extreme_rate });
                return Err(MLError::ValidationError { ... });
            }
        }

        Ok(ValidatedPrediction {
            value: prediction,
            is_outlier: !flags.is_empty(),
            z_score,
            should_override: z_score.abs() > self.z_score_threshold,
            validation_flags: flags,
        })
    }

    /// Update statistics with new prediction (exponential moving average)
    pub fn update_statistics(&self, prediction: f64) {
        let mut stats = self.prediction_stats.write().unwrap();

        // Exponential moving average
        let alpha = 0.05; // Weight for new samples
        stats.mean = (1.0 - alpha) * stats.mean + alpha * prediction;

        // Update standard deviation (Welford's online algorithm)
        stats.sample_count += 1;
        // ... (standard deviation update logic)
    }
}

Integration Points

1. InferenceEngine::process_onnx_inference(): Validate before returning result
2. EnsembleCoordinator::aggregate_predictions(): Validate before aggregation
3. Database logging: Store is_outlier flag in ensemble_predictions.metadata

Statistical Initialization

Bootstrap Phase (first 1000 predictions):

• Use conservative defaults: mean=0.0, std_dev=0.3
• Gradually refine statistics as data accumulates
• After 1000 samples, switch to exponential moving average

---

3. Ensemble Anomaly Detector (SEC-003)

Design Goals

• Temporal Pattern Detection: Identify sudden signal shifts (>50%)
• Coordinated Attack Detection: Flag when >80% models predict extreme values
• Model Drift Detection: Track per-model behavioral changes
• Performance: <20μs overhead per ensemble decision

Implementation Components

3.1 EnsembleAnomalyDetector

File: ml/src/security/anomaly_detector.rs (NEW)

pub struct EnsembleAnomalyDetector {
    // Historical ensemble signal distribution
    signal_history: RwLock<VecDeque<f64>>,
    window_size: usize,  // Default: 100

    // Per-model tracking
    model_signal_history: RwLock<HashMap<String, VecDeque<f64>>>,
    model_window_size: usize,  // Default: 50

    // Anomaly thresholds
    sudden_shift_threshold: f64,        // Default: 0.5 (50% signal change)
    coordinated_attack_threshold: f64,  // Default: 0.8 (80% models agree on extreme)
    drift_threshold: f64,               // Default: 0.7
}

pub struct AnomalyReport {
    pub has_anomalies: bool,
    pub anomalies: Vec<Anomaly>,
    pub timestamp: DateTime<Utc>,
    pub severity: AnomalySeverity,
}

pub enum Anomaly {
    SuddenShift {
        previous: f64,
        current: f64,
        magnitude: f64,
    },
    CoordinatedAttack {
        extreme_ratio: f64,
        affected_models: Vec<String>,
    },
    ModelDrift {
        model_id: String,
        historical_mean: f64,
        current_signal: f64,
        drift_magnitude: f64,
    },
}

pub enum AnomalySeverity {
    Low,      // Single outlier
    Medium,   // Multiple outliers or moderate drift
    High,     // Coordinated attack or extreme shift
    Critical, // System-wide compromise suspected
}

3.2 Detection Logic

impl EnsembleAnomalyDetector {
    /// Detect anomalies in ensemble prediction
    pub fn detect_anomaly(
        &self,
        decision: &EnsembleDecision,
    ) -> AnomalyReport {
        let mut anomalies = Vec::new();

        // Detection 1: Sudden signal shift
        let signal_history = self.signal_history.read().unwrap();
        if let Some(prev_signal) = signal_history.back() {
            let shift = (decision.signal - prev_signal).abs();
            if shift > self.sudden_shift_threshold {
                anomalies.push(Anomaly::SuddenShift {
                    previous: *prev_signal,
                    current: decision.signal,
                    magnitude: shift,
                });
            }
        }

        // Detection 2: Coordinated attack (all models extreme)
        let extreme_count = decision.model_votes.values()
            .filter(|v| v.signal.abs() > 0.9)
            .count();
        let extreme_ratio = extreme_count as f64 / decision.model_votes.len() as f64;

        if extreme_ratio > self.coordinated_attack_threshold {
            anomalies.push(Anomaly::CoordinatedAttack {
                extreme_ratio,
                affected_models: decision.model_votes.keys().cloned().collect(),
            });
        }

        // Detection 3: Per-model behavioral drift
        let model_history = self.model_signal_history.read().unwrap();
        for (model_id, vote) in &decision.model_votes {
            if let Some(history) = model_history.get(model_id) {
                let mean = history.iter().sum::<f64>() / history.len() as f64;
                let drift = (vote.signal - mean).abs();

                if drift > self.drift_threshold {
                    anomalies.push(Anomaly::ModelDrift {
                        model_id: model_id.clone(),
                        historical_mean: mean,
                        current_signal: vote.signal,
                        drift_magnitude: drift,
                    });
                }
            }
        }

        // Determine severity
        let severity = self.calculate_severity(&anomalies);

        // Log to security audit system
        if severity >= AnomalySeverity::High {
            self.log_security_event(SecurityEvent::EnsembleAnomaly { ... });
        }

        AnomalyReport {
            has_anomalies: !anomalies.is_empty(),
            anomalies,
            timestamp: Utc::now(),
            severity,
        }
    }

    /// Update history with new decision
    pub fn update_history(&self, decision: &EnsembleDecision) {
        // Update ensemble signal history
        let mut signal_history = self.signal_history.write().unwrap();
        signal_history.push_back(decision.signal);
        if signal_history.len() > self.window_size {
            signal_history.pop_front();
        }

        // Update per-model history
        let mut model_history = self.model_signal_history.write().unwrap();
        for (model_id, vote) in &decision.model_votes {
            model_history
                .entry(model_id.clone())
                .or_insert_with(VecDeque::new)
                .push_back(vote.signal);

            let history = model_history.get_mut(model_id).unwrap();
            if history.len() > self.model_window_size {
                history.pop_front();
            }
        }
    }
}

Integration Points

1. EnsembleCoordinator::aggregate_predictions(): Detect anomalies after aggregation
2. Automated rollback: Trigger model rollback if severity is Critical
3. Monitoring: Export anomaly metrics to Prometheus

---

4. Security Event Logging

Database Schema

File: migrations/024_ml_security_events.sql (NEW)

-- ML security events table
CREATE TABLE ml_security_events (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    timestamp TIMESTAMPTZ NOT NULL DEFAULT NOW(),

    -- Event classification
    event_type VARCHAR(50) NOT NULL,  -- signature_failure, outlier_detected, etc.
    severity VARCHAR(20) NOT NULL,    -- low, medium, high, critical

    -- Context
    model_id VARCHAR(50),
    checkpoint_id VARCHAR(255),
    prediction_id UUID REFERENCES ensemble_predictions(id),

    -- Event details
    description TEXT NOT NULL,
    metadata JSONB,

    -- Response
    action_taken VARCHAR(100),  -- rejected, flagged, alerted, rollback

    CONSTRAINT chk_severity CHECK (severity IN ('low', 'medium', 'high', 'critical'))
);

-- Indexes for fast querying
CREATE INDEX idx_ml_security_events_timestamp ON ml_security_events (timestamp DESC);
CREATE INDEX idx_ml_security_events_severity ON ml_security_events (severity)
    WHERE severity IN ('high', 'critical');
CREATE INDEX idx_ml_security_events_type ON ml_security_events (event_type);
CREATE INDEX idx_ml_security_events_model ON ml_security_events (model_id)
    WHERE model_id IS NOT NULL;

-- TimescaleDB hypertable for time-series data
SELECT create_hypertable('ml_security_events', 'timestamp');

-- Retention policy: Keep high/critical events for 1 year, others for 90 days
SELECT add_retention_policy('ml_security_events', INTERVAL '90 days');

Event Types

pub enum SecurityEventType {
    // Checkpoint security
    CheckpointSignatureFailure,
    CheckpointSignatureMissing,
    CheckpointTamperingDetected,

    // Prediction validation
    PredictionOutlierDetected,
    PredictionOutOfBounds,
    ExtremeRateExceeded,

    // Ensemble anomalies
    EnsembleSuddenShift,
    CoordinatedAttackSuspected,
    ModelBehavioralDrift,

    // System events
    AutomaticRollback,
    ManualIntervention,
}

---

5. Testing Strategy

Unit Tests

File: ml/src/checkpoint/signer_tests.rs

• Test HMAC signature generation
• Test signature verification (valid/invalid)
• Test key rotation
• Test Vault integration (mocked)

File: ml/src/security/prediction_validator_tests.rs

• Test range validation
• Test Z-score outlier detection
• Test extreme rate limiting
• Test statistics update

File: ml/src/security/anomaly_detector_tests.rs

• Test sudden shift detection
• Test coordinated attack detection
• Test model drift detection
• Test severity calculation

Integration Tests

File: ml/tests/security_integration_test.rs

• End-to-end checkpoint signing workflow
• Prediction validation with database logging
• Ensemble anomaly detection with rollback
• Security event logging

Adversarial Tests

File: ml/tests/adversarial_tests.rs

• Tampered checkpoint rejection
• Poisoned model detection (synthetic adversarial predictions)
• Coordinated attack simulation
• Performance under attack

---

6. Performance Benchmarks

Target Performance

Operation	Target	Expected
Checkpoint signing	<100μs	~50μs
Checkpoint verification	<100μs	~50μs
Prediction validation	<10μs	~5μs
Anomaly detection	<20μs	~15μs
Security event logging	<500μs	~300μs (async)

Benchmark Suite

File: ml/benches/security_benchmarks.rs

#[bench]
fn bench_checkpoint_signing(b: &mut Bencher) { ... }

#[bench]
fn bench_prediction_validation(b: &mut Bencher) { ... }

#[bench]
fn bench_anomaly_detection(b: &mut Bencher) { ... }

---

7. Monitoring & Alerting

Prometheus Metrics

// Checkpoint security metrics
checkpoint_signature_failures_total
checkpoint_signature_verification_duration_seconds

// Prediction validation metrics
prediction_outliers_total
prediction_out_of_bounds_total
prediction_extreme_rate

// Ensemble anomaly metrics
ensemble_anomalies_total{severity="high|critical"}
ensemble_sudden_shifts_total
model_drift_events_total

Alerting Rules

File: monitoring/prometheus/alerts/ml_security_alerts.yml

groups:
  - name: ml_security
    interval: 30s
    rules:
      - alert: CheckpointSignatureFailure
        expr: rate(checkpoint_signature_failures_total[5m]) > 0
        for: 1m
        severity: critical
        annotations:
          summary: "Checkpoint signature verification failed"

      - alert: HighPredictionOutlierRate
        expr: prediction_extreme_rate > 0.1
        for: 5m
        severity: high
        annotations:
          summary: "More than 10% of predictions are outliers - possible model poisoning"

      - alert: EnsembleCoordinatedAttack
        expr: ensemble_anomalies_total{severity="critical"} > 0
        for: 1m
        severity: critical
        annotations:
          summary: "Coordinated attack detected - multiple models producing extreme predictions"

---

8. Implementation Timeline

Week 1: Core Implementation

Day 1-2: Checkpoint signature system

• Extend CheckpointMetadata
• Implement CheckpointSigner
• Vault integration for key management
• Unit tests

Day 3-4: Prediction validator

• Implement PredictionValidator
• Statistical bounds checking
• Rate limiting logic
• Unit tests

Day 5: Ensemble anomaly detector

• Implement EnsembleAnomalyDetector
• Temporal pattern detection
• Unit tests

Week 2: Integration & Testing

Day 6-7: Integration

• Integrate signer into CheckpointManager
• Integrate validator into InferenceEngine
• Integrate detector into EnsembleCoordinator

Day 8: Security event logging

• Database migration
• Logger implementation
• Async logging integration

Day 9: Testing

• Integration tests
• Adversarial tests
• E2E security tests

Day 10: Documentation & deployment

• Update security documentation
• Performance benchmarks
• Monitoring setup
• Final validation

---

9. Rollout Plan

Phase 1: Staging Deployment (Day 11-12)

1. Deploy to staging environment
2. Run 24-hour security validation
3. Monitor false positive rate
4. Tune thresholds

Phase 2: Production Deployment (Day 13-14)

1. Enable checkpoint signing (all new checkpoints)
2. Enable prediction validation (monitoring only)
3. Enable anomaly detection (alerting enabled)
4. Monitor for 7 days

Phase 3: Full Enforcement (Day 21)

1. Reject unsigned checkpoints
2. Reject invalid predictions
3. Automatic rollback on critical anomalies

---

10. Success Criteria

• ✅ All critical security issues (SEC-001, SEC-002, SEC-003) resolved
• ✅ 100% test coverage for security features
• ✅ Performance targets met (<100μs overhead)
• ✅ Zero false positives in 7-day production trial
• ✅ Monitoring and alerting operational
• ✅ Security event logging complete
• ✅ Documentation updated

---

11. Future Enhancements

Q1 2026

• Add Ed25519 public key signatures (stronger than HMAC)
• Implement checkpoint certificate chains
• Add hardware security module (HSM) support

Q2 2026

• Machine learning-based anomaly detection (LSTM autoencoder)
• Adversarial training for robust models
• Automatic retraining on poisoning detection

---

Design Status: ✅ Complete Next Step: Begin implementation (Week 1, Day 1) Owner: Agent 122 Reviewers: Security Team, ML Team Lead