foxhunt/docs/TLI_SECURITY_DOCUMENTATION.md

TLI Security Documentation

Foxhunt Trading System - Security Guide

Version: 1.0 Last Updated: 2025-01-23 Document Classification: Security Sensitive

---

Table of Contents

1. Security Overview
2. Authentication Setup
3. Certificate Management
4. Access Control (RBAC)
5. API Security
6. Network Security
7. Security Best Practices
8. Incident Response
9. Compliance Framework
10. Security Monitoring

---

Security Overview

The TLI system implements enterprise-grade security controls designed for financial trading environments, meeting regulatory requirements including:

• SOX (Sarbanes-Oxley): Financial reporting and audit trail requirements
• FINRA: Broker-dealer regulation compliance
• ISO 27001: Information security management standards
• PCI DSS: Payment card industry security (where applicable)

Security Architecture

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   TLI Client    │────│  mTLS Channel    │────│ Trading Service │
│  (Authenticated)│    │  (Certificate    │    │   (Secured)     │
│                 │    │   Validation)    │    │                 │
└─────────────────┘    └──────────────────┘    └─────────────────┘
         │                       │                       │
         │              ┌────────┴────────┐              │
         │              │  Auth Service   │              │
         └──────────────│ - Session Mgmt  │──────────────┘
                        │ - API Keys      │
                        │ - RBAC          │
                        │ - Audit Logs    │
                        └─────────────────┘

Security Layers

1. Transport Security: TLS 1.3 with mutual authentication
2. Authentication: Multi-factor authentication with session management
3. Authorization: Role-based access control (RBAC)
4. Network Security: Firewall, VPN, and network segmentation
5. Audit & Compliance: Comprehensive logging and monitoring
6. Data Protection: Encryption at rest and in transit

---

Authentication Setup

User Authentication

Password-Based Authentication

# Create user with secure password policy
cargo run --bin tli-admin -- create-user \
  --username "trader_001" \
  --email "trader@company.com" \
  --role "trader" \
  --force-password-change

# Set password policy
cargo run --bin tli-admin -- set-password-policy \
  --min-length 12 \
  --require-uppercase \
  --require-lowercase \
  --require-numbers \
  --require-symbols \
  --max-age-days 90 \
  --history-count 12

Multi-Factor Authentication (MFA)

# Enable MFA for user
cargo run --bin tli-admin -- enable-mfa \
  --username "trader_001" \
  --method "totp" \
  --backup-codes 10

# Verify MFA setup
cargo run --bin tli-client -- login \
  --username "trader_001" \
  --password "secure_password" \
  --mfa-code "123456"

API Key Management

Creating API Keys

use tli::auth::*;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let auth_service = AuthenticationService::new(security_config).await?;

    // Create API key with specific permissions
    let api_key = auth_service.create_api_key(
        "user_id_123",
        "Trading Algorithm v2.1",
        vec![
            "api:access".to_string(),
            "trade:execute".to_string(),
            "market_data:read".to_string(),
            "positions:read".to_string(),
        ],
        Some(90) // 90 days expiration
    ).await?;

    println!("API Key ID: {}", api_key.id);
    println!("API Key: {}", api_key.key);
    println!("Expires: {}", api_key.expires_at);

    Ok(())
}

API Key Rotation

# Automated rotation script
#!/bin/bash
set -e

USER_ID="trading_bot_001"
OLD_KEY_ID="api_key_123"
KEY_NAME="Trading Bot - Auto Rotated"

# Create new API key
NEW_KEY=$(cargo run --bin tli-admin -- create-api-key \
  --user-id "$USER_ID" \
  --name "$KEY_NAME" \
  --permissions "api:access,trade:execute,market_data:read" \
  --expires-days 90 \
  --output json)

# Extract new key details
NEW_KEY_ID=$(echo "$NEW_KEY" | jq -r '.id')
NEW_KEY_VALUE=$(echo "$NEW_KEY" | jq -r '.key')

# Update application configuration
echo "Updating application with new API key..."
kubectl create secret generic trading-bot-api-key \
  --from-literal=api-key="$NEW_KEY_VALUE" \
  --dry-run=client -o yaml | kubectl apply -f -

# Restart application pods
kubectl rollout restart deployment/trading-bot

# Wait for successful deployment
kubectl rollout status deployment/trading-bot

# Revoke old API key
cargo run --bin tli-admin -- revoke-api-key --key-id "$OLD_KEY_ID"

echo "API key rotation completed successfully"
echo "New API Key ID: $NEW_KEY_ID"

Session Management

Session Configuration

[security.session]
timeout_seconds = 3600          # 1 hour session timeout
max_sessions_per_user = 3       # Maximum concurrent sessions
token_length = 32               # Session token length
refresh_interval_seconds = 300  # 5 minute refresh requirement
secure_cookies = true           # Secure cookie settings
same_site = "Strict"           # CSRF protection

Session Monitoring

# Monitor active sessions
cargo run --bin tli-admin -- list-sessions \
  --active-only \
  --format table

# Force logout suspicious sessions
cargo run --bin tli-admin -- revoke-session \
  --session-id "session_123" \
  --reason "Security incident"

# Monitor session statistics
cargo run --bin tli-admin -- session-stats \
  --period "24h" \
  --group-by user

---

Certificate Management

TLS Certificate Setup

Generating Production Certificates

#!/bin/bash
# Generate CA certificate
openssl genrsa -out ca.key 4096
openssl req -new -x509 -days 3650 -key ca.key -out ca.crt \
  -subj "/C=US/ST=NY/L=NYC/O=Trading Company/OU=IT/CN=Foxhunt CA"

# Generate server certificate
openssl genrsa -out server.key 4096
openssl req -new -key server.key -out server.csr \
  -subj "/C=US/ST=NY/L=NYC/O=Trading Company/OU=Trading/CN=trading.company.com"

# Create server certificate extensions
cat > server.ext << EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = @alt_names

[alt_names]
DNS.1 = trading.company.com
DNS.2 = localhost
IP.1 = 127.0.0.1
IP.2 = 10.0.0.100
EOF

# Sign server certificate
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key \
  -CAcreateserial -out server.crt -days 365 -extensions v3_ext -extfile server.ext

# Generate client certificate for mTLS
openssl genrsa -out client.key 4096
openssl req -new -key client.key -out client.csr \
  -subj "/C=US/ST=NY/L=NYC/O=Trading Company/OU=Clients/CN=trading-client"

openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key \
  -CAcreateserial -out client.crt -days 365

# Set secure permissions
chmod 600 *.key
chmod 644 *.crt
chown foxhunt:foxhunt *.key *.crt

# Install certificates
mkdir -p /etc/foxhunt/tls
cp ca.crt server.crt server.key client.crt client.key /etc/foxhunt/tls/

Certificate Validation

# Verify certificate chain
openssl verify -CAfile /etc/foxhunt/tls/ca.crt /etc/foxhunt/tls/server.crt

# Check certificate expiration
openssl x509 -in /etc/foxhunt/tls/server.crt -text -noout | grep "Not After"

# Test TLS connection
openssl s_client -connect localhost:50051 -CAfile /etc/foxhunt/tls/ca.crt \
  -cert /etc/foxhunt/tls/client.crt -key /etc/foxhunt/tls/client.key

Automated Certificate Renewal

#!/bin/bash
# Certificate renewal script
set -e

CERT_DIR="/etc/foxhunt/tls"
BACKUP_DIR="/etc/foxhunt/tls/backup/$(date +%Y%m%d)"
DAYS_BEFORE_EXPIRY=30

# Check certificate expiration
EXPIRY_DATE=$(openssl x509 -in "$CERT_DIR/server.crt" -noout -enddate | cut -d= -f2)
EXPIRY_EPOCH=$(date -d "$EXPIRY_DATE" +%s)
CURRENT_EPOCH=$(date +%s)
DAYS_UNTIL_EXPIRY=$(( (EXPIRY_EPOCH - CURRENT_EPOCH) / 86400 ))

if [ $DAYS_UNTIL_EXPIRY -gt $DAYS_BEFORE_EXPIRY ]; then
    echo "Certificate valid for $DAYS_UNTIL_EXPIRY days, renewal not needed"
    exit 0
fi

echo "Certificate expires in $DAYS_UNTIL_EXPIRY days, renewing..."

# Backup existing certificates
mkdir -p "$BACKUP_DIR"
cp "$CERT_DIR"/*.crt "$CERT_DIR"/*.key "$BACKUP_DIR/"

# Generate new certificates (reuse CA)
# ... (certificate generation commands) ...

# Test new certificates
if openssl verify -CAfile "$CERT_DIR/ca.crt" "$CERT_DIR/server.crt"; then
    echo "New certificate verified successfully"

    # Restart services with new certificates
    systemctl reload foxhunt-trading-service
    systemctl reload foxhunt-backtesting-service

    echo "Certificate renewal completed successfully"
else
    echo "Certificate verification failed, rolling back"
    cp "$BACKUP_DIR"/* "$CERT_DIR/"
    exit 1
fi

Certificate Monitoring

# Monitor certificate expiration
cargo run --bin tli-admin -- cert-status \
  --warn-days 30 \
  --critical-days 7

# Certificate health check
cargo run --bin tli-admin -- health-check \
  --component certificates \
  --format json

---

Access Control (RBAC)

Role Definitions

Predefined Roles

# /etc/foxhunt/rbac/roles.yaml
roles:
  admin:
    description: "System administrator with full access"
    permissions:
      - "system:*"
      - "user:*"
      - "audit:*"
      - "config:*"

  trader:
    description: "Trader with execution permissions"
    permissions:
      - "trade:execute"
      - "order:place"
      - "order:cancel"
      - "positions:read"
      - "market_data:read"
      - "risk:read"

  analyst:
    description: "Research analyst with read-only access"
    permissions:
      - "market_data:read"
      - "positions:read"
      - "risk:read"
      - "backtest:run"
      - "backtest:read"

  risk_manager:
    description: "Risk management specialist"
    permissions:
      - "risk:*"
      - "positions:read"
      - "order:cancel"
      - "emergency:stop"
      - "limits:modify"

  auditor:
    description: "Compliance and audit access"
    permissions:
      - "audit:read"
      - "logs:read"
      - "compliance:read"
      - "reports:generate"

  api_service:
    description: "Service account for API access"
    permissions:
      - "api:access"
      - "trade:execute"
      - "market_data:read"
      - "positions:read"

Custom Permissions

# Create custom permission
cargo run --bin tli-admin -- create-permission \
  --name "algo:deploy" \
  --description "Deploy algorithmic trading strategies" \
  --resource "algorithm" \
  --action "deploy"

# Assign permission to role
cargo run --bin tli-admin -- add-permission-to-role \
  --role "senior_trader" \
  --permission "algo:deploy"

# Create user with custom role
cargo run --bin tli-admin -- create-user \
  --username "algo_trader" \
  --role "senior_trader" \
  --department "quantitative_trading"

Permission Management

Granting Permissions

use tli::auth::*;

// Grant temporary elevated permissions
let rbac_manager = RbacManager::new(rbac_config).await?;

rbac_manager.grant_temporary_permission(
    "user_123",
    "emergency:override",
    chrono::Duration::hours(1)
).await?;

// Revoke permissions
rbac_manager.revoke_permission("user_123", "emergency:override").await?;

Permission Auditing

# Audit user permissions
cargo run --bin tli-admin -- audit-permissions \
  --user "trader_001" \
  --output detailed

# Check effective permissions
cargo run --bin tli-admin -- effective-permissions \
  --user "trader_001" \
  --resource "order" \
  --action "place"

# Permission usage report
cargo run --bin tli-admin -- permission-usage-report \
  --period "30d" \
  --format csv

---

API Security

Rate Limiting

Rate Limit Configuration

[security.rate_limiting]
# Per-user limits
authenticated_rpm = 1000        # Requests per minute for authenticated users
api_key_rpm = 5000             # Requests per minute for API keys

# Trading-specific limits
trading_burst = 100            # Burst allowance for trading operations
order_rpm = 300               # Orders per minute limit
cancel_rpm = 500              # Cancellations per minute limit

# Global limits
global_rpm = 50000            # Global system limit
window_seconds = 60           # Rate limiting window

# Security limits
failed_auth_limit = 5         # Failed authentication attempts
lockout_duration_minutes = 15 # Account lockout duration

Rate Limiting Implementation

use tli::auth::rate_limiter::*;

// Custom rate limiter for trading operations
let trading_limiter = RateLimiter::new(RateLimitConfig {
    requests_per_window: 100,
    window_duration: Duration::from_secs(60),
    burst_size: 10,
    rate_limit_type: RateLimitType::SlidingWindow,
})?;

// Check rate limit before processing
match trading_limiter.check_limit(&client_id).await {
    Ok(_) => {
        // Process trading request
        process_trade_request(request).await?;
    }
    Err(RateLimitError::LimitExceeded { limit, window, .. }) => {
        return Err(TradingError::RateLimitExceeded {
            limit,
            window,
            retry_after: calculate_retry_after(),
        });
    }
}

Input Validation

Request Validation

use validator::{Validate, ValidationError};
use serde::{Deserialize, Serialize};

#[derive(Debug, Deserialize, Validate)]
pub struct OrderRequest {
    #[validate(length(min = 1, max = 20, message = "Symbol must be 1-20 characters"))]
    #[validate(regex = "SYMBOL_REGEX", message = "Invalid symbol format")]
    pub symbol: String,

    #[validate(range(min = 0.01, max = 1000000.0, message = "Quantity must be between 0.01 and 1,000,000"))]
    pub quantity: f64,

    #[validate(range(min = 0.01, message = "Price must be positive"))]
    pub price: Option<f64>,

    #[validate(custom = "validate_time_in_force")]
    pub time_in_force: String,
}

fn validate_time_in_force(tif: &str) -> Result<(), ValidationError> {
    match tif {
        "DAY" | "GTC" | "IOC" | "FOK" => Ok(()),
        _ => Err(ValidationError::new("Invalid time in force")),
    }
}

// Middleware for automatic validation
pub async fn validate_request<T: Validate>(request: T) -> Result<T, ValidationError> {
    request.validate()?;
    Ok(request)
}

SQL Injection Prevention

// Use parameterized queries
use sqlx::{PgPool, query};

pub async fn get_user_orders(
    pool: &PgPool,
    user_id: &str,
    symbol: Option<&str>
) -> Result<Vec<Order>, sqlx::Error> {
    let query = match symbol {
        Some(sym) => {
            query!(
                "SELECT * FROM orders WHERE user_id = $1 AND symbol = $2 ORDER BY created_at DESC",
                user_id,
                sym
            )
        }
        None => {
            query!(
                "SELECT * FROM orders WHERE user_id = $1 ORDER BY created_at DESC",
                user_id
            )
        }
    };

    // Execute query safely with parameters
    query.fetch_all(pool).await
}

---

Network Security

Firewall Configuration

iptables Rules

#!/bin/bash
# Firewall configuration for TLI services

# Clear existing rules
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X

# Default policies
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT

# Allow loopback
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

# Allow established connections
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# SSH access (from management network only)
iptables -A INPUT -p tcp --dport 22 -s 10.0.1.0/24 -j ACCEPT

# TLI services (from trusted networks only)
iptables -A INPUT -p tcp --dport 50051 -s 10.0.0.0/16 -j ACCEPT
iptables -A INPUT -p tcp --dport 50052 -s 10.0.0.0/16 -j ACCEPT

# Health check endpoint (internal monitoring)
iptables -A INPUT -p tcp --dport 8080 -s 10.0.0.100 -j ACCEPT

# Database access (from application servers only)
iptables -A INPUT -p tcp --dport 5432 -s 10.0.0.50 -j ACCEPT
iptables -A INPUT -p tcp --dport 5432 -s 10.0.0.51 -j ACCEPT

# Log dropped packets
iptables -A INPUT -j LOG --log-prefix "DROPPED: " --log-level 4
iptables -A INPUT -j DROP

# Save rules
iptables-save > /etc/iptables/rules.v4

Network Segmentation

# Network segmentation design
networks:
  management:
    cidr: "10.0.1.0/24"
    purpose: "Administrative access"
    access: "SSH, monitoring"

  trading:
    cidr: "10.0.0.0/24"
    purpose: "Trading applications"
    access: "TLI services, databases"

  market_data:
    cidr: "10.0.2.0/24"
    purpose: "Market data feeds"
    access: "Market data providers"

  dmz:
    cidr: "192.168.100.0/24"
    purpose: "External facing services"
    access: "Limited internet access"

VPN Configuration

WireGuard Setup

# /etc/wireguard/wg0.conf
[Interface]
PrivateKey = <server_private_key>
Address = 10.0.10.1/24
ListenPort = 51820
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

# Trading team access
[Peer]
PublicKey = <trader_public_key>
AllowedIPs = 10.0.10.10/32
PersistentKeepalive = 25

# Risk management access
[Peer]
PublicKey = <risk_manager_public_key>
AllowedIPs = 10.0.10.20/32
PersistentKeepalive = 25

---

Security Best Practices

Secure Coding Practices

Input Sanitization

use regex::Regex;
use lazy_static::lazy_static;

lazy_static! {
    static ref SYMBOL_REGEX: Regex = Regex::new(r"^[A-Z]{1,10}$").unwrap();
    static ref ORDER_ID_REGEX: Regex = Regex::new(r"^[A-Za-z0-9\-_]{1,50}$").unwrap();
}

pub fn validate_symbol(symbol: &str) -> Result<String, ValidationError> {
    if !SYMBOL_REGEX.is_match(symbol) {
        return Err(ValidationError::new("Invalid symbol format"));
    }

    // Additional validation
    if symbol.len() > 10 {
        return Err(ValidationError::new("Symbol too long"));
    }

    Ok(symbol.to_uppercase())
}

pub fn sanitize_order_id(order_id: &str) -> Result<String, ValidationError> {
    if !ORDER_ID_REGEX.is_match(order_id) {
        return Err(ValidationError::new("Invalid order ID format"));
    }

    // Remove any potentially dangerous characters
    let sanitized = order_id.chars()
        .filter(|c| c.is_alphanumeric() || *c == '-' || *c == '_')
        .collect::<String>();

    Ok(sanitized)
}

Secure Error Handling

use tracing::{error, warn};

pub enum TradingError {
    InvalidCredentials,
    InsufficientFunds { required: f64, available: f64 },
    OrderNotFound { order_id: String },
    SystemError { correlation_id: String },
}

impl TradingError {
    pub fn to_client_error(&self) -> ClientError {
        match self {
            TradingError::InvalidCredentials => {
                // Log security event but don't expose details
                warn!("Authentication attempt failed");
                ClientError::Unauthenticated("Authentication required".to_string())
            }
            TradingError::InsufficientFunds { required, available } => {
                ClientError::BusinessLogic(format!(
                    "Insufficient funds: required ${:.2}, available ${:.2}",
                    required, available
                ))
            }
            TradingError::OrderNotFound { order_id } => {
                // Log potential enumeration attempt
                warn!("Order lookup for non-existent order: {}", order_id);
                ClientError::NotFound("Order not found".to_string())
            }
            TradingError::SystemError { correlation_id } => {
                // Log detailed error internally, return generic message
                error!("System error occurred: {}", correlation_id);
                ClientError::Internal(format!("Internal error (ref: {})", correlation_id))
            }
        }
    }
}

Environment Hardening

Service Configuration

# Secure service configuration
[security]
# Disable unnecessary features
enable_debug_endpoints = false
enable_metrics_export = false  # Only enable in monitoring environment
enable_pprof = false

# Restrict file access
config_file_permissions = "0600"
log_file_permissions = "0640"
data_directory = "/var/lib/foxhunt"

# Process isolation
run_as_user = "foxhunt"
run_as_group = "foxhunt"
enable_seccomp = true
enable_apparmor = true

# Resource limits
max_memory_mb = 4096
max_file_descriptors = 8192
max_cpu_percent = 80

Docker Security

# Multi-stage build for security
FROM rust:1.75-slim as builder
WORKDIR /app
COPY . .
RUN cargo build --release

FROM debian:bullseye-slim
# Create non-root user
RUN groupadd -r foxhunt && useradd -r -g foxhunt foxhunt

# Install security updates only
RUN apt-get update && apt-get install -y --no-install-recommends \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# Copy binary and set permissions
COPY --from=builder /app/target/release/trading-service /usr/local/bin/
RUN chmod +x /usr/local/bin/trading-service

# Create necessary directories
RUN mkdir -p /var/lib/foxhunt /var/log/foxhunt && \
    chown -R foxhunt:foxhunt /var/lib/foxhunt /var/log/foxhunt

# Switch to non-root user
USER foxhunt

# Security labels
LABEL security.scan="enabled"
LABEL security.compliance="SOX,FINRA"

EXPOSE 50051
CMD ["/usr/local/bin/trading-service"]

Secret Management

HashiCorp Vault Integration

use vault::VaultClient;
use serde_json::Value;

pub struct SecretManager {
    vault_client: VaultClient,
    mount_path: String,
}

impl SecretManager {
    pub async fn new(vault_url: &str, token: &str) -> Result<Self, VaultError> {
        let client = VaultClient::new(vault_url, token)?;

        Ok(Self {
            vault_client: client,
            mount_path: "foxhunt-secrets".to_string(),
        })
    }

    pub async fn get_database_credentials(&self) -> Result<DatabaseCredentials, VaultError> {
        let secret_path = format!("{}/database/primary", self.mount_path);
        let secret = self.vault_client.read_secret(&secret_path).await?;

        Ok(DatabaseCredentials {
            username: secret["username"].as_str().unwrap().to_string(),
            password: secret["password"].as_str().unwrap().to_string(),
            host: secret["host"].as_str().unwrap().to_string(),
            port: secret["port"].as_u64().unwrap() as u16,
        })
    }

    pub async fn get_api_encryption_key(&self) -> Result<Vec<u8>, VaultError> {
        let secret_path = format!("{}/encryption/api-keys", self.mount_path);
        let secret = self.vault_client.read_secret(&secret_path).await?;

        let key_b64 = secret["key"].as_str().unwrap();
        base64::decode(key_b64).map_err(|e| VaultError::DecodingError(e.to_string()))
    }
}

---

Incident Response

Security Incident Procedures

Incident Classification

Level	Description	Response Time	Escalation
P1	Active breach, trading impact	5 minutes	CTO, Legal, PR
P2	Security vulnerability, no breach	30 minutes	Security Team
P3	Policy violation, minor impact	2 hours	Team Lead
P4	Information gathering, no impact	24 hours	Security Team

Incident Response Playbook

#!/bin/bash
# Incident response automation script
set -e

INCIDENT_ID="INC-$(date +%Y%m%d-%H%M%S)"
INCIDENT_LEVEL="$1"
INCIDENT_TYPE="$2"
DESCRIPTION="$3"

echo "Starting incident response for $INCIDENT_ID"
echo "Level: $INCIDENT_LEVEL"
echo "Type: $INCIDENT_TYPE"

# Immediate containment actions
case "$INCIDENT_LEVEL" in
    "P1")
        echo "P1 Incident - Executing immediate containment"

        # Emergency stop all trading
        cargo run --bin tli-admin -- emergency-stop \
            --type full_shutdown \
            --reason "Security incident $INCIDENT_ID" \
            --confirm

        # Revoke all active sessions
        cargo run --bin tli-admin -- revoke-all-sessions \
            --reason "Security incident"

        # Block all external access
        iptables -A INPUT -s 0.0.0.0/0 -j DROP

        # Alert incident response team
        curl -X POST "$SLACK_WEBHOOK" \
            -H 'Content-type: application/json' \
            --data "{\"text\":\"🚨 P1 Security Incident: $INCIDENT_ID - $DESCRIPTION\"}"
        ;;

    "P2")
        echo "P2 Incident - Standard containment"

        # Increase monitoring sensitivity
        cargo run --bin tli-admin -- set-alert-level --level high

        # Capture system state
        /opt/foxhunt/scripts/capture-system-state.sh "$INCIDENT_ID"
        ;;
esac

# Evidence collection
mkdir -p "/var/log/incidents/$INCIDENT_ID"
cp /var/log/foxhunt/audit.log "/var/log/incidents/$INCIDENT_ID/"
cp /var/log/foxhunt/security.log "/var/log/incidents/$INCIDENT_ID/"

# Generate incident report
cargo run --bin tli-admin -- generate-incident-report \
    --incident-id "$INCIDENT_ID" \
    --level "$INCIDENT_LEVEL" \
    --type "$INCIDENT_TYPE" \
    --output "/var/log/incidents/$INCIDENT_ID/report.json"

echo "Incident response initiated for $INCIDENT_ID"

Evidence Collection

# Automated evidence collection script
#!/bin/bash
INCIDENT_ID="$1"
EVIDENCE_DIR="/secure/incidents/$INCIDENT_ID"

mkdir -p "$EVIDENCE_DIR"
cd "$EVIDENCE_DIR"

# System state
ps aux > processes.txt
netstat -tulnp > network_connections.txt
lsof > open_files.txt
df -h > disk_usage.txt
free -h > memory_usage.txt

# Network traffic capture
tcpdump -i any -w network_capture.pcap -c 10000 &
TCPDUMP_PID=$!

# Application logs
cp /var/log/foxhunt/*.log ./
cp /var/log/nginx/access.log ./
cp /var/log/postgresql/postgresql.log ./

# Configuration files
tar -czf configs.tar.gz /etc/foxhunt/

# Database snapshot
pg_dump foxhunt_db > database_snapshot.sql

# Stop network capture
sleep 60
kill $TCPDUMP_PID

# Create evidence manifest
sha256sum * > evidence_manifest.txt

# Encrypt evidence
tar -czf - * | gpg --cipher-algo AES256 --compress-algo 1 \
    --symmetric --output "evidence-$INCIDENT_ID.tar.gz.gpg"

echo "Evidence collection completed for incident $INCIDENT_ID"

Recovery Procedures

System Recovery

#!/bin/bash
# System recovery after security incident
set -e

INCIDENT_ID="$1"
RECOVERY_TYPE="$2"  # full, partial, config-only

echo "Starting recovery for incident $INCIDENT_ID"

case "$RECOVERY_TYPE" in
    "full")
        echo "Full system recovery"

        # Stop all services
        systemctl stop foxhunt-*

        # Restore from clean backup
        systemctl stop postgresql
        pg_restore -d foxhunt_db /backup/clean/database.sql
        systemctl start postgresql

        # Restore configuration from vault
        /opt/foxhunt/scripts/restore-config-from-vault.sh

        # Regenerate all certificates
        /opt/foxhunt/scripts/generate-certificates.sh --force

        # Reset all passwords and API keys
        cargo run --bin tli-admin -- reset-all-credentials

        # Start services
        systemctl start foxhunt-trading-service
        systemctl start foxhunt-backtesting-service
        ;;

    "partial")
        echo "Partial recovery - configuration and credentials only"

        # Revoke compromised credentials
        cargo run --bin tli-admin -- revoke-compromised-credentials \
            --incident-id "$INCIDENT_ID"

        # Rotate API keys
        /opt/foxhunt/scripts/rotate-all-api-keys.sh

        # Update firewall rules
        /opt/foxhunt/scripts/update-firewall-rules.sh --strict
        ;;
esac

# Verify system integrity
cargo run --bin tli-admin -- verify-system-integrity

# Test all critical functions
cargo run --bin tli-admin -- run-health-checks --comprehensive

echo "Recovery completed for incident $INCIDENT_ID"

---

Compliance Framework

Regulatory Requirements

SOX Compliance

# SOX compliance configuration
sox_compliance:
  financial_reporting:
    - audit_trail_retention: "7_years"
    - transaction_logging: "all_trades"
    - access_controls: "segregation_of_duties"
    - change_management: "approval_required"

  internal_controls:
    - user_access_reviews: "quarterly"
    - privilege_escalation_monitoring: "real_time"
    - configuration_change_approval: "required"
    - backup_verification: "monthly"

  documentation:
    - control_descriptions: "detailed"
    - risk_assessments: "annual"
    - testing_procedures: "documented"
    - remediation_tracking: "automated"

FINRA Compliance

# FINRA compliance monitoring
#!/bin/bash

# Trade reporting validation
cargo run --bin compliance-validator -- trade-reporting \
    --date "$(date +%Y-%m-%d)" \
    --format FINRA_CAT

# Order audit trail
cargo run --bin compliance-validator -- order-audit-trail \
    --include-cancellations \
    --include-modifications \
    --output /compliance/reports/oats-$(date +%Y%m%d).xml

# Supervision and surveillance
cargo run --bin compliance-validator -- surveillance-report \
    --type "unusual_activity" \
    --threshold-config /etc/foxhunt/surveillance-thresholds.yaml

Audit Trail Management

Comprehensive Logging

use tracing::{info, warn, error};
use serde_json::json;

#[derive(Debug, Serialize)]
pub struct AuditEvent {
    pub event_id: String,
    pub timestamp: DateTime<Utc>,
    pub user_id: Option<String>,
    pub session_id: Option<String>,
    pub client_ip: String,
    pub event_type: AuditEventType,
    pub resource: String,
    pub action: String,
    pub outcome: AuditOutcome,
    pub details: serde_json::Value,
}

impl AuditEvent {
    pub fn trading_operation(
        user_id: &str,
        client_ip: &str,
        operation: &str,
        symbol: &str,
        quantity: f64,
        outcome: AuditOutcome,
    ) -> Self {
        Self {
            event_id: Uuid::new_v4().to_string(),
            timestamp: Utc::now(),
            user_id: Some(user_id.to_string()),
            session_id: None, // Set by middleware
            client_ip: client_ip.to_string(),
            event_type: AuditEventType::Trading,
            resource: "order".to_string(),
            action: operation.to_string(),
            outcome,
            details: json!({
                "symbol": symbol,
                "quantity": quantity,
                "timestamp_nanos": Utc::now().timestamp_nanos(),
            }),
        }
    }
}

// Audit middleware for all gRPC requests
pub async fn audit_middleware<T>(
    request: Request<T>,
    handler: impl Future<Output = Result<Response<U>, Status>>,
) -> Result<Response<U>, Status> {
    let start_time = Instant::now();
    let user_id = extract_user_id(&request)?;
    let client_ip = extract_client_ip(&request)?;

    let result = handler.await;
    let duration = start_time.elapsed();

    let outcome = match &result {
        Ok(_) => AuditOutcome::Success,
        Err(status) => AuditOutcome::Failure {
            error_code: status.code().to_string(),
            error_message: status.message().to_string(),
        },
    };

    // Log audit event
    let audit_event = AuditEvent {
        event_id: Uuid::new_v4().to_string(),
        timestamp: Utc::now(),
        user_id: Some(user_id),
        client_ip,
        event_type: AuditEventType::Api,
        resource: extract_resource_from_request(&request),
        action: extract_action_from_request(&request),
        outcome,
        details: json!({
            "duration_ms": duration.as_millis(),
            "request_size": request.get_ref().len(),
        }),
    };

    AUDIT_LOGGER.log_event(audit_event).await?;

    result
}

Audit Log Retention

#!/bin/bash
# Audit log retention and archival script

RETENTION_YEARS=7
AUDIT_LOG_DIR="/var/log/foxhunt/audit"
ARCHIVE_DIR="/archive/audit"

# Find logs older than retention period
find "$AUDIT_LOG_DIR" -name "*.log" -mtime +$((RETENTION_YEARS * 365)) -print0 | \
while IFS= read -r -d '' log_file; do
    echo "Archiving old audit log: $log_file"

    # Compress and encrypt
    gzip "$log_file"
    gpg --cipher-algo AES256 --compress-algo 1 --symmetric \
        --output "${log_file}.gz.gpg" "${log_file}.gz"

    # Move to archive
    mkdir -p "$ARCHIVE_DIR/$(date -r "$log_file" +%Y/%m)"
    mv "${log_file}.gz.gpg" "$ARCHIVE_DIR/$(date -r "$log_file" +%Y/%m)/"

    # Remove original
    rm -f "${log_file}.gz"
done

# Verify archive integrity
find "$ARCHIVE_DIR" -name "*.gpg" -exec gpg --list-packets {} \; > /dev/null

---

Security Monitoring

Real-time Monitoring

Security Event Detection

use tokio::sync::mpsc;
use tracing::{info, warn, error};

pub struct SecurityMonitor {
    alert_sender: mpsc::Sender<SecurityAlert>,
    thresholds: SecurityThresholds,
    state: MonitoringState,
}

impl SecurityMonitor {
    pub async fn monitor_authentication_events(&self) {
        let mut failed_attempts = HashMap::new();
        let mut event_stream = self.get_auth_event_stream().await;

        while let Some(event) = event_stream.next().await {
            match event.event_type {
                AuthEventType::FailedLogin => {
                    let count = failed_attempts.entry(event.client_ip.clone())
                        .and_modify(|c| *c += 1)
                        .or_insert(1);

                    if *count >= self.thresholds.max_failed_attempts {
                        let alert = SecurityAlert {
                            severity: AlertSeverity::High,
                            alert_type: AlertType::BruteForceAttempt,
                            source_ip: event.client_ip.clone(),
                            description: format!(
                                "Brute force attempt detected from {} ({} failed attempts)",
                                event.client_ip, count
                            ),
                            recommended_action: "Block IP address".to_string(),
                        };

                        self.alert_sender.send(alert).await.ok();
                    }
                }
                AuthEventType::SuccessfulLogin => {
                    // Clear failed attempt counter on successful login
                    failed_attempts.remove(&event.client_ip);
                }
                _ => {}
            }
        }
    }

    pub async fn monitor_trading_anomalies(&self) {
        let mut trading_stream = self.get_trading_event_stream().await;

        while let Some(event) = trading_stream.next().await {
            // Detect unusual trading patterns
            if self.is_unusual_trading_pattern(&event).await {
                let alert = SecurityAlert {
                    severity: AlertSeverity::Medium,
                    alert_type: AlertType::UnusualTradingActivity,
                    source_ip: event.client_ip.clone(),
                    description: format!(
                        "Unusual trading pattern detected for user {} - {} orders in {}",
                        event.user_id, event.order_count, event.time_window
                    ),
                    recommended_action: "Review user activity".to_string(),
                };

                self.alert_sender.send(alert).await.ok();
            }
        }
    }
}

Automated Response

#!/bin/bash
# Automated security response script
set -e

ALERT_TYPE="$1"
SOURCE_IP="$2"
USER_ID="$3"
SEVERITY="$4"

echo "Processing security alert: $ALERT_TYPE from $SOURCE_IP"

case "$ALERT_TYPE" in
    "brute_force")
        echo "Brute force attack detected - blocking IP $SOURCE_IP"

        # Block IP immediately
        iptables -A INPUT -s "$SOURCE_IP" -j DROP

        # Add to permanent blocklist
        echo "$SOURCE_IP" >> /etc/foxhunt/security/blocked_ips.txt

        # Notify security team
        send_security_alert "Brute force attack blocked" "$SOURCE_IP" "high"
        ;;

    "unusual_trading")
        echo "Unusual trading activity detected for user $USER_ID"

        # Temporarily restrict user
        cargo run --bin tli-admin -- restrict-user \
            --user-id "$USER_ID" \
            --duration "1h" \
            --reason "Unusual trading activity"

        # Require additional verification
        cargo run --bin tli-admin -- require-mfa \
            --user-id "$USER_ID" \
            --next-login

        # Alert risk management
        send_risk_alert "Unusual trading activity" "$USER_ID" "medium"
        ;;

    "privilege_escalation")
        echo "Privilege escalation attempt detected"

        # Revoke all sessions for user
        cargo run --bin tli-admin -- revoke-user-sessions \
            --user-id "$USER_ID"

        # Lock account
        cargo run --bin tli-admin -- lock-account \
            --user-id "$USER_ID" \
            --reason "Security incident"

        # Immediate security team notification
        send_security_alert "Privilege escalation attempt" "$USER_ID" "critical"
        ;;
esac

# Log response action
logger -t foxhunt-security "Automated response executed for $ALERT_TYPE: $SOURCE_IP/$USER_ID"

Vulnerability Management

Security Scanning

#!/bin/bash
# Automated security scanning script

SCAN_DATE=$(date +%Y%m%d)
REPORT_DIR="/var/log/security/scans/$SCAN_DATE"
mkdir -p "$REPORT_DIR"

echo "Starting security scan for $SCAN_DATE"

# Dependency vulnerability scanning
cargo audit --format json > "$REPORT_DIR/dependency_audit.json"

# Container image scanning
if command -v trivy &> /dev/null; then
    trivy image --format json --output "$REPORT_DIR/container_scan.json" \
        foxhunt/trading-service:latest
fi

# Network service scanning
nmap -sS -O -sV --script vuln localhost > "$REPORT_DIR/network_scan.txt"

# Configuration security check
/opt/foxhunt/scripts/security-config-check.sh > "$REPORT_DIR/config_check.txt"

# TLS configuration test
testssl.sh --jsonfile "$REPORT_DIR/tls_scan.json" localhost:50051

# Generate summary report
python3 /opt/foxhunt/scripts/generate-security-report.py \
    --input-dir "$REPORT_DIR" \
    --output "$REPORT_DIR/security_summary.html"

echo "Security scan completed. Report available at: $REPORT_DIR/security_summary.html"

Patch Management

#!/bin/bash
# Security patch management script

# Check for security updates
apt list --upgradable 2>/dev/null | grep -i security > /tmp/security_updates.txt

if [ -s /tmp/security_updates.txt ]; then
    echo "Security updates available:"
    cat /tmp/security_updates.txt

    # Create system snapshot before patching
    lvcreate -L 10G -s -n system-snapshot-$(date +%Y%m%d) /dev/vg0/root

    # Apply security updates
    apt update
    apt upgrade -y $(grep "security" /tmp/security_updates.txt | cut -d'/' -f1)

    # Restart affected services
    systemctl restart foxhunt-trading-service
    systemctl restart foxhunt-backtesting-service

    # Verify system functionality
    /opt/foxhunt/scripts/post-patch-verification.sh

    if [ $? -eq 0 ]; then
        echo "Patching completed successfully"
        # Remove snapshot after successful verification
        lvremove -f /dev/vg0/system-snapshot-$(date +%Y%m%d)
    else
        echo "Patching failed - rolling back"
        # Rollback to snapshot
        lvconvert --merge /dev/vg0/system-snapshot-$(date +%Y%m%d)
        reboot
    fi
else
    echo "No security updates available"
fi

---

This security documentation is maintained by the Security Team. For security incidents or questions, contact: security@company.com

Classification: Security Sensitive - Authorized Personnel Only