# ML Infrastructure Guide - Master Index

**Status**: 🎯 Production Ready
**Last Updated**: 2025-10-14
**Total Documentation**: 894 files, 11.7 MB
**Purpose**: Central navigation hub for Foxhunt ML infrastructure

---

## 📖 Quick Navigation

| Category | Count | Description |
|----------|-------|-------------|
| [Training Guides](#training-guides) | 371 docs | Model training, checkpoints, hyperparameters |
| [Deployment](#deployment-guides) | 546 docs | Production deployment, infrastructure, operations |
| [Analysis & Reports](#analysis-reports) | 738 docs | Performance analysis, audits, investigations |
| [API Reference](#api-reference) | 716 docs | gRPC endpoints, integrations, service interfaces |
| [Architecture](#architecture-docs) | 463 docs | System design, components, infrastructure |
| [Troubleshooting](#troubleshooting) | 667 docs | Debug guides, fixes, known issues |
| [Quick Start](#quick-start-guides) | 129 docs | Getting started, tutorials, runbooks |

---

## 🚀 Getting Started (Essential Reading)

### New to Foxhunt?
1. **[CLAUDE.md](/home/jgrusewski/Work/foxhunt/CLAUDE.md)** - System overview, architecture, current status (MUST READ)
2. **[README.md](/home/jgrusewski/Work/foxhunt/README.md)** - Project introduction
3. **[Architecture Overview](/home/jgrusewski/Work/foxhunt/docs/ARCHITECTURE.md)** - Core system design

### Setting Up Development Environment
1. **[Production Deployment Runbook V3](/home/jgrusewski/Work/foxhunt/docs/PRODUCTION_DEPLOYMENT_RUNBOOK_V3.md)** - Comprehensive setup (57.4K)
2. **[Docker Deployment Guide](/home/jgrusewski/Work/foxhunt/DOCKER_DEPLOYMENT.md)** - Container orchestration
3. **[Database Architecture](/home/jgrusewski/Work/foxhunt/docs/DATABASE_ARCHITECTURE.md)** - PostgreSQL/TimescaleDB setup

### Running Your First Model
1. **[GPU Benchmark Guide](/home/jgrusewski/Work/foxhunt/ml/docs/GPU_BENCHMARK_GUIDE.md)** - Test GPU training (55.3K)
2. **[ML Training Roadmap](/home/jgrusewski/Work/foxhunt/ML_TRAINING_ROADMAP.md)** - 4-6 week training plan
3. **[Agent 78: DQN Production Training](/home/jgrusewski/Work/foxhunt/AGENT_78_DQN_PRODUCTION_TRAINING_SUCCESS.md)** - Real training example

---

## 🎓 Training Guides

### Core Training Documentation
| Document | Size | Description |
|----------|------|-------------|
| [ML Training Roadmap](/home/jgrusewski/Work/foxhunt/ML_TRAINING_ROADMAP.md) | 22.6K | 4-6 week realistic training plan |
| [GPU Benchmark Guide](/home/jgrusewski/Work/foxhunt/ml/docs/GPU_BENCHMARK_GUIDE.md) | 55.3K | RTX 3050 Ti performance testing |
| [Data Plan](/home/jgrusewski/Work/foxhunt/DATA_PLAN.md) | 99.3K | 90-day data acquisition strategy |
| [Feature Engineering Report](/home/jgrusewski/Work/foxhunt/FEATURE_ENGINEERING_ENHANCEMENT_REPORT.md) | 18.9K | 16 features + 10 indicators |

### Model-Specific Training

#### DQN (Deep Q-Network)
- **[Agent 25: DQN Training Report](/home/jgrusewski/Work/foxhunt/AGENT_25_DQN_TRAINING_REPORT.md)** - Initial training results
- **[Agent 42: DQN Checkpoint Validation](/home/jgrusewski/Work/foxhunt/AGENT_42_DQN_CHECKPOINT_VALIDATION_REPORT.md)** - Checkpoint analysis
- **[Agent 78: DQN Production Success](/home/jgrusewski/Work/foxhunt/AGENT_78_DQN_PRODUCTION_TRAINING_SUCCESS.md)** - Production training
- **[DQN Checkpoint Analysis](/home/jgrusewski/Work/foxhunt/DQN_CHECKPOINT_ANALYSIS_REPORT.md)** - Comprehensive checkpoint review
- **[Checkpoint Selection Framework](/home/jgrusewski/Work/foxhunt/docs/CHECKPOINT_SELECTION_FRAMEWORK.md)** - How to select best checkpoints

#### PPO (Proximal Policy Optimization)
- **[Agent 32: PPO Fix Summary](/home/jgrusewski/Work/foxhunt/AGENT32_PPO_FIX_SUMMARY.md)** - Critical bug fixes
- **[Agent 79: PPO Validation Report](/home/jgrusewski/Work/foxhunt/AGENT_79_PPO_VALIDATION_REPORT.md)** - Production validation
- **[PPO Checkpoint Analysis](/home/jgrusewski/Work/foxhunt/PPO_CHECKPOINT_ANALYSIS_REPORT.md)** - Checkpoint review
- **[PPO Value Network Deep Dive](/home/jgrusewski/Work/foxhunt/PPO_VALUE_NETWORK_DEEP_DIVE.md)** - Architecture details
- **[PPO Value Network Fix](/home/jgrusewski/Work/foxhunt/PPO_VALUE_NETWORK_FIX.md)** - Critical fixes

#### MAMBA-2 (State Space Model)
- **[MAMBA-2 Hyperparameter Tuning](/home/jgrusewski/Work/foxhunt/MAMBA2_HYPERPARAMETER_TUNING_REPORT.md)** - Optuna tuning results

#### TFT (Temporal Fusion Transformer)
- **Training documentation in progress** - See Wave 160 reports

#### TLOB (Tick-Level Order Book)
- **[TLOB Training Status](/home/jgrusewski/Work/foxhunt/TLOB_TRAINING_INTEGRATION_STATUS.md)** - Level-2 data requirements
- **Status**: Inference-only, training requires order book data (not available)

### Checkpoint Management
- **[Checkpoint Selection Framework](/home/jgrusewski/Work/foxhunt/docs/CHECKPOINT_SELECTION_FRAMEWORK.md)** - Systematic selection methodology
- **[Checkpoint Selection Quickstart](/home/jgrusewski/Work/foxhunt/docs/CHECKPOINT_SELECTION_QUICKSTART.md)** - Quick reference
- **[Checkpoint Selection Summary](/home/jgrusewski/Work/foxhunt/docs/CHECKPOINT_SELECTION_SUMMARY.txt)** - Executive summary
- **[DQN Checkpoint Analysis Script](/home/jgrusewski/Work/foxhunt/ml/examples/analyze_dqn_checkpoints.rs)** - Rust analysis tool
- **[Quick Checkpoint Analysis Script](/home/jgrusewski/Work/foxhunt/ml/examples/quick_checkpoint_analysis.rs)** - Fast checkpoint review

### Hyperparameter Tuning
- **[Optuna Tuning Integration](/home/jgrusewski/Work/foxhunt/OPTUNA_TUNING_INTEGRATION_REPORT.md)** - HPO framework (26.8K)
- **[Tuning Quickstart Guide](/home/jgrusewski/Work/foxhunt/TUNING_QUICKSTART_GUIDE.md)** - TLI tuning commands
- **[MAMBA-2 Tuning Report](/home/jgrusewski/Work/foxhunt/MAMBA2_HYPERPARAMETER_TUNING_REPORT.md)** - Model-specific tuning
- **Configuration**: `tuning_config.yaml` - Search spaces for all models

---

## 🏗️ Deployment Guides

### Production Deployment
| Document | Size | Description |
|----------|------|-------------|
| [Production Runbook V3](/home/jgrusewski/Work/foxhunt/docs/PRODUCTION_DEPLOYMENT_RUNBOOK_V3.md) | 57.4K | Complete deployment guide |
| [Production Deployment Guide V2](/home/jgrusewski/Work/foxhunt/docs/PRODUCTION_DEPLOYMENT_GUIDE_V2.md) | 51.5K | Detailed procedures |
| [Production Runbook (Root)](/home/jgrusewski/Work/foxhunt/PRODUCTION_DEPLOYMENT_RUNBOOK.md) | 54.8K | Original runbook |
| [Comprehensive Deployment Guide](/home/jgrusewski/Work/foxhunt/docs/COMPREHENSIVE_DEPLOYMENT_GUIDE.md) | 33.5K | All-in-one reference |
| [Docker Deployment](/home/jgrusewski/Work/foxhunt/DOCKER_DEPLOYMENT.md) | 14.2K | Container orchestration |

### Ensemble & Paper Trading
- **[Ensemble Production Deployment Strategy](/home/jgrusewski/Work/foxhunt/ENSEMBLE_PRODUCTION_DEPLOYMENT_STRATEGY.md)** - Multi-model deployment (43.2K)
- **[Ensemble Runbook](/home/jgrusewski/Work/foxhunt/ENSEMBLE_RUNBOOK.md)** - Operations guide (36.9K)
- **[Paper Trading Deployment Plan](/home/jgrusewski/Work/foxhunt/PAPER_TRADING_DEPLOYMENT_PLAN.md)** - Safe testing (39.2K)
- **[Ensemble Paper Trading Summary](/home/jgrusewski/Work/foxhunt/ENSEMBLE_PAPER_TRADING_EXECUTIVE_SUMMARY.md)** - Executive overview (9.4K)
- **[Deployment Executive Summary](/home/jgrusewski/Work/foxhunt/DEPLOYMENT_EXECUTIVE_SUMMARY.md)** - High-level overview (21.3K)

### Infrastructure & Scaling
- **[Load Balancing & Scaling](/home/jgrusewski/Work/foxhunt/LOAD_BALANCING_SCALING.md)** - Horizontal scaling (35.6K)
- **[CI/CD Pipeline](/home/jgrusewski/Work/foxhunt/CI_CD_PIPELINE.md)** - Automation (32.6K)
- **[Rollout Timeline](/home/jgrusewski/Work/foxhunt/ROLLOUT_TIMELINE.md)** - Phased deployment (30.7K)

### Security & Compliance
- **[Security Hardening](/home/jgrusewski/Work/foxhunt/docs/SECURITY_HARDENING.md)** - Production security (33.5K)
- **[Security Audit Report](/home/jgrusewski/Work/foxhunt/SECURITY_AUDIT_REPORT.md)** - Comprehensive audit (40.5K)
- **[Security Incident Response](/home/jgrusewski/Work/foxhunt/docs/SECURITY_INCIDENT_RESPONSE.md)** - IR procedures (21.2K)
- **[TLI Security Documentation](/home/jgrusewski/Work/foxhunt/docs/TLI_SECURITY_DOCUMENTATION.md)** - Client security (39.5K)
- **[TLI Compliance Documentation](/home/jgrusewski/Work/foxhunt/docs/TLI_COMPLIANCE_DOCUMENTATION.md)** - Regulatory compliance (50.5K)

### SOX Compliance
- **[SOX Compliance Guide](/home/jgrusewski/Work/foxhunt/docs/sox/SOX_COMPLIANCE_GUIDE.md)** - Full SOX implementation
- **[Audit Trail Queries](/home/jgrusewski/Work/foxhunt/docs/sox/AUDIT_TRAIL_QUERIES.md)** - SQL queries for auditors
- **[Separation of Duties](/home/jgrusewski/Work/foxhunt/docs/sox/SEPARATION_OF_DUTIES.md)** - Access control
- **[Change Control Templates](/home/jgrusewski/Work/foxhunt/docs/sox/CHANGE_CONTROL_TEMPLATES.md)** - Change management

---

## 📊 Analysis & Reports

### Wave Reports (Phase-based Development)

#### Wave 160 (Current Phase - ML Training Complete)
- **[Wave 160 Phase 4 Complete](/home/jgrusewski/Work/foxhunt/WAVE_160_PHASE4_COMPLETE.md)** - 19 agents, 4 models (46.4K)
- **[Wave 160 Phase 3 Complete](/home/jgrusewski/Work/foxhunt/WAVE_160_PHASE3_COMPLETE.md)** - Bug fixes + GPU training (29.3K)

#### Wave 159 (ML Training Infrastructure)
- **[Wave 159 Training Fix Report](/home/jgrusewski/Work/foxhunt/WAVE_159_TRAINING_FIX_REPORT.md)** - 22 parallel agents (31.3K)

#### Wave 152 (GPU Benchmark System)
- **[Wave 152 GPU Benchmark Summary](/home/jgrusewski/Work/foxhunt/WAVE_152_GPU_BENCHMARK_SUMMARY.md)** - Benchmark system (31.7K)

#### Wave 154 (TLI Token Persistence)
- **[Wave 154 Final Summary](/home/jgrusewski/Work/foxhunt/WAVE_154_FINAL_SUMMARY.md)** - Token storage fix (32.0K)

#### Wave 141 (Production Readiness)
- **[Wave 141 Production Readiness](/home/jgrusewski/Work/foxhunt/WAVE_141_PRODUCTION_READINESS_REPORT.md)** - Full system validation (36.0K)

#### Wave 150 (Infrastructure)
- **[Wave 150 Progress Report](/home/jgrusewski/Work/foxhunt/WAVE_150_PROGRESS_REPORT.md)** - Milestone achievements (11.8K)

### ML Model Analysis
- **[ML Validation Metrics Framework](/home/jgrusewski/Work/foxhunt/ML_VALIDATION_METRICS_FRAMEWORK.md)** - Testing methodology (43.3K)
- **[ML Model Diversity Strategy](/home/jgrusewski/Work/foxhunt/ML_MODEL_DIVERSITY_STRATEGY.md)** - Multi-model approach (39.3K)
- **[ML Research Summary 2025](/home/jgrusewski/Work/foxhunt/ML_RESEARCH_SUMMARY_2025.md)** - State of the art (38.8K)
- **[Ensemble Strategy Deep Analysis](/home/jgrusewski/Work/foxhunt/ENSEMBLE_STRATEGY_DEEP_ANALYSIS.md)** - Model combination (57.1K)
- **[Convergence Analysis Report](/home/jgrusewski/Work/foxhunt/CONVERGENCE_ANALYSIS_REPORT.md)** - Training convergence (27.5K)
- **[Convergence Executive Summary](/home/jgrusewski/Work/foxhunt/CONVERGENCE_EXECUTIVE_SUMMARY.md)** - High-level overview (16.9K)

### Data Quality & Strategy
- **[90-Day Data Expansion Plan](/home/jgrusewski/Work/foxhunt/90_DAY_DATA_EXPANSION_PLAN.md)** - Data acquisition (30.0K)
- **[90-Day Data Quality Report](/home/jgrusewski/Work/foxhunt/90_DAY_DATA_QUALITY_REPORT.md)** - Data validation (14.0K)
- **[90-Day Data Status Summary](/home/jgrusewski/Work/foxhunt/90_DAY_DATA_STATUS_SUMMARY.md)** - Current status (13.0K)
- **[ML Data Validation Report](/home/jgrusewski/Work/foxhunt/ML_DATA_VALIDATION_REPORT.md)** - Real data testing (24.3K)
- **[Multi-Symbol Integration Complete](/home/jgrusewski/Work/foxhunt/MULTI_SYMBOL_INTEGRATION_COMPLETE.md)** - ES/NQ/ZN/6E (11.0K)

### Performance & Benchmarking
- **[Performance Summary](/home/jgrusewski/Work/foxhunt/PERFORMANCE_SUMMARY.md)** - System benchmarks (27.5K)
- **[Order Matching Benchmark Report](/home/jgrusewski/Work/foxhunt/ORDER_MATCHING_BENCHMARK_REPORT.md)** - 1-6μs P99 (13.1K)
- **[Wave 71: Performance Benchmarks](/home/jgrusewski/Work/foxhunt/docs/WAVE71_AGENT4_PERFORMANCE_BENCHMARKS.md)** - Comprehensive testing (15.7K)

### Agent-Specific Reports
- **[Agent 78: DQN Production Training Success](/home/jgrusewski/Work/foxhunt/AGENT_78_DQN_PRODUCTION_TRAINING_SUCCESS.md)** - Production model
- **[Agent 79: PPO Validation Report](/home/jgrusewski/Work/foxhunt/AGENT_79_PPO_VALIDATION_REPORT.md)** - PPO production validation
- **[Agent 71: Model Validation Report](/home/jgrusewski/Work/foxhunt/AGENT_71_MODEL_VALIDATION_REPORT.md)** - Model testing
- **[Agent 72: DBN Parser Fix Report](/home/jgrusewski/Work/foxhunt/AGENT_72_DBN_PARSER_FIX_REPORT.md)** - Data loading fix
- **[Agent 86: Quickstart](/home/jgrusewski/Work/foxhunt/AGENT_86_QUICKSTART.md)** - Quick reference

---

## 🔌 API Reference

### gRPC Services

#### API Gateway (Port 50051)
- **Authentication & Authorization**
  - `Login(LoginRequest) → LoginResponse` - JWT + MFA authentication
  - `ValidateToken(ValidateTokenRequest) → ValidateTokenResponse` - Token validation
  - `RefreshToken(RefreshTokenRequest) → RefreshTokenResponse` - Token renewal

- **Configuration Management**
  - `GetConfig(GetConfigRequest) → GetConfigResponse` - Retrieve configuration
  - `UpdateConfig(UpdateConfigRequest) → UpdateConfigResponse` - Update settings

- **Health & Monitoring**
  - `HealthCheck(HealthCheckRequest) → HealthCheckResponse` - Service health
  - Standard gRPC health protocol

#### Trading Service (Port 50052)
- **Order Management**
  - `SubmitOrder(SubmitOrderRequest) → SubmitOrderResponse` - Place orders
  - `CancelOrder(CancelOrderRequest) → CancelOrderResponse` - Cancel orders
  - `GetOrderStatus(GetOrderStatusRequest) → GetOrderStatusResponse` - Order status

- **Position Management**
  - `GetPositions(GetPositionsRequest) → GetPositionsResponse` - Current positions
  - `GetPortfolio(GetPortfolioRequest) → GetPortfolioResponse` - Portfolio summary

- **Market Data**
  - `StreamMarketData(StreamMarketDataRequest) → Stream<MarketDataUpdate>` - Real-time data
  - `GetMarketSnapshot(GetMarketSnapshotRequest) → GetMarketSnapshotResponse` - Current prices

#### Backtesting Service (Port 50053)
- **Backtest Execution**
  - `RunBacktest(RunBacktestRequest) → RunBacktestResponse` - Execute backtest
  - `GetBacktestResults(GetBacktestResultsRequest) → GetBacktestResultsResponse` - Retrieve results

- **Strategy Management**
  - `ListStrategies(ListStrategiesRequest) → ListStrategiesResponse` - Available strategies
  - `ValidateStrategy(ValidateStrategyRequest) → ValidateStrategyResponse` - Strategy validation

#### ML Training Service (Port 50054)
- **Model Training**
  - `TrainModel(TrainModelRequest) → TrainModelResponse` - Train ML models
  - `GetTrainingStatus(GetTrainingStatusRequest) → GetTrainingStatusResponse` - Training progress
  - `StreamTrainingProgress(StreamTrainingProgressRequest) → Stream<TrainingProgressUpdate>` - Real-time updates

- **Checkpoint Management**
  - `ListCheckpoints(ListCheckpointsRequest) → ListCheckpointsResponse` - Available checkpoints
  - `LoadCheckpoint(LoadCheckpointRequest) → LoadCheckpointResponse` - Load model
  - `DeleteCheckpoint(DeleteCheckpointRequest) → DeleteCheckpointResponse` - Remove checkpoint

- **Hyperparameter Tuning**
  - `StartTuningJob(StartTuningJobRequest) → StartTuningJobResponse` - Begin Optuna tuning
  - `GetTuningStatus(GetTuningStatusRequest) → GetTuningStatusResponse` - Tuning progress
  - `GetBestHyperparameters(GetBestHyperparametersRequest) → GetBestHyperparametersResponse` - Optimal params
  - `StopTuningJob(StopTuningJobRequest) → StopTuningJobResponse` - Cancel tuning

### TLI Commands (Terminal Client)

#### Authentication
```bash
tli login --username <user> --password <pass> [--mfa-code <code>]
tli logout
```

#### Trading Operations
```bash
tli order submit --symbol ES.FUT --side buy --quantity 10 --price 4500.0
tli order cancel --order-id <uuid>
tli order status --order-id <uuid>
tli positions list
tli portfolio summary
```

#### Backtesting
```bash
tli backtest run --strategy moving_average --symbol ES.FUT --start 2024-01-01 --end 2024-12-31
tli backtest results --backtest-id <uuid>
tli backtest list
```

#### ML Training
```bash
tli train start --model DQN --symbol ES.FUT --epochs 100
tli train status --job-id <uuid>
tli train list
tli checkpoints list --model DQN
tli checkpoints load --checkpoint-id <uuid>
```

#### Hyperparameter Tuning
```bash
tli tune start --model DQN --trials 50 --watch
tli tune status --job-id <uuid>
tli tune best --job-id <uuid>
tli tune stop --job-id <uuid>
```

#### Configuration & Health
```bash
tli config get --key <key>
tli config set --key <key> --value <value>
tli health check [--service api-gateway|trading|backtesting|ml-training]
```

---

## 🏛️ Architecture Documentation

### Core Architecture
- **[Architecture Overview](/home/jgrusewski/Work/foxhunt/docs/ARCHITECTURE.md)** - System design (20.0K)
- **[Database Architecture](/home/jgrusewski/Work/foxhunt/docs/DATABASE_ARCHITECTURE.md)** - PostgreSQL/TimescaleDB (10.8K)
- **[Security Architecture](/home/jgrusewski/Work/foxhunt/docs/SECURITY.md)** - Security design (28.2K)

### Component Documentation
- **[Trading Engine](/home/jgrusewski/Work/foxhunt/trading_engine/README.md)** - Core HFT engine
- **[ML Pipeline](/home/jgrusewski/Work/foxhunt/ml/README.md)** - ML infrastructure
- **[Risk Management](/home/jgrusewski/Work/foxhunt/risk/README.md)** - VaR, circuit breakers
- **[TLI Client](/home/jgrusewski/Work/foxhunt/tli/README.md)** - Terminal interface

### Service Architecture
- **API Gateway**: Single entry point, auth, rate limiting, audit logging
- **Trading Service**: Order execution, position management, risk integration
- **Backtesting Service**: Strategy testing with real DBN data (0.70ms load time)
- **ML Training Service**: Model training, HPO, checkpoint management

---

## 🔧 Troubleshooting

### Common Issues

#### Port Conflicts
```bash
# Check port usage
lsof -i :50051  # API Gateway
lsof -i :50052  # Trading Service
lsof -i :50053  # Backtesting Service
lsof -i :50054  # ML Training Service

# Kill conflicting process
kill -9 $(lsof -ti:50051)
```

#### Database Connection
```bash
# Test PostgreSQL connection
psql postgresql://foxhunt:foxhunt_dev_password@localhost:5432/foxhunt

# Run migrations
cargo sqlx migrate run

# Check migration status
cargo sqlx migrate info
```

#### GPU/CUDA Issues
```bash
# Verify GPU
nvidia-smi

# Check CUDA version
nvcc --version

# Test CUDA availability
python3 -c "import torch; print(torch.cuda.is_available())"
```

#### Service Health
```bash
# Check all services
docker-compose ps

# View logs
docker-compose logs -f api_gateway
docker-compose logs -f trading_service
docker-compose logs -f backtesting_service
docker-compose logs -f ml_training_service

# Restart services
docker-compose restart
```

### Known Issues & Fixes
- **[Wave 145: JWT Fix Results](/home/jgrusewski/Work/foxhunt/WAVE_145_JWT_FIX_RESULTS.md)** - JWT authentication fixes
- **[Migration Verification Report](/home/jgrusewski/Work/foxhunt/MIGRATION_VERIFICATION_REPORT.md)** - Database migration issues
- **[Agent 72: DBN Parser Fix](/home/jgrusewski/Work/foxhunt/AGENT_72_DBN_PARSER_FIX_REPORT.md)** - Data loading fixes

### Debugging Guides
- **[Troubleshooting Guide](/home/jgrusewski/Work/foxhunt/docs/TROUBLESHOOTING_GUIDE.md)** - Comprehensive debugging (10.4K)
- **[Compilation Victory](/home/jgrusewski/Work/foxhunt/docs/COMPILATION_VICTORY.md)** - Build issues (8.0K)
- **[Wave 101: Compilation Fixes](/home/jgrusewski/Work/foxhunt/docs/WAVE101_COMPILATION_FIXES.md)** - Build troubleshooting (16.2K)

---

## ⚡ Quick Start Guides

### 5-Minute Quickstarts
1. **[Checkpoint Selection Quickstart](/home/jgrusewski/Work/foxhunt/docs/CHECKPOINT_SELECTION_QUICKSTART.md)** - Choose best model
2. **[Tuning Quickstart Guide](/home/jgrusewski/Work/foxhunt/TUNING_QUICKSTART_GUIDE.md)** - Start hyperparameter tuning
3. **[Ensemble Weight Optimization Quickstart](/home/jgrusewski/Work/foxhunt/ENSEMBLE_WEIGHT_OPTIMIZATION_QUICKSTART.md)** - Optimize ensemble
4. **[Agent 86 Quickstart](/home/jgrusewski/Work/foxhunt/AGENT_86_QUICKSTART.md)** - Quick reference
5. **[Ensemble Metrics Quick Reference](/home/jgrusewski/Work/foxhunt/ENSEMBLE_METRICS_QUICK_REFERENCE.md)** - Key metrics

### Essential Scripts
```bash
# GPU Training Benchmark (30-60 min)
cargo run -p ml --example gpu_training_benchmark --release

# Quick Checkpoint Analysis
cargo run -p ml --example quick_checkpoint_analysis --release

# DQN Checkpoint Deep Dive
cargo run -p ml --example analyze_dqn_checkpoints --release

# Verify Dataset Coverage
./verify_dataset_coverage.sh

# Test DQN Checkpoints
./test_dqn_checkpoints_quick.sh
```

### Step-by-Step Tutorials
1. **Set up development environment**: Docker + PostgreSQL + Redis + Vault
2. **Run GPU benchmark**: Determine training platform (local vs cloud)
3. **Download 90-day data**: ES/NQ/ZN/6E futures (~$2)
4. **Train first model**: DQN with ES.FUT data
5. **Validate checkpoint**: Select best performing checkpoint
6. **Run backtest**: Test strategy with real data
7. **Deploy paper trading**: Safe live testing

---

## 📚 Additional Resources

### Testing Documentation
- **[Testing Guide](/home/jgrusewski/Work/foxhunt/tests/README.md)** - Comprehensive testing (45.7K)
- **[Testing Plan](/home/jgrusewski/Work/foxhunt/TESTING_PLAN.md)** - ML testing strategy (28.6K)
- **[Adaptive Strategy E2E Report](/home/jgrusewski/Work/foxhunt/ADAPTIVE_STRATEGY_E2E_REPORT.md)** - E2E testing (14.4K)

### Strategy Development
- **[Adaptive Strategy Stub Analysis](/home/jgrusewski/Work/foxhunt/ADAPTIVE_STRATEGY_STUB_ANALYSIS.md)** - Strategy patterns (29.1K)
- **[Adaptive ML Integration Report](/home/jgrusewski/Work/foxhunt/ADAPTIVE_ML_INTEGRATION_REPORT.md)** - ML integration (17.0K)
- **[Comprehensive Backtest Design](/home/jgrusewski/Work/foxhunt/COMPREHENSIVE_BACKTEST_DESIGN.md)** - Backtest framework (19.6K)
- **[Comprehensive Backtest Summary](/home/jgrusewski/Work/foxhunt/COMPREHENSIVE_BACKTEST_SUMMARY.md)** - Results analysis (18.4K)

### Advanced Features
- **[Early Stopping Implementation Guide](/home/jgrusewski/Work/foxhunt/EARLY_STOPPING_IMPLEMENTATION_GUIDE.md)** - Training optimization (26.1K)
- **[Ensemble Implementation Guide](/home/jgrusewski/Work/foxhunt/ENSEMBLE_IMPLEMENTATION_GUIDE.md)** - Multi-model ensemble (29.6K)
- **[Streaming Progress Implementation](/home/jgrusewski/Work/foxhunt/STREAMING_PROGRESS_IMPLEMENTATION.md)** - Real-time updates (12.0K)
- **[AB Testing Implementation Status](/home/jgrusewski/Work/foxhunt/AB_TESTING_IMPLEMENTATION_STATUS.md)** - A/B testing (19.0K)
- **[AB Testing Final Summary](/home/jgrusewski/Work/foxhunt/AB_TESTING_FINAL_SUMMARY.md)** - Results (9.2K)

### Data Providers
- **[Databento CL.FUT Download Report](/home/jgrusewski/Work/foxhunt/docs/databento_cl_fut_download_report.md)** - Data acquisition
- **[Multi-Symbol Integration Complete](/home/jgrusewski/Work/foxhunt/MULTI_SYMBOL_INTEGRATION_COMPLETE.md)** - ES/NQ/ZN/6E support

### Configuration
- **[Runtime Config Integration](/home/jgrusewski/Work/foxhunt/docs/runtime_config_integration.md)** - Dynamic configuration (9.4K)
- **[Wave 76: Secrets Config](/home/jgrusewski/Work/foxhunt/docs/WAVE76_AGENT5_SECRETS_CONFIG.md)** - Vault integration (6.8K)

---

## 🗂️ Documentation Organization

### Root Directory (`/home/jgrusewski/Work/foxhunt/`)
- **421 markdown files** - Primarily wave reports, agent reports, executive summaries
- **Focus**: High-level reports, analysis, strategic planning
- **Audience**: Leadership, architects, project managers

### `/docs` Directory
- **306 markdown files** - Technical documentation, guides, runbooks
- **Focus**: Implementation details, operations, procedures
- **Audience**: Developers, operators, DevOps engineers

### Model-Specific Directories
- **`/ml/docs`** - ML-specific documentation (GPU benchmarking, training guides)
- **`/tests/`** - Test documentation and patterns
- **`/docs/sox`** - SOX compliance documentation

---

## 🔍 Search Index

### By Topic
- **Authentication**: JWT, MFA, token management → Security section
- **Backtesting**: Strategy testing, performance → Backtesting section
- **Checkpoints**: Model saving, loading, selection → Training Guides
- **Deployment**: Production, Docker, Kubernetes → Deployment Guides
- **GPU**: CUDA, RTX 3050 Ti, benchmarking → Training Guides
- **Hyperparameters**: Tuning, Optuna, optimization → Tuning section
- **Models**: DQN, PPO, MAMBA-2, TFT, TLOB → Training Guides
- **Performance**: Benchmarks, profiling, optimization → Analysis section
- **Security**: TLS, audit trails, compliance → Security section
- **Testing**: Unit tests, integration tests, E2E → Testing section

### By File Size (Top 20 Largest)
1. DATA_PLAN.md (99.3K) - 90-day data strategy
2. TLI_PLAN.md (57.5K) - TLI design
3. docs/PRODUCTION_DEPLOYMENT_RUNBOOK_V3.md (57.4K) - Deployment
4. ENSEMBLE_STRATEGY_DEEP_ANALYSIS.md (57.1K) - Ensemble analysis
5. ml/docs/GPU_BENCHMARK_GUIDE.md (55.3K) - GPU testing
6. PRODUCTION_DEPLOYMENT_RUNBOOK.md (54.8K) - Operations
7. docs/PRODUCTION_DEPLOYMENT_GUIDE_V2.md (51.5K) - Deployment
8. docs/TLI_COMPLIANCE_DOCUMENTATION.md (50.5K) - Compliance
9. WAVE_160_PHASE4_COMPLETE.md (46.4K) - Phase 4 report
10. tests/README.md (45.7K) - Testing guide
11. ML_VALIDATION_METRICS_FRAMEWORK.md (43.3K) - ML testing
12. ENSEMBLE_PRODUCTION_DEPLOYMENT_STRATEGY.md (43.2K) - Ensemble
13. SECURITY_AUDIT_REPORT.md (40.5K) - Security audit
14. ML_MODEL_DIVERSITY_STRATEGY.md (39.3K) - Model strategy
15. PAPER_TRADING_DEPLOYMENT_PLAN.md (39.2K) - Paper trading
16. docs/TLI_SECURITY_DOCUMENTATION.md (39.5K) - TLI security
17. ML_RESEARCH_SUMMARY_2025.md (38.8K) - ML research
18. docs/WAVE76_AGENT11_FINAL_CERTIFICATION.md (38.2K) - Certification
19. ENSEMBLE_RUNBOOK.md (36.9K) - Operations
20. WAVE_141_PRODUCTION_READINESS_REPORT.md (36.0K) - Production

---

## 📅 Recent Updates

### 2025-10-14
- Created ML Infrastructure Guide (master index)
- Analyzed 894 documentation files (11.7 MB total)
- Categorized documentation by topic and priority
- Established navigation structure

### 2025-10-13 (Wave 160 Phase 4)
- Completed ML training pipeline (19 agents, 4 models)
- DQN production training successful
- PPO validation complete
- System 100% production ready

### 2025-10-12 (Wave 160 Phase 3)
- Critical bug fixes in ML training
- GPU-accelerated training operational
- DBN parser fixes for real data

---

## 🎯 Next Steps

### Immediate (This Week)
1. **Execute GPU Training Benchmark** (30-60 min)
   - Command: `cargo run -p ml --example gpu_training_benchmark --release`
   - Decision: Local RTX 3050 Ti vs Cloud A100

2. **Download 90-Day Data** (~$2)
   - ES.FUT, NQ.FUT, ZN.FUT, 6E.FUT
   - ~180,000 bars total

3. **Start Model Training** (4-6 weeks)
   - Week 1: Data prep + feature engineering
   - Week 2: MAMBA-2 training
   - Week 3: DQN + PPO training
   - Week 4: TFT training
   - Week 5-6: Integration + validation

### Short-term (1-2 Months)
1. Complete ML model training (all 4 models)
2. Validate models with production data
3. Deploy paper trading (safe live testing)
4. Increase test coverage (47% → >60%)

### Long-term (3-6 Months)
1. Production deployment (live trading)
2. External penetration testing ($50K-$75K)
3. SOX/MiFID II audit (Q1 2026)
4. Multi-region deployment

---

## 💡 Tips for Documentation Users

### Finding Information
1. **Start with this guide** - Master index for all documentation
2. **Use Ctrl+F** - Search this document for keywords
3. **Check recent wave reports** - Latest changes and features
4. **Review agent reports** - Detailed implementation notes
5. **Consult quickstart guides** - Fast answers for common tasks

### Contributing to Documentation
1. **Update this master index** when adding new docs
2. **Use clear, descriptive titles** for new documents
3. **Add cross-references** to related documentation
4. **Include file sizes and dates** in listings
5. **Tag documents** with relevant keywords

### Maintaining Documentation
1. **Archive obsolete docs** - Move to `/docs/archive`
2. **Consolidate duplicates** - Merge similar documents
3. **Update cross-references** - Keep links current
4. **Version control** - Track major changes
5. **Regular audits** - Quarterly documentation review

---

## 📧 Support & Contact

### Documentation Issues
- **Missing documentation?** Create GitHub issue with `docs` label
- **Broken links?** Submit PR with fix
- **Outdated content?** File issue with current status

### Technical Support
- **Development**: Check `/docs/TROUBLESHOOTING_GUIDE.md`
- **Deployment**: Review production runbooks
- **ML Training**: Consult training guides
- **Performance**: See performance benchmarks

---

**Document Version**: 1.0
**Created**: 2025-10-14
**Last Updated**: 2025-10-14
**Maintainer**: Foxhunt Development Team
**Status**: 🎯 Active Maintenance