ebca31b559
WAVE 7: Complete cleanup of obsolete documentation and logs Documentation Cleanup: - Archived 34 historical MD files to docs/archive/feature_reduction_campaign_2025_11_23/ - Created comprehensive INDEX.md with catalog of all archived documents - Kept 5 essential reference files in /tmp - Result: 91% reduction in /tmp feature files (43 → 5) Log Cleanup: - Archived 6 valuable production logs (compressed, 68 MB) - Deleted ~600 obsolete log files from /tmp - Space freed: 4.2 GB (89% reduction) - Archived logs: dqn_hyperopt_baseline, epoch1_norm_100epoch, production runs Checkpoint Cleanup: - Deleted 39 obsolete DQN model checkpoints - Kept 3 most recent production checkpoints (891 KB) - Space freed: 12 MB - Updated .gitignore to prevent future checkpoint spam CLAUDE.md Updates: - Added Feature Reduction Campaign Complete section (lines 10-33) - Updated ML Model Status table with 54-feature architecture - Updated 5 legacy references (225→54 features) - Preserved historical Wave D context Files Modified: - .gitignore: Added checkpoint patterns - CLAUDE.md: +41 lines (campaign summary + updates) - docs/archive/: +34 MD files + 6 compressed logs + INDEX.md - ml/trained_models/: -39 obsolete checkpoint files Impact: - /tmp space freed: 4.2 GB - Archived documentation: 34 files (69 MB) - Clean project structure with comprehensive historical archive - Updated documentation reflects current 54-feature architecture Next: Phase 3 Production Validation (100-epoch DQN training) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Foxhunt Documentation Index
Last Updated: 2025-10-22 Status: Organized and Indexed + Wave 5 Operational Documentation Complete Total Documentation: 940 files, 12.4 MB (includes 28 new operational docs)
🎯 Start Here
New to Foxhunt?
- CLAUDE.md - System overview, architecture, current status (MUST READ)
- README.md - Project introduction
- ML Infrastructure Guide - Master documentation index
Quick Start Guides
- Quick Start: Training - Train your first model (5-7 weeks)
- Quick Start: Tuning - Optimize hyperparameters (3-4 days)
📁 Documentation Categories
Operational Guides (deployment/, runbooks/, troubleshooting/, monitoring/, templates/) 🆕
28 documents (Wave 5) - Production deployment, incident response, troubleshooting
- Deployment Guides (5 docs): Docker, Kubernetes, Cloud (AWS/GCP/Azure), Zero-Downtime, Rollback Procedures
- Operational Runbooks (6 docs): Incident Response (P0-P4), Service Restart, Database Migration, Disaster Recovery, Scaling, Security Incidents
- Troubleshooting Guides (7 docs): High Latency, Memory Leaks, Service Crashes, Database Issues, GPU Errors, Network Errors, Circuit Breakers
- Monitoring Playbooks (5 docs): Prometheus Setup, Grafana Setup, Alerting Rules, SLO/SLI Tracking, Log Aggregation
- Templates & Checklists (5 docs): Deployment Checklist (25 items), Incident Report, Change Request, Runbook Template, On-Call Handoff
Key Files:
- Docker Deployment Guide - 15-20 min deployment
- Incident Response Runbook - P0-P4 incident handling
- High Latency Troubleshooting - P99 < 500ms target
- Rollback Procedures - L1-L4 rollback strategies (RTO: 15 min)
Training Guides (training/)
371 documents - ML model training, checkpoints, hyperparameters
- DQN, PPO, MAMBA-2, TFT training
- Checkpoint management
- Feature engineering
- GPU optimization
Key Files:
- ML Training Roadmap
- GPU Benchmark Guide
- Agent 78: DQN Production Training
- Checkpoint Selection Framework
Deployment Guides (deployment/)
546 documents - Production deployment, infrastructure, operations
- Production runbooks
- Docker deployment
- Infrastructure scaling
- Security hardening
Key Files:
- Production Deployment Runbook V3
- Ensemble Production Deployment
- Paper Trading Deployment
- Docker Deployment
Analysis & Reports (analysis/)
738 documents - Performance analysis, audits, investigations
- Wave reports (488 files)
- Agent reports
- Performance benchmarks
- Security audits
Key Files:
API Reference (api/)
716 documents - gRPC endpoints, integrations, service interfaces
- API Gateway (22 methods)
- Trading Service
- Backtesting Service
- ML Training Service
Key Files:
- ML Infrastructure Guide - API Section
- gRPC proto files in service directories
Quick Start Guides (guides/)
129 documents - Getting started, tutorials, runbooks
- Training guides
- Tuning guides
- Deployment guides
- Troubleshooting guides
Key Files:
Troubleshooting (troubleshooting/)
667 documents - Debug guides, fixes, known issues
- Port conflicts
- GPU/CUDA issues
- Database connection
- Service health
Key Files:
Archive (archive/)
50+ candidates - Obsolete and historical documentation
- Superseded versions
- Completed wave reports
- Temporary handoffs
- Duplicate content
🔍 Find Documentation By...
By Topic
- Authentication → Security section
- Backtesting → Training guides + Deployment
- Checkpoints → Training guides
- Deployment → Deployment guides
- GPU/CUDA → Training guides
- Hyperparameters → Tuning guides
- Models (DQN/PPO/MAMBA-2/TFT) → Training guides
- Performance → Analysis section
- Security → Deployment guides
- Testing → Analysis section
By Use Case
| I want to... | Start here |
|---|---|
| Train a model | Quick Start: Training |
| Optimize hyperparameters | Quick Start: Tuning |
| Deploy to production | Production Deployment Runbook V3 |
| Troubleshoot an issue | Troubleshooting Guide |
| Understand the API | ML Infrastructure Guide - API Section |
| Set up paper trading | Paper Trading Deployment Plan |
📊 Documentation Statistics
By Category
- Analysis/Reports: 738 files (80.9%)
- API Reference: 716 files (78.5%)
- Troubleshooting: 667 files (73.1%)
- Deployment: 546 files (59.9%)
- Wave Reports: 488 files (53.5%)
- Architecture: 463 files (50.8%)
- Training: 371 files (40.7%)
By Size
- Total: 11.7 MB (404,079 lines)
- Largest: DATA_PLAN.md (99.3K)
- Average: 13.1K per file
By Location
- Root directory: 421 files (46%)
- Docs directory: 334 files (37%)
- Other directories: 157 files (17%)
🔧 Contributing to Documentation
Adding New Documentation
- Choose appropriate category directory
- Follow naming convention (UPPERCASE_SNAKE_CASE.md)
- Add entry to ML_INFRASTRUCTURE_GUIDE.md
- Include cross-references to related docs
- Update this README if adding new category
Updating Existing Documentation
- Update file content
- Update "Last Updated" date
- Update cross-references if structure changes
- Update ML_INFRASTRUCTURE_GUIDE.md if major changes
Archiving Documentation
- Move to
docs/archive/YYYY-MM-DD-reason/ - Create README in archive directory
- Update ML_INFRASTRUCTURE_GUIDE.md
- Remove from this index
📅 Recent Updates
2025-10-14 (Documentation Consolidation)
- Created ML Infrastructure Guide (master index)
- Created 2 quick-start guides (Training, Tuning)
- Organized directory structure (7 categories)
- Added 200+ cross-references
- Identified 50+ archive candidates
2025-10-13 (Wave 160 Phase 4)
- ML training pipeline complete
- 19 agents, 4 models trained
- System 100% production ready
🎯 Next Steps
Phase 2 (Short-term - 1-2 weeks)
- Move files to category directories
- Create consolidated guides (API, Training, Deployment)
- Archive obsolete documentation
- Add more cross-references
Phase 3 (Medium-term - 1 month)
- Consolidate wave reports (488 → 20 phase summaries)
- Enhance troubleshooting guide
- Search optimization (keywords, metadata)
- Documentation tests (link validation)
📞 Support
Documentation Issues
- Missing documentation? Create GitHub issue with
docslabel - Broken links? Submit PR with fix
- Outdated content? File issue with current status
Technical Support
- Development: See Troubleshooting Guide
- Deployment: Review production runbooks
- ML Training: Consult training guides
- Performance: See performance benchmarks
Document Version: 1.0 Created: 2025-10-14 Last Updated: 2025-10-14 Maintained by: Foxhunt Development Team