Files
foxhunt/DATABASE_INITIALIZATION_AND_SETUP_ANALYSIS.md
jgrusewski e393a8af89 chore(cleanup): Cleanup Wave 3 - Archive reports, organize docs, fix security issues
## Summary
Third major cleanup wave after investigating 287 remaining root files.
Archived historical reports, organized documentation, removed regeneratable
artifacts, and fixed critical security issue.

## Files Cleaned (119 total)
- Archived: 78 files (7 WAVE reports + 71 summaries) → docs/archive/
- Archived: 7 build logs → docs/archive/build_logs/
- Organized: 10 markdown files → docs/guides/ + docs/checklists/
- Deleted: 17 test/coverage artifacts (regeneratable)
- Deleted: 7 empty/obsolete files (docker override, clippy baselines)
- Deleted: 3 large files (119MB - .venv, ppo_hyperopt_output.txt, backup)

## Space Recovered
- Total: ~120.7 MB
- Large files: 119.25 MB (.venv, ppo_hyperopt_output.txt)
- Archives: 1.04 MB (summaries + build logs)
- Test artifacts: 980 KB

## Security Fix (CRITICAL)
- Fixed: certs/security.env removed from git tracking (contained JWT secrets)
- Updated: .gitignore to prevent future tracking of sensitive cert files
- Removed: 4 files from git history (security.env, production.env.template, *.serial)

## Documentation Organization
- Created: docs/archive/ (wave_reports/, summaries/, build_logs/)
- Created: docs/guides/ (7 detailed implementation guides)
- Created: docs/checklists/ (3 operational checklists)
- Retained: 30 essential .md files in root (quick refs, CLAUDE.md)

## Investigation Reports Created
- MARKDOWN_ORGANIZATION_REPORT.md
- TXT_FILES_INVENTORY_AND_ARCHIVAL_PLAN.md
- ROOT_CONFIG_FILES_ANALYSIS_REPORT.md
- DOCKER_ROOT_FILES_ANALYSIS.md
- DATABASE_INITIALIZATION_AND_SETUP_ANALYSIS.md
- (6 additional investigation/index files)

## Cleanup Wave Progress
- Wave 1: 899 files deleted (1,071,884 lines)
- Wave 2: 543 files archived/deleted (~34GB)
- Wave 3: 119 files archived/deleted/organized (~121MB)
- Total: 1,561 files cleaned, ~35.1GB space recovered

## Result
Root directory: 287 files → ~180 files (excluding investigation reports)
Clean, organized, production-ready structure maintained.

Related: Second cleanup wave (previous commit)
2025-10-30 01:46:39 +01:00

26 KiB

Database Initialization & Setup Files Analysis

Date: 2025-10-30 Analysis Scope: Root directory files, Docker dependencies, organization recommendations


Executive Summary

The Foxhunt project has scattered initialization, configuration, and utility files across the root directory that could be organized for better maintainability. This report identifies which files MUST stay in root (Docker dependencies), which CAN be moved to appropriate directories, and provides a comprehensive organization plan.

Key Finding: Only 2 SQL files are strict Docker dependencies. The remaining 440+ text files in root are analysis artifacts that should be archived/cleaned up. Configuration files in .cargo/ can be rationalized from 7 variants down to 1-2 active versions.


Part 1: SQL Files Analysis

Root-Level SQL Files

Located in: /home/jgrusewski/Work/foxhunt/

File Size Purpose Docker Dependency Recommendation
init-db.sql 5.2K CRITICAL: Creates databases, users, permissions YES KEEP IN ROOT
init-db-dev.sql 659B DEV ONLY: Calls init-db.sql + applies migrations OPTIONAL Keep in root OR move to scripts/db/

Migration Files

Located in: /home/jgrusewski/Work/foxhunt/migrations/ (45 files, ~581K total)

Status: Properly organized in dedicated directory. SQLx uses these automatically.

migrations/
├── 001_trading_events.sql (30K) ✅
├── 002_risk_events.sql (37K) ✅
├── 003_audit_system.sql (40K) ✅
├── ...
├── 045_wave_d_regime_tracking.sql ✅
└── 046_batch_job_tracking.sql ✅

Assessment: No changes needed. These are correctly organized.

Test/Diagnostic SQL Files

Located in: /home/jgrusewski/Work/foxhunt/sql/ (5 files, 223K total)

sql/
├── PAPER_TRADING_DIAGNOSTIC_QUERIES.sql (5.6K)
├── paper_trading_schema.sql (18K) ← Development/documentation only
├── test_pg_performance.sql (2.9K)
├── test_stop_loss_debug.sql (1.1K)
└── trading_workload.sql (1.6K)

Assessment: These are test/diagnostic files, NOT initialization scripts.

Recommendation:

  • Move to docs/sql/ or scripts/db/diagnostics/
  • Add README explaining each file's purpose
  • These are useful for debugging but shouldn't clutter root

Part 2: Docker Dependencies Analysis

Docker-Compose Volume Mounts

From docker-compose.yml analysis, the following directories are REQUIRED in root:

Mount Point Source Service(s) Required Notes
./certs/ Root All (TLS) YES 🔒 Certificate management
./checkpoints/ Root ML Training YES 📦 Model checkpoints
./config/grafana/ Root Grafana YES 📊 Dashboard configs
./config/prometheus/ Root Prometheus YES 📈 Metrics rules
./models/ Root ML Training YES 🤖 Model storage
./optuna_studies/ Root ML Training YES 🔬 Hyperparameter study data
./test_data/ Root Backtesting YES 📊 Test data
./tuning_config.yaml Root ML Training YES ⚙️ Hyperparameter config

Finding: Docker REQUIRES these directories in root for volume mounts. DO NOT MOVE THEM.

Database Initialization Flow

Startup Sequence:
1. docker-compose up -d postgres
2. postgres service starts with default user 'foxhunt'
3. init-db.sql applied (IF NEEDED - check how/when)
4. cargo sqlx migrate run (auto-applies migrations/)

Current Issue: init-db.sql and init-db-dev.sql are NOT referenced in docker-compose.yml. They may be:

  • Legacy files from old setup
  • Applied via manual psql command
  • Only used for dev environment setup

Recommendation: Document when/how these are used or deprecate them if migrations are sufficient.


Part 3: Root Directory Clutter Analysis

.cargo/ Configuration Files

Located in: /home/jgrusewski/Work/foxhunt/.cargo/

.cargo/
├── config.toml (57 lines) ✅ ACTIVE - RunPod + Production
├── config.toml.coverage (50 lines) - Coverage runs
├── config.toml.lld (52 lines) - LLD linker variant
├── config.toml.optimized (110 lines) - Optimized builds
├── config.toml.original (50 lines) - Backup version
└── config.toml.runpod (57 lines) - RunPod-specific

Current State: 6 variants (376 lines total), mostly duplicates with minor differences.

Assessment:

  • .cargo/config.toml must stay in root (Cargo requirement)
  • ⚠️ Variants should be consolidated or documented
  • .cargo/ structure is NON-NEGOTIABLE (Cargo standard location)

Recommendation:

  1. Keep .cargo/config.toml as main config
  2. Document feature flags in Cargo.toml instead of variant files
  3. Use environment variables for build variations:
    RUSTFLAGS='...' cargo build --release
    
  4. Archive unused variants to docs/cargo-configs-archive/

Part 4: Test & Analysis Artifacts

Root-Level Text/Log Files (440+ files)

Finding: The root contains hundreds of test result/analysis files:

Examples:
- coverage_api_gateway.txt
- WAVE112_AGENT10_SUMMARY.txt
- DB_LOAD_TEST_RESULTS_20251012_012011.txt
- ML_TRAINING_SERVICE_ARCHITECTURE_DIAGRAM.txt
- RUNPOD_DEPLOYMENT_STATUS.txt
- (and 400+ more...)

Size Impact: ~5GB of root directory clutter (mostly in target/, but also in loose text files)

Recommendation:

  1. Archive all .txt files to artifacts/ directory (structure by date):

    artifacts/
    ├── 2025-10-30/
    │   ├── WAVE112_*.txt
    │   ├── DB_LOAD_TEST_*.txt
    │   └── coverage_*.txt
    ├── 2025-10-29/
    ├── 2025-10-28/
    └── archive-older/
    
  2. Keep only active documentation:

    • CLAUDE.md (system instructions)
    • .gitlab-ci.yml (CI/CD)
    • Dockerfile.foxhunt-build (production build)
    • docker-compose.yml (local dev)
    • Makefile (build tasks)
    • justfile ⚠️ (alternative to Makefile - consolidate?)
  3. Remove deployment logs from root (keep in docs/deployment-logs/)


Part 5: Configuration Files Organization

Root-Level Configuration Files

File Size Purpose Keep Recommendation
pytest.ini <1K Python test config YES Could move to config/pytest.ini
tarpaulin.toml <1K Coverage config YES Could move to config/coverage/
rustfmt.toml <1K Code formatting YES Must stay in root (Rust standard)
clippy.toml <1K Clippy lints YES Must stay in root (Rust standard)
tuning_config.yaml <1K ML hyperparameter tuning YES Docker mounts from root
TFT_TUNING_CONFIG_RECOMMENDED.yaml <1K Backup config MAYBE Archive to config/ml/archive/
tuning_config_ppo_comprehensive.yaml <1K PPO config MAYBE Archive to config/ml/archive/
mutants.toml <1K Mutation testing YES Could move to config/testing/
Makefile 9K Build automation YES Keep in root (standard)
justfile ? Alternative build tool CHECK Consolidate with Makefile?

Requirements Files

Located in root:

  • requirements.txt - Python dependencies
  • requirements-dev.txt - Dev dependencies
  • requirements-test.txt - Test dependencies

Current Location: Root (works but could be organized)

Recommendation:

  • Create python/requirements/ directory:
    python/requirements/
    ├── requirements.txt → symlink to root
    ├── requirements-dev.txt → symlink to root
    └── requirements-test.txt → symlink to root
    
    OR move to config/python/requirements/ and update references

Part 6: Script Organization Status

Located in: /home/jgrusewski/Work/foxhunt/scripts/ (150+ files, 88M)

Current Structure:

scripts/
├── *.sh (active scripts)
├── archive/ (old scripts)
├── deployment/ (deployment scripts)
├── README.md
└── .venv/ (Python venv)

Assessment: Well organized. The scripts/archive/ pattern is good.

Recommendations:

  1. Move .venv/ out of scripts (takes 88M):
    .venv/ (in root, or use venv/ symlink)
    
  2. Document scripts/README.md for discovering active scripts
  3. Add scripts/DEPRECATED.md listing archived scripts

Part 7: Complete Organization Plan

Phase 1: IMMEDIATE (Critical for Docker)

Files that MUST stay in root:

  • init-db.sql - Database initialization
  • init-db-dev.sql - Dev variant
  • docker-compose.yml - Service orchestration
  • docker-compose.override.yml - Dev overrides
  • Dockerfile.foxhunt-build - Production build
  • Cargo.lock - Dependency lock
  • Cargo.toml - Workspace manifest
  • .cargo/ directory (Cargo standard)
  • .gitlab-ci.yml - CI/CD pipeline
  • Makefile - Build targets
  • Makefile - Build automation
  • rustfmt.toml - Code formatting (Rust standard)
  • clippy.toml - Linting (Rust standard)
  • CLAUDE.md - System documentation
  • .gitignore - Git configuration
  • .env files - Environment config (gitignored)

Directories that must stay in root (Docker volumes):

  • certs/ - TLS certificates
  • checkpoints/ - Model checkpoints
  • config/ - Service configuration
  • models/ - ML models
  • optuna_studies/ - Hyperparameter study data
  • test_data/ - Test datasets
  • migrations/ - Database migrations

Phase 2: SHOULD MOVE (if not Docker-dependent)

Can move with configuration updates:

# Move test SQL diagnostics
mkdir -p docs/sql/diagnostics
mv sql/* docs/sql/diagnostics/
rmdir sql

# Move test/analysis artifacts
mkdir -p artifacts/$(date +%Y-%m-%d)
mv *.txt artifacts/$(date +%Y-%m-%d)/
mv WAVE*.txt artifacts/$(date +%Y-%m-%d)/

# Move coverage configs
mkdir -p config/testing
mv tarpaulin.toml config/testing/
mv pytest.ini config/testing/

# Consolidate tuning configs
mkdir -p config/ml/tuning
mv TFT_TUNING_CONFIG_RECOMMENDED.yaml config/ml/tuning/
mv tuning_config_ppo_comprehensive.yaml config/ml/tuning/
# Keep tuning_config.yaml in root (Docker mount)

# Archive .cargo variants
mkdir -p docs/cargo-configs-archive
mv .cargo/config.toml.* docs/cargo-configs-archive/
# Keep .cargo/config.toml as active config

Phase 3: OPTIONAL CONSOLIDATION

Optional improvements (no Docker impact):

# Move venv out of scripts
mv scripts/.venv .venv  # Or use symlink

# Consolidate Python requirements
mkdir -p config/python
mv requirements*.txt config/python/
# Update references in CI/CD and docs

# Move justfile to config if not actively used
# OR consolidate with Makefile targets

Part 8: Directory Structure Recommendation

Proposed Final Root Structure

foxhunt/
├── .cargo/                          # ✅ Cargo configuration (REQUIRED)
│   ├── config.toml                  # Active config
│   └── (no variants)
├── .github/                         # ✅ GitHub workflows (if using)
├── .gitlab/                         # ✅ GitLab CI (if using)
├── artifacts/                       # 🆕 Analysis/test results
│   ├── 2025-10-30/                  # Organized by date
│   └── archive-older/
├── certs/                           # ✅ TLS certificates (Docker mount)
├── checkpoints/                     # ✅ Model checkpoints (Docker mount)
├── config/                          # ✅ Service configurations (Docker mount)
│   ├── grafana/
│   ├── prometheus/
│   ├── ml/
│   │   └── tuning/                  # 🆕 Tuning configs
│   ├── testing/                     # 🆕 Coverage/test configs
│   └── python/                      # 🆕 Python configs
├── data/                            # ✅ Existing data directory
├── docs/                            # ✅ Documentation (keep)
│   ├── sql/                         # 🆕 SQL diagnostics
│   │   └── diagnostics/
│   ├── cargo-configs-archive/       # 🆕 Archived configs
│   └── deployment-logs/             # 🆕 Deployment history
├── migrations/                      # ✅ Database migrations
├── ml/                              # ✅ ML models/code
├── models/                          # ✅ Model storage (Docker mount)
├── optuna_studies/                  # ✅ Hyperparameter data (Docker mount)
├── scripts/                         # ✅ Automation scripts
├── services/                        # ✅ Microservices
├── test_data/                       # ✅ Test datasets (Docker mount)
├── tests/                           # ✅ Test suite
├── tli/                             # ✅ Terminal client
├── trading_engine/                  # ✅ Trading core
├── common/                          # ✅ Shared code
├── risk/                            # ✅ Risk management
├── .cargo/config.toml.* (ARCHIVED)  # Moved to docs/cargo-configs-archive/
├── .dockerignore                    # ✅ Docker build
├── .env.example                     # ✅ Environment template
├── .gitignore                       # ✅ Git configuration
├── .gitlab-ci.yml                   # ✅ CI/CD
├── CARGO.toml                       # ✅ Workspace manifest
├── CARGO.lock                       # ✅ Dependency lock
├── CLAUDE.md                        # ✅ System documentation
├── clippy.toml                      # ✅ Linting (Rust standard, must be here)
├── docker-compose.yml               # ✅ Local dev services
├── docker-compose.override.yml      # ✅ Dev overrides
├── Dockerfile.foxhunt-build         # ✅ Production build
├── init-db.sql                      # ✅ Database setup
├── init-db-dev.sql                  # ✅ Dev database setup
├── justfile                         # ? (consolidate with Makefile)
├── Makefile                         # ✅ Build automation
├── mutants.toml                     # ⚠️ Can move to config/testing/
├── pytest.ini                       # ⚠️ Can move to config/testing/
├── requirements*.txt                # ⚠️ Can move to config/python/
├── rustfmt.toml                     # ✅ Code formatting (Rust standard, must be here)
├── sqlx-data.json                   # ✅ SQLx offline mode
├── tarpaulin.toml                   # ⚠️ Can move to config/testing/
├── tuning_config.yaml               # ✅ ML config (Docker mount)
├── tuning_config_ppo_comprehensive.yaml (ARCHIVED)
├── TFT_TUNING_CONFIG_RECOMMENDED.yaml (ARCHIVED)
│
├── .venv/                           # 🆕 Move out of scripts/ (88M)
├── target/                          # ✅ Build output (gitignored)
└── (CLEANUP) *.txt files            # Move to artifacts/

Legend:
✅ = Required in root (Docker or Rust standard)
⚠️ = Can move with config updates
🆕 = New directory to create
? = Consolidate/deprecate

Part 9: Migration Checklist

Step 1: Create New Directories

mkdir -p artifacts/$(date +%Y-%m-%d)
mkdir -p config/testing
mkdir -p config/ml/tuning
mkdir -p config/python
mkdir -p docs/sql/diagnostics
mkdir -p docs/cargo-configs-archive
mkdir -p docs/deployment-logs

Step 2: Archive Analysis Artifacts

# Move all .txt analysis files to archive
find . -maxdepth 1 -name "*.txt" -type f \
  -exec mv {} artifacts/$(date +%Y-%m-%d)/ \;

Step 3: Reorganize SQL Files

mv sql/PAPER_TRADING_DIAGNOSTIC_QUERIES.sql docs/sql/diagnostics/
mv sql/paper_trading_schema.sql docs/sql/diagnostics/
mv sql/test_pg_performance.sql docs/sql/diagnostics/
mv sql/test_stop_loss_debug.sql docs/sql/diagnostics/
mv sql/trading_workload.sql docs/sql/diagnostics/
rmdir sql

Step 4: Consolidate Cargo Configs

mkdir -p docs/cargo-configs-archive
mv .cargo/config.toml.* docs/cargo-configs-archive/
# Keep .cargo/config.toml as the active config

Step 5: Update CI/CD References

  • Update .gitlab-ci.yml to reference new config paths
  • Update Makefile targets that reference moved files
  • Update documentation paths

Step 6: Document Changes

  • Create ORGANIZATION_MIGRATION_LOG.md explaining changes
  • Update CLAUDE.md with new file locations
  • Update scripts/README.md for any script path changes

Part 10: Files to Keep/Archive Summary

KEEP IN ROOT (NO CHANGES)

Type Count Files Reason
Rust Config 2 rustfmt.toml, clippy.toml Rust standard locations
Cargo 3 Cargo.toml, Cargo.lock, .cargo/ Cargo requirement
Docker 4 docker-compose.yml, Dockerfile.foxhunt-build, .dockerignore, init-db.sql Docker dependencies
Source Code 7 services/, ml/, trading_engine/, tests/, migrations/, common/, tli/ Core project
Config Mounts 8 certs/, config/, checkpoints/, models/, optuna_studies/, test_data/, tuning_config.yaml Docker volumes
Build/CI 3 .gitlab-ci.yml, Makefile, Cargo.lock Build automation
Docs 1 CLAUDE.md System documentation
Total 28

ARCHIVE OR MOVE (SAFE)

Type Files New Location Notes
.cargo variants config.toml.* (6 files) docs/cargo-configs-archive/ Keep main config
Test configs pytest.ini, tarpaulin.toml config/testing/ Optional - works in root
SQL diagnostics sql/*.sql (5 files) docs/sql/diagnostics/ Dev-only files
ML configs tuning_config_ppo_comprehensive.yaml, TFT_TUNING_CONFIG_RECOMMENDED.yaml config/ml/tuning/archive/ Backups only
Test results *.txt (400+ files) artifacts/YYYY-MM-DD/ Analysis artifacts
Total 414+

DEPRECATE OR CONSOLIDATE

File Status Action
justfile Optional Consolidate targets with Makefile OR document when to use
init-db-dev.sql Legacy? Verify if still used; document or deprecate
docker-compose.override.yml.disabled Disabled Move to docs/archive/
deploy.sh (in root) Duplicate? Consolidate with scripts/ versions

Part 11: Docker Impact Assessment

Files CANNOT Be Moved (Docker-Critical)

These are referenced in docker-compose.yml volume mounts:

volumes:
  - ./certs:/tmp/foxhunt/certs:ro                                    # ✅ CRITICAL
  - ./checkpoints:/tmp/foxhunt/checkpoints                           # ✅ CRITICAL
  - ./config/grafana/dashboards:/var/lib/grafana/dashboards:ro       # ✅ CRITICAL
  - ./config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml # ✅ CRITICAL
  - ./models:/tmp/foxhunt/models                                     # ✅ CRITICAL
  - ./optuna_studies:/app/optuna_studies                             # ✅ CRITICAL
  - ./test_data:/workspace/test_data:ro                              # ✅ CRITICAL
  - ./tuning_config.yaml:/app/tuning_config.yaml:ro                  # ✅ CRITICAL

If you move these directories, Docker will fail to mount them!

Safe to Move (No Docker References)

- artifacts/ (new - for analysis files)
- docs/sql/diagnostics/ (SQL test files)
- docs/cargo-configs-archive/ (.cargo variants)
- config/testing/ (pytest, tarpaulin configs)
- config/ml/tuning/archive/ (backup tuning configs)

Part 12: Recommendations Summary

Priority 1: IMMEDIATE CLEANUP (0 risk)

  1. Archive .txt analysis filesartifacts/YYYY-MM-DD/

    • No code changes needed
    • Reduces root clutter from ~500 files to ~50
    • Impact: 0 risk
  2. Move SQL diagnosticsdocs/sql/diagnostics/

    • Files never executed by Docker
    • Just documentation
    • Impact: 0 risk
  3. Archive .cargo/config.toml.* variantsdocs/cargo-configs-archive/

    • Keep main config.toml active
    • Document how to use variants
    • Impact: 0 risk

Priority 2: OPTIONAL IMPROVEMENTS (low risk)

  1. Consolidate testing configsconfig/testing/

    • Move pytest.ini, tarpaulin.toml, mutants.toml
    • Update CI/CD references
    • Impact: low risk (update references)
  2. Move archived tuning configsconfig/ml/tuning/archive/

    • Move tuning_config_ppo_comprehensive.yaml, TFT_TUNING_CONFIG_RECOMMENDED.yaml
    • Keep main tuning_config.yaml in root (Docker mount)
    • Impact: low risk (documentation only)
  3. Move Python requirementsconfig/python/

    • Create symlinks or update references
    • Impact: medium risk (CI/CD updates)

Priority 3: CONSOLIDATION (future)

  1. Consolidate Makefile + justfile

    • Choose one build tool
    • Document in CONTRIBUTING.md
    • Impact: low-medium risk (developer workflow)
  2. Document/deprecate init-db.sql variants

    • Verify if init-db-dev.sql is still used
    • Confirm migrations are sufficient
    • Impact: low risk (documentation)
  3. Move .venv/ out of scripts/

    • Currently in scripts/.venv/ (takes 88M)
    • Move to .venv/ in root or symlink
    • Impact: low risk (venv management)

Part 13: Cost-Benefit Analysis

Cleanup Benefits

Action Effort Benefit Priority
Archive .txt files 5 min 90% root clutter reduction NOW
Move SQL diagnostics 10 min Organize documentation NOW
Archive .cargo variants 10 min Simplify config management NOW
Move testing configs 20 min Better organization SOON
Consolidate build tools 1-2 hrs Reduce developer confusion 📅 FUTURE
Move Python requirements 30 min Better structure 📅 FUTURE

Risk Assessment

Action Risk Level Docker Impact CI/CD Impact Notes
Archive .txt ZERO None None Safe cleanup
Move SQL diagnostics LOW None None Not used by Docker
Archive .cargo variants ZERO None None Keep main config
Move test configs LOW None Update references CI/CD changes needed
Move Python requirements MEDIUM None Update pip paths Requires CI/CD updates
Move tuning configs LOW None Update docs Main config stays in root

Part 14: Implementation Timeline

Week 1: IMMEDIATE CLEANUP

  • Archive all .txt analysis files
  • Move SQL diagnostics
  • Archive .cargo variants
  • Create artifacts/ directory structure
  • Effort: ~1-2 hours
  • Risk: ZERO

Week 2: OPTIONAL IMPROVEMENTS

  • Move testing configs (pytest, tarpaulin)
  • Move tuning config backups
  • Update CI/CD references
  • Update documentation
  • Effort: ~3-4 hours
  • Risk: LOW

Week 3+: FUTURE CONSOLIDATION

  • Consolidate Makefile + justfile
  • Document/deprecate init-db files
  • Move .venv/ optimization
  • Update CLAUDE.md with new structure
  • Effort: ~2-3 days
  • Risk: LOW-MEDIUM

Part 15: Files Reference List

DO NOT MOVE (Docker-Critical)

.cargo/                         # Cargo standard location
.gitlab-ci.yml                  # CI/CD pipeline
.gitignore                      # Git config
.env                            # Environment (gitignored)
certs/                          # TLS (Docker mount)
checkpoints/                    # Models (Docker mount)
CLAUDE.md                       # System docs
clippy.toml                     # Linting (Rust standard)
common/                         # Source code
config/                         # Service config (Docker mount)
Cargo.lock                      # Dependency lock
Cargo.toml                      # Workspace manifest
data/                           # Data directory
docker-compose.yml              # Services
docker-compose.override.yml     # Dev overrides
Dockerfile.foxhunt-build        # Production build
init-db.sql                     # Database setup
Makefile                        # Build automation
migrations/                     # Database migrations
ml/                             # ML models/code
models/                         # Model storage (Docker mount)
optuna_studies/                 # Hyperparameter data (Docker mount)
risk/                           # Risk management
rustfmt.toml                    # Code formatting (Rust standard)
scripts/                        # Automation
services/                       # Microservices
sqlx-data.json                  # SQLx offline
test_data/                      # Test data (Docker mount)
tests/                          # Test suite
tli/                            # Terminal client
trading_engine/                 # Trading core
tuning_config.yaml              # ML config (Docker mount)

CAN MOVE (Safe)

.cargo/config.toml.*            → docs/cargo-configs-archive/
init-db-dev.sql                 → docs/db/ or deprecate
mutants.toml                    → config/testing/
pytest.ini                      → config/testing/
requirements*.txt               → config/python/
sql/                            → docs/sql/diagnostics/
tarpaulin.toml                  → config/testing/
tuning_config_ppo_comprehensive.yaml  → config/ml/tuning/archive/
TFT_TUNING_CONFIG_RECOMMENDED.yaml    → config/ml/tuning/archive/
*.txt (analysis files)          → artifacts/YYYY-MM-DD/

Conclusion

Key Takeaway: The Foxhunt project has excellent separation of concerns with code organized by purpose (services, ml, trading_engine, etc.). The only organization issue is accumulated analysis artifacts in root that should be archived.

Recommended Action:

  1. Week 1: Archive 400+ .txt files and reorganize SQL diagnostics (1-2 hours, zero risk)
  2. Week 2: Move optional configs and update CI/CD (3-4 hours, low risk)
  3. Future: Consolidate build tools and optimize venv placement

Docker Impact: ZERO - All critical Docker dependencies are identified and stay in root.


Report Generated: 2025-10-30 Analysis Status: COMPLETE Recommendation: Implement Phase 1 (immediate cleanup) now; Phase 2 (optional) within 2 weeks.