docs: add stub detection implementation plan

9-task plan: 6 parallel audit agents, aggregate report, pre-commit lint rules, and TLI→CLI rename. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 01:42:48 +01:00
parent decf9ef957
commit 5f7a913a7b
1 changed files with 408 additions and 0 deletions
--- a/docs/plans/2026-02-24-stub-detection-plan.md
+++ b/docs/plans/2026-02-24-stub-detection-plan.md
@@ -0,0 +1,408 @@
+# Stub Detection & Prevention Implementation Plan
+
+> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
+
+**Goal:** Find and fix all fake/stub code hiding behind passing tests across the full foxhunt codebase, then add pre-commit lint rules to prevent new stubs.
+
+**Architecture:** 6 parallel audit agents scan crate groups for 14 stub patterns (8 grep-detectable, 6 semantic). Each agent produces a findings report, fixes what it can, and flags ambiguous stubs. A lint prevention layer is then added to the existing pre-commit hook.
+
+**Tech Stack:** Rust (cargo check), grep/ripgrep patterns, bash pre-commit hook, Claude Code Task agents
+
+**OOM Constraints:** RTX 3050 Ti 4GB GPU, ~4GB free RAM. All cargo builds must use `CARGO_BUILD_JOBS=1`. Never run builds concurrently with training.
+
+---
+
+## Task 1: ML Agent — Audit `ml/` and `ml-data/`
+
+**Files to scan:**
+- `ml/src/**/*.rs` (excluding `ml/tests/`, `ml/examples/`, `ml/benches/`)
+- `ml-data/src/**/*.rs`
+
+**Step 1: Launch ML audit agent**
+
+Spawn a Task agent (subagent_type: `researcher`) with this prompt:
+
+```
+Audit ml/src/ and ml-data/src/ for stub/fake code that compiles and passes tests but doesn't implement real logic.
+
+SCAN PROTOCOL:
+1. Grep for these patterns in src/ files ONLY (not tests/examples/benches):
+   - `return 0.0` or `return 1.0` or `return 100.0` or `return vec![]` or `return Ok(vec![])` in non-trivial functions
+   - `"placeholder"` or `"stub"` or `"fake"` or `"hardcoded"` or `"dummy"` in string literals
+   - `rand::` usage in production code paths (not test helpers)
+   - `todo!()` or `unimplemented!()`
+   - `sin(` or `cos(` in data retrieval/market functions
+   - `..Default::default()` in financial struct construction
+
+2. For each grep hit, READ the surrounding function (50 lines context).
+   Classify as:
+   - STUB: Returns hardcoded/fake data where real computation expected
+   - FALSE_POSITIVE: Legitimate usage (e.g., `return 0.0` as valid zero signal)
+   - ALREADY_FIXED: Was a stub but has been replaced with real logic
+
+3. For confirmed STUBs, classify severity:
+   - Critical: Affects model training, inference, or hyperopt results
+   - Important: Affects metrics, logging, or data quality
+   - Minor: Unused code path or deprecated feature
+
+4. Output a structured report as a markdown table:
+   | File | Line | Function | Pattern | Severity | Description | Can Auto-Fix? |
+
+Focus areas: DQN, PPO, TFT, Mamba2, TGGN, TLOB, Liquid, KAN, xLSTM, Diffusion model code.
+Known already-fixed stubs (skip these): Kelly sin-wave, PPO random actions, DQN/PPO validation.
+```
+
+**Step 2: Review ML agent findings**
+
+Read the agent's report. For each "Can Auto-Fix: Yes" finding, verify the fix is correct.
+
+**Step 3: Apply ML fixes**
+
+Edit files as identified by the agent. Use `SQLX_OFFLINE=true CARGO_BUILD_JOBS=1 cargo check -p ml --lib` to verify.
+
+**Step 4: Commit ML fixes**
+
+```bash
+git add ml/src/ ml-data/src/
+git commit -m "fix(ml): remove stub code identified by audit"
+```
+
+---
+
+## Task 2: Risk Agent — Audit `risk/` and `risk-data/`
+
+**Files to scan:**
+- `risk/src/**/*.rs`
+- `risk-data/src/**/*.rs`
+
+**Step 1: Launch Risk audit agent**
+
+Spawn a Task agent (subagent_type: `researcher`) with this prompt:
+
+```
+Audit risk/src/ and risk-data/src/ for stub/fake code.
+
+SCAN PROTOCOL: Same as ML Agent (grep + semantic analysis).
+
+Focus areas:
+- VaR calculations (monte_carlo.rs, parametric.rs, historical.rs)
+- Position sizing (position_tracker.rs, kelly calculations)
+- Kill switches and circuit breakers (safety/*.rs)
+- Risk engine (risk_engine.rs — 25 stub markers found by pre-scan)
+- Emergency response (emergency_response.rs)
+- Position limiter (position_limiter.rs — 6 matches found)
+
+Known already-fixed: Kelly position sizing stubs, risk metrics returning 0.0.
+
+Output: Markdown table with File | Line | Function | Pattern | Severity | Description | Can Auto-Fix?
+Critical = affects risk calculations, position limits, kill switch logic.
+```
+
+**Step 2–4:** Review, apply fixes, commit (same as Task 1 but with `cargo check -p risk --lib`).
+
+---
+
+## Task 3: Trading Agent — Audit `trading_engine/` and `adaptive-strategy/`
+
+**Files to scan:**
+- `trading_engine/src/**/*.rs`
+- `adaptive-strategy/src/**/*.rs`
+
+**Step 1: Launch Trading audit agent**
+
+Spawn a Task agent (subagent_type: `researcher`) with this prompt:
+
+```
+Audit trading_engine/src/ and adaptive-strategy/src/ for stub/fake code.
+
+SCAN PROTOCOL: Same grep + semantic protocol.
+
+Focus areas:
+- Order execution and FIX protocol (trading_engine)
+- Ensemble coordinator and model bridge (adaptive-strategy)
+- Microstructure analysis
+- Signal generation and position management
+- Any match arms that always return the same trading action
+
+Known already-fixed: ML orders always-BUY, fake risk metrics, EnsembleCoordinator initialized with real adapters.
+
+Output: Same markdown table format.
+Critical = affects order execution, signal generation, or ensemble decisions.
+```
+
+**Step 2–4:** Review, apply fixes, commit.
+
+---
+
+## Task 4: Services Agent — Audit `services/`
+
+**Files to scan:**
+- `services/*/src/**/*.rs` (all 8 microservices)
+
+**Step 1: Launch Services audit agent**
+
+Spawn a Task agent (subagent_type: `researcher`) with this prompt:
+
+```
+Audit services/*/src/ for stub/fake code across all 8 microservices:
+- trading_service, trading_agent_service, ml_training_service
+- backtesting_service, broker_gateway_service, data_acquisition_service
+- api_gateway
+
+SCAN PROTOCOL: Same grep + semantic protocol.
+
+Focus areas:
+- gRPC handler implementations (any returning Status::unimplemented or placeholder responses)
+- Service integration points (hardcoded mock responses)
+- Database queries (returning empty results instead of real queries)
+- Configuration loading (hardcoded values instead of reading config)
+
+Known already-fixed: Zero Status::unimplemented remaining (per MEMORY.md), debug JWT logging removed.
+
+Output: Same markdown table format.
+Critical = affects live service behavior or data flow between services.
+```
+
+**Step 2–4:** Review, apply fixes, commit.
+
+---
+
+## Task 5: Infra Agent — Audit `common/`, `config/`, `data/`, `database/`, `storage/`
+
+**Files to scan:**
+- `common/src/**/*.rs`
+- `config/src/**/*.rs`
+- `data/src/**/*.rs`
+- `database/src/**/*.rs` (24 stub markers found by pre-scan)
+- `storage/src/**/*.rs` (7 matches found)
+
+**Step 1: Launch Infra audit agent**
+
+Spawn a Task agent (subagent_type: `researcher`) with this prompt:
+
+```
+Audit common/src/, config/src/, data/src/, database/src/, storage/src/ for stub/fake code.
+
+SCAN PROTOCOL: Same grep + semantic protocol.
+
+Focus areas:
+- database/src/query.rs — 24 stub/placeholder matches (highest count in codebase)
+- storage/src/models.rs — 7 matches
+- config/src/ (symbol_config.rs, asset_classification.rs, vault.rs, schemas.rs, data_providers.rs)
+- data/src/providers/traits.rs — has unimplemented!
+- common/src/features/ (statistical.rs, microstructure.rs, technical_indicators.rs)
+- common/src/ml_strategy.rs — 4 matches
+
+Output: Same markdown table format.
+Critical = affects data loading, feature extraction, or shared utilities used by trading crates.
+```
+
+**Step 2–4:** Review, apply fixes, commit.
+
+---
+
+## Task 6: Frontend Agent — Audit `web-gateway/`, `tli/`, `backtesting/`
+
+**Files to scan:**
+- `web-gateway/src/**/*.rs`
+- `tli/src/**/*.rs`
+- `backtesting/src/**/*.rs`
+
+**Step 1: Launch Frontend audit agent**
+
+Spawn a Task agent (subagent_type: `researcher`) with this prompt:
+
+```
+Audit web-gateway/src/, tli/src/, backtesting/src/ for stub/fake code.
+
+SCAN PROTOCOL: Same grep + semantic protocol.
+
+Focus areas:
+- web-gateway/src/routes/auth.rs — 5 matches (known dev-only stub login)
+- tli/src/ (commands, auth/token_manager.rs)
+- backtesting/src/ (strategy_runner.rs — 3 matches, dbn_replay.rs — 3 matches, slippage.rs — 1 match)
+
+Note: tli/ is being kept as CLI only (future rename to cli/). Still audit for stubs but lower priority.
+web-gateway auth.rs login endpoint is a KNOWN dev stub — flag but don't auto-fix.
+
+Output: Same markdown table format.
+Critical = affects backtesting accuracy or web API correctness.
+```
+
+**Step 2–4:** Review, apply fixes, commit.
+
+---
+
+## Task 7: Aggregate Findings Report
+
+**Step 1: Collect all 6 agent reports**
+
+Read each agent's output and merge into a single findings file.
+
+**Step 2: Create aggregate report**
+
+Write to `docs/plans/2026-02-24-stub-audit-results.md`:
+
+```markdown
+# Stub Audit Results — 2026-02-24
+
+## Summary
+- Total findings: X
+- Critical: X (all fixed)
+- Important: X (Y fixed, Z deferred)
+- Minor: X
+- False positives: X
+
+## Findings by Crate Group
+[Merged tables from all 6 agents]
+
+## Deferred Items
+[Items that couldn't be auto-fixed with justification]
+```
+
+**Step 3: Commit aggregate report**
+
+```bash
+git add docs/plans/2026-02-24-stub-audit-results.md
+git commit -m "docs: add stub audit results report"
+```
+
+---
+
+## Task 8: Add Stub Detection to Pre-Commit Hook
+
+**Files:**
+- Modify: `.git/hooks/pre-commit` (lines 50-80, after the existing common issues section)
+
+**Step 1: Read current pre-commit hook**
+
+Read `.git/hooks/pre-commit` to find the insertion point (after the TODO/FIXME check block).
+
+**Step 2: Add stub detection checks**
+
+Insert after the existing TODO/FIXME check (around line 72), before the final "All pre-commit checks passed" message:
+
+```bash
+    # Check for stub patterns in staged src/ files (not tests/examples)
+    SRC_FILES=$(echo "$STAGED_FILES" | grep "/src/" || true)
+    if [ -n "$SRC_FILES" ]; then
+        # Hardcoded return values (common stub pattern)
+        STUB_RETURNS=$(echo "$SRC_FILES" | xargs grep -n "return 0\.0\b\|return 100\.0\|return 1\.0\b\|return vec!\[\]" 2>/dev/null | grep -v "// ok:" | grep -v "// legitimate" || true)
+        if [ -n "$STUB_RETURNS" ]; then
+            echo "⚠️  Warning: Possible stub return values in staged files:"
+            echo "$STUB_RETURNS" | head -5 | sed 's/^/    /'
+            echo "   Add '// ok: <reason>' comment to suppress if intentional"
+            echo ""
+        fi
+
+        # Production randomness (rand:: in src/ outside explicitly random code)
+        PROD_RAND=$(echo "$SRC_FILES" | xargs grep -n "rand::" 2>/dev/null | grep -v "// ok:" | grep -v "sampling\|exploration\|noise\|shuffle" || true)
+        if [ -n "$PROD_RAND" ]; then
+            echo "⚠️  Warning: rand:: usage in production code:"
+            echo "$PROD_RAND" | head -5 | sed 's/^/    /'
+            echo "   Verify this is intentional randomness, not a stub"
+            echo ""
+        fi
+
+        # Explicit stub markers
+        STUB_MARKERS=$(echo "$SRC_FILES" | xargs grep -n '"placeholder"\|"stub"\|"fake"\|"hardcoded"\|"dummy"' 2>/dev/null | grep -v "// ok:" || true)
+        if [ -n "$STUB_MARKERS" ]; then
+            echo "⚠️  Warning: Stub marker strings in production code:"
+            echo "$STUB_MARKERS" | head -5 | sed 's/^/    /'
+            echo ""
+        fi
+    fi
+```
+
+**Step 3: Verify hook works**
+
+```bash
+# Make a trivial change to test the hook
+echo "" >> ml/src/lib.rs
+git add ml/src/lib.rs
+git commit --dry-run 2>&1 | head -20
+git checkout -- ml/src/lib.rs
+```
+
+**Step 4: Commit hook update**
+
+The pre-commit hook is not tracked by git (it's in `.git/hooks/`). Document the hook content in a tracked script:
+
+```bash
+cp .git/hooks/pre-commit scripts/pre-commit-hook.sh
+git add scripts/pre-commit-hook.sh
+git commit -m "chore: add stub detection to pre-commit hook"
+```
+
+---
+
+## Task 9: TLI Rename to CLI
+
+**Files:**
+- Rename: `tli/` → `cli/`
+- Modify: `Cargo.toml` (workspace root)
+- Modify: `tli/Cargo.toml` → `cli/Cargo.toml` (package name)
+- Modify: any files referencing `tli` crate by name
+
+**Step 1: Find all references to tli**
+
+```bash
+grep -r '"tli"' Cargo.toml services/*/Cargo.toml
+grep -rn 'use tli::' --include='*.rs'
+grep -rn 'extern crate tli' --include='*.rs'
+```
+
+**Step 2: Rename directory**
+
+```bash
+git mv tli cli
+```
+
+**Step 3: Update workspace Cargo.toml**
+
+Change `"tli"` → `"cli"` in members list and `tli = { path = "tli" }` → `cli = { path = "cli" }`.
+
+**Step 4: Update cli/Cargo.toml**
+
+Change `name = "tli"` → `name = "cli"` (or `name = "foxhunt-cli"` to be more specific).
+
+**Step 5: Update any cross-crate references**
+
+Update `use tli::` imports if any exist outside the crate.
+
+**Step 6: Update scripts**
+
+```bash
+grep -rl 'tli' scripts/*.sh | head -10
+# Update script references from tli to cli
+```
+
+**Step 7: Verify compilation**
+
+```bash
+SQLX_OFFLINE=true CARGO_BUILD_JOBS=1 cargo check -p cli
+```
+
+**Step 8: Commit**
+
+```bash
+git add -A
+git commit -m "refactor: rename tli/ to cli/ (CLI-only usage)"
+```
+
+---
+
+## Execution Order
+
+Tasks 1-6 are **independent and can run in parallel** (6 audit agents).
+Task 7 depends on Tasks 1-6 (aggregate report).
+Task 8 is independent (pre-commit hook).
+Task 9 is independent (TLI rename).
+
+Recommended execution:
+1. Launch Tasks 1-6 in parallel (single message with 6 Task tool calls)
+2. After all 6 complete, run Task 7 (aggregate)
+3. Run Tasks 8 and 9 in parallel
+4. Final `SQLX_OFFLINE=true CARGO_BUILD_JOBS=1 cargo check --workspace` to verify everything
+
+**OOM safety:** Do NOT run cargo check while agents are scanning. Agents should use Grep/Read tools only, not Bash cargo commands. Only run cargo check after agent work is complete and agents have exited.