From decf9ef9574c75fc65e93d4dea6a215e5352e960 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Tue, 24 Feb 2026 01:39:42 +0100 Subject: [PATCH] docs: add stub detection & prevention design Approved design for full-codebase audit using 6 parallel agents scanning 14 stub patterns, plus lint prevention in pre-commit hook. Co-Authored-By: Claude Opus 4.6 --- .../plans/2026-02-24-stub-detection-design.md | 108 ++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 docs/plans/2026-02-24-stub-detection-design.md diff --git a/docs/plans/2026-02-24-stub-detection-design.md b/docs/plans/2026-02-24-stub-detection-design.md new file mode 100644 index 000000000..5f680690f --- /dev/null +++ b/docs/plans/2026-02-24-stub-detection-design.md @@ -0,0 +1,108 @@ +# Stub Detection & Prevention Design + +**Date**: 2026-02-24 +**Status**: Approved +**Goal**: Find and fix all fake/stub code hiding behind passing tests, then prevent new stubs from entering the codebase. + +## Problem + +The foxhunt codebase (37+ crates, ~200K lines of production Rust) contains stub code that compiles and passes tests but doesn't implement real logic. Examples already found and fixed: + +- Kelly `get_historical_returns()` returned `sin(x)` instead of real market data +- `get_current_market_price()` hardcoded `100.0` +- PPO hyperopt used random actions with fake log_probs +- DQN/PPO validation computed market volatility instead of model performance +- ML orders always returned BUY regardless of prediction +- Risk metrics returned hardcoded `0.0` values + +These stubs are dangerous because they **compile, pass tests, and look real at a glance**. + +## Approach + +Two-phase strategy: +1. **Agent swarm audit** — one-time deep scan across all crates, fix everything possible +2. **Lint prevention** — add pattern-based checks to pre-commit hook and CI + +## Phase 1: Stub Pattern Taxonomy (14 patterns) + +### Grep-Detectable + +| # | Pattern | Detection | +|---|---------|-----------| +| 1 | Hardcoded returns | `return 100.0`, `return vec![]`, `return 0.0` in non-test code | +| 2 | Placeholder strings | `"TODO"`, `"placeholder"`, `"fixme"`, `"stub"` in string literals | +| 3 | Random in production | `rand::` usage in `src/` (not tests/benchmarks/examples) | +| 4 | Dead integration points | `unimplemented`, `Status::unimplemented`, `StatusCode::NOT_IMPLEMENTED` | +| 5 | Discarded real values | `_` prefixed variables from important return tuples | +| 6 | Config value shadowing | Function params that are never read (shadowed by hardcoded values) | +| 7 | Feature-gated stubs | `#[cfg(feature = "...")]` where default branch is a stub | +| 8 | Always-same match arms | All match arms returning the same variant | + +### Semantic (Agent-Analyzed) + +| # | Pattern | Detection | +|---|---------|-----------| +| 9 | Fake data generators | Math functions (`sin`, `cos`) in data retrieval methods | +| 10 | Wrong metric | Function name vs. what it actually computes (e.g., "validation" computing volatility) | +| 11 | Zeroed structs | `..Default::default()` where financial fields should be computed | +| 12 | Passthrough no-ops | Functions that return input unchanged in pipeline stages | +| 13 | Test-shaped production | Production code that only works for test scenarios | +| 14 | Stub with matching test | Test assertions that match function's hardcoded returns | + +## Phase 1: Agent Swarm Architecture + +6 parallel agents, each responsible for a crate group: + +| Agent | Crates | Focus | +|-------|--------|-------| +| ML Agent | `ml/`, `ml-data/` | Model training, inference, hyperopt | +| Risk Agent | `risk/`, `risk-data/` | Position sizing, VaR, kill switches | +| Trading Agent | `trading_engine/`, `adaptive-strategy/` | Execution, ensemble, microstructure | +| Services Agent | `services/` (all 8) | gRPC endpoints, integration points | +| Infra Agent | `common/`, `config/`, `data/`, `database/`, `storage/` | Shared utilities, data loading | +| Frontend Agent | `web-gateway/`, `tli/`, `backtesting/` | API routes, CLI, backtesting | + +### Agent Protocol + +Each agent: +1. **Grep scan** — run all 8 grep-detectable patterns +2. **Semantic scan** — read flagged functions, assess if real stubs +3. **Classify** — severity (Critical/Important/Minor) + confidence (High/Medium/Low) +4. **Fix** — auto-fix where real implementation is clear +5. **Report** — structured findings list with file, line, severity, fix status + +### Severity Classification + +- **Critical**: Affects trading decisions, risk calculations, or model training (must fix) +- **Important**: Affects data quality, metrics accuracy, or integration correctness (should fix) +- **Minor**: Cosmetic stubs, unused code paths, or deprecated features (can defer) + +## Phase 2: Lint Prevention Layer + +Add to `scripts/pre-commit-check.sh`: + +### New Checks + +1. **Hardcoded returns** in non-test `.rs` files: `return 0\.0`, `return 100\.0`, `return vec![]` +2. **Stub markers**: `todo!`, `unimplemented!`, `placeholder`, `hardcoded`, `FIXME` +3. **Production randomness**: `rand::` in `src/` (not tests/benches/examples) +4. **Clippy**: `#[deny(clippy::unused_self)]` for passthrough no-ops + +### Behavior + +- **Pre-commit**: Warn on pattern matches (non-blocking) +- **CI**: Block on Critical patterns in `ml/`, `risk/`, `trading_engine/`, `services/` + +## Follow-up: TLI Rename + +Separate commit to rename `tli/` → `cli/`: +- Rename crate directory +- Update `Cargo.toml` workspace members +- Update `[[bin]]` target name +- Update imports/references in services + +## Success Criteria + +- Zero Critical stubs remaining in production code paths +- Pre-commit hook catches new stub patterns +- Each finding has a classification (severity + confidence) and either a fix or documented justification