docs: add stub detection & prevention design

Approved design for full-codebase audit using 6 parallel agents
scanning 14 stub patterns, plus lint prevention in pre-commit hook.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-02-24 01:39:42 +01:00
parent 1d9e7fe64c
commit decf9ef957

View File

@@ -0,0 +1,108 @@
# Stub Detection & Prevention Design
**Date**: 2026-02-24
**Status**: Approved
**Goal**: Find and fix all fake/stub code hiding behind passing tests, then prevent new stubs from entering the codebase.
## Problem
The foxhunt codebase (37+ crates, ~200K lines of production Rust) contains stub code that compiles and passes tests but doesn't implement real logic. Examples already found and fixed:
- Kelly `get_historical_returns()` returned `sin(x)` instead of real market data
- `get_current_market_price()` hardcoded `100.0`
- PPO hyperopt used random actions with fake log_probs
- DQN/PPO validation computed market volatility instead of model performance
- ML orders always returned BUY regardless of prediction
- Risk metrics returned hardcoded `0.0` values
These stubs are dangerous because they **compile, pass tests, and look real at a glance**.
## Approach
Two-phase strategy:
1. **Agent swarm audit** — one-time deep scan across all crates, fix everything possible
2. **Lint prevention** — add pattern-based checks to pre-commit hook and CI
## Phase 1: Stub Pattern Taxonomy (14 patterns)
### Grep-Detectable
| # | Pattern | Detection |
|---|---------|-----------|
| 1 | Hardcoded returns | `return 100.0`, `return vec![]`, `return 0.0` in non-test code |
| 2 | Placeholder strings | `"TODO"`, `"placeholder"`, `"fixme"`, `"stub"` in string literals |
| 3 | Random in production | `rand::` usage in `src/` (not tests/benchmarks/examples) |
| 4 | Dead integration points | `unimplemented`, `Status::unimplemented`, `StatusCode::NOT_IMPLEMENTED` |
| 5 | Discarded real values | `_` prefixed variables from important return tuples |
| 6 | Config value shadowing | Function params that are never read (shadowed by hardcoded values) |
| 7 | Feature-gated stubs | `#[cfg(feature = "...")]` where default branch is a stub |
| 8 | Always-same match arms | All match arms returning the same variant |
### Semantic (Agent-Analyzed)
| # | Pattern | Detection |
|---|---------|-----------|
| 9 | Fake data generators | Math functions (`sin`, `cos`) in data retrieval methods |
| 10 | Wrong metric | Function name vs. what it actually computes (e.g., "validation" computing volatility) |
| 11 | Zeroed structs | `..Default::default()` where financial fields should be computed |
| 12 | Passthrough no-ops | Functions that return input unchanged in pipeline stages |
| 13 | Test-shaped production | Production code that only works for test scenarios |
| 14 | Stub with matching test | Test assertions that match function's hardcoded returns |
## Phase 1: Agent Swarm Architecture
6 parallel agents, each responsible for a crate group:
| Agent | Crates | Focus |
|-------|--------|-------|
| ML Agent | `ml/`, `ml-data/` | Model training, inference, hyperopt |
| Risk Agent | `risk/`, `risk-data/` | Position sizing, VaR, kill switches |
| Trading Agent | `trading_engine/`, `adaptive-strategy/` | Execution, ensemble, microstructure |
| Services Agent | `services/` (all 8) | gRPC endpoints, integration points |
| Infra Agent | `common/`, `config/`, `data/`, `database/`, `storage/` | Shared utilities, data loading |
| Frontend Agent | `web-gateway/`, `tli/`, `backtesting/` | API routes, CLI, backtesting |
### Agent Protocol
Each agent:
1. **Grep scan** — run all 8 grep-detectable patterns
2. **Semantic scan** — read flagged functions, assess if real stubs
3. **Classify** — severity (Critical/Important/Minor) + confidence (High/Medium/Low)
4. **Fix** — auto-fix where real implementation is clear
5. **Report** — structured findings list with file, line, severity, fix status
### Severity Classification
- **Critical**: Affects trading decisions, risk calculations, or model training (must fix)
- **Important**: Affects data quality, metrics accuracy, or integration correctness (should fix)
- **Minor**: Cosmetic stubs, unused code paths, or deprecated features (can defer)
## Phase 2: Lint Prevention Layer
Add to `scripts/pre-commit-check.sh`:
### New Checks
1. **Hardcoded returns** in non-test `.rs` files: `return 0\.0`, `return 100\.0`, `return vec![]`
2. **Stub markers**: `todo!`, `unimplemented!`, `placeholder`, `hardcoded`, `FIXME`
3. **Production randomness**: `rand::` in `src/` (not tests/benches/examples)
4. **Clippy**: `#[deny(clippy::unused_self)]` for passthrough no-ops
### Behavior
- **Pre-commit**: Warn on pattern matches (non-blocking)
- **CI**: Block on Critical patterns in `ml/`, `risk/`, `trading_engine/`, `services/`
## Follow-up: TLI Rename
Separate commit to rename `tli/``cli/`:
- Rename crate directory
- Update `Cargo.toml` workspace members
- Update `[[bin]]` target name
- Update imports/references in services
## Success Criteria
- Zero Critical stubs remaining in production code paths
- Pre-commit hook catches new stub patterns
- Each finding has a classification (severity + confidence) and either a fix or documented justification