Commit Graph

22 Commits

Author SHA1 Message Date
jgrusewski
3e410f29e3 docs: Phase 4 production readiness design and implementation plan
16 tasks across 4 pillars: Safety & Crash Elimination, Trading
Correctness, Compliance & Code Quality, ML Pipeline Quality.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 20:21:01 +01:00
jgrusewski
86712a1216 docs: production readiness phase 3 implementation plan — 18 tasks
Bite-sized steps for each task with exact file paths, code snippets,
verification commands, and parallel execution strategy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 19:00:43 +01:00
jgrusewski
aa6669ede0 docs: production readiness phase 3 design — 18 tasks across 4 pillars
Safety, live trading, security, and observability gaps identified
from comprehensive codebase audit. Organized for parallel swarm execution.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 18:53:38 +01:00
jgrusewski
4069eb473c docs: backtesting vertical slice implementation plan
8-task TDD plan to bridge 8 disconnected layers in the backtesting
pipeline: DBN converter, replay engine, 51-dim feature wiring,
model loader, registry startup, position tracking, PnL tracking,
and end-to-end integration test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 15:30:00 +01:00
jgrusewski
4354098d11 docs: Phase 1-2 data pipeline and trading service implementation plan
13-task plan covering data pipeline fixes (Databento stream, data acquisition,
DBN uploader) and trading service core wiring (RiskEngine, MLEngine, VaR,
feature extraction, order matching, API gateway, Prometheus panics).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 15:25:17 +01:00
jgrusewski
30241e858d docs: backtesting vertical slice production readiness design
8-task plan to bridge disconnected backtesting pipeline layers:
DBN parser → feature extraction → ML inference → position tracking → PnL

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 15:21:22 +01:00
jgrusewski
e2a712593a docs: full system production readiness design (6-phase plan)
Covers data pipeline, trading service core, broker connectivity
(FIX 4.4 + TWS), execution algorithms, security/compliance, and
observability — based on comprehensive non-ML audit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 15:18:19 +01:00
jgrusewski
faa7b85cea docs: real-data ensemble backtest and hyperopt deploy design
Two-phase design:
A) Real 6E.FUT data → 4-model ensemble → PaperBroker → P&L metrics
B) Best hyperopt params → train → deploy to ensemble → ValidationHarness

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 15:04:58 +01:00
jgrusewski
2dd3326aec docs: ensemble real inference design and implementation plan
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 14:58:36 +01:00
jgrusewski
919773a741 docs: full-stack ML integration design (cleanup → ensemble → hyperopt → paper trading)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 13:19:17 +01:00
jgrusewski
fbf9c8a5aa docs: capital-ready validation roadmap implementation plan
14 TDD tasks across 4 sections: data integrity (TemporalGuard,
SlippageModel), walk-forward robustness (DegradationTracker, NoiseInjector,
SensitivityAnalyzer), risk enforcement (RiskAction, RiskEnforcer,
GraduatedRecovery, CorrelationMonitor), and E2E integration (pipeline
traits, RiskGate, operating modes, DriftResponder, observability).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 10:39:54 +01:00
jgrusewski
9641250382 docs: capital-ready validation roadmap design
Bottom-up design covering data integrity (leakage prevention, dynamic
slippage), walk-forward robustness (degradation curves, noise injection),
risk auto-enforcement (drawdown→position reduction, kill switch→liquidation),
and end-to-end pipeline integration with operating mode transitions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 10:33:54 +01:00
jgrusewski
4b51c300ad feat(validation): add PPO adapter for validation harness (MLP + LSTM)
Create PpoStrategy and PpoLstmStrategy implementing ValidatableStrategy
to run PPO through walk-forward validation with DSR, PBO, and permutation
tests. Both variants validated on real 6E.FUT data (29,937 bars, 15 folds).

Key implementation detail: LSTM hidden states are detached from the
computation graph after each step to prevent stack overflow from
unbounded graph growth across 30k+ sequential forward passes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 10:23:48 +01:00
jgrusewski
b4e1ce30e1 docs: add gradient accumulation implementation plan
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 21:59:14 +01:00
jgrusewski
1d6663027e docs: add gradient accumulation design
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 21:53:45 +01:00
jgrusewski
079f3192eb docs: add DQN pipeline production readiness design
4-phase plan: fix tests + OOM safety, inference path,
longer training, hyperopt end-to-end.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 20:49:41 +01:00
jgrusewski
ee601149de docs: add DQN training smoke test implementation plan
4-task plan: add trainer accessors, write smoke test with 6 core
assertions (loss convergence, finite losses, Q-value divergence,
checkpoint integrity, epsilon decay), add Sharpe comparison vs
untrained baseline, final verification.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 20:03:10 +01:00
jgrusewski
525355fe73 docs: add DQN training smoke test design
Single integration test that verifies the complete train → checkpoint →
validate pipeline works on real 6E.FUT data. 7 assertions covering
loss convergence, Q-value divergence, checkpoint round-trip, epsilon
decay, and Sharpe improvement vs untrained baseline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 19:56:20 +01:00
jgrusewski
6007f98c26 docs: Add DQN algorithm fix implementation plan (11 tasks)
Detailed TDD implementation plan for:
- Task 1: Bug fixes (NaN panic, Adam eps, dedup enum)
- Task 2: State dimension consolidation (→51)
- Task 3-5: CQL offline RL regularization
- Task 4,6-9: IQN distributional RL (replaces broken C51)
- Task 8: CVaR risk-aware action selection
- Task 10-11: Integration test and verification

Each task has exact file paths, code, test commands, and safety gates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 14:44:44 +01:00
jgrusewski
91af2b9334 docs: Add DQN algorithm fix & 2026 modernization design
Research-validated design for fixing critical DQN issues:
- CQL regularization for offline RL training on historical data
- IQN integration replacing broken C51 (Candle scatter_add bug)
- CVaR risk-aware action selection
- State dimension consolidation (51 is canonical)
- NaN panic fix, Adam epsilon fix per Rainbow paper

Verified against 20+ papers and 2026 SOTA.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 14:39:58 +01:00
jgrusewski
2b914728c6 docs: Add codebase deep clean implementation plan
16-task bottom-up execution plan with exact file paths, commands,
and safety gates. Covers: doc purge, dead code removal, module
consolidation, test reorganization, file splitting, and folder
restructuring.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 13:21:25 +01:00
jgrusewski
cec36d3072 docs: Add codebase deep clean design document
Comprehensive cleanup plan covering: documentation purge, dead code
removal (~4,846 lines across 11 files), module consolidation,
test reorganization, large file splitting, and folder restructuring.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 13:08:10 +01:00