jgrusewski 3c61ee52ce plan(dqn): Thompson sampling rollout — Plans A+B+C+D (4 sequential phases) ...

Implementation plans for the distributional-RL aggregation spec
(docs/superpowers/specs/2026-04-26-distributional-rl-aggregation-design.md).

Plan A — Phase 0: TDD hypothesis verification (8 tasks)
  - Standalone GPU test kernel: sample_c51_inverse_cdf,
    sample_iqn_quantile_interp, compute_e_c51, compute_e_iqn,
    thompson_direction_test, argmax_eq_test
  - 6 unit tests (5 GPU + 1 CPU synthetic edge)
  - GPU integration on converged checkpoint (Test 0.F)

Plan B — Phase 1: existing-lever audit (8 tasks)
  - 6 unit tests covering B.2/CF/PopArt/Q-target audit fixture
  - Exit gate: all 6 PASS = no reward-shaping bug; Phase 2 unblocked

Plan C — Phase 2: Thompson sampling integration (11 tasks)
  - Direction branch of experience_action_select rewritten:
    eps-greedy + Boltzmann → Thompson (training) + argmax E[Q] (eval)
  - C51 + IQN buffers wired to gpu_dqn_trainer + gpu_backtest_evaluator
  - train_active_frac HEALTH_DIAG instrumentation
  - 4 GPU-direct tests against production kernel
  - Aggregation Contract table + memory pearl

Plan D — Phase 3: long verification + dead-code cleanup (8 tasks)
  - Test 3.A: regression anchor against original C51 Flat bias
  - L4: 1 seed × 6 folds × 30 epoch (~1 hour)
  - L5: 5 seed × 6 fold matrix per Plan 5 Task 5 (Tier 1+2+3 PASS)
  - Direction-only eps_dir adaptive boost + Boltzmann tau-floor cleanup
    (per feedback_no_partial_refactor.md)
  - nsys regression check; final architecture/spec footer

Plans gated sequentially: each phase exit gate must pass before next.

2026-04-27 08:10:42 +02:00

..

2026-03-10-zero-cpu-hotpath.md

docs: add zero-CPU DQN training hot path implementation plan

2026-03-10 15:49:40 +01:00

2026-03-11-dqn-factored-actions.md

feat(dqn): enable Branching DQN with 45 factored actions (5×3×3)

2026-03-11 22:00:13 +01:00

2026-03-13-gpu-test-workflow.md

fix(plan): reviewer round 3 — tggn/tgnn module mapping, exit code, canonical imports

2026-03-13 11:51:58 +01:00

2026-03-14-kanidm-netbird-sso.md

docs: add Kanidm SSO + NetBird mesh design spec and implementation plan

2026-03-14 19:41:40 +01:00

2026-03-15-fused-cuda-training.md

refactor(ml): delete mixed_precision module — BF16 unconditional on CUDA

2026-03-16 16:11:48 +01:00

2026-03-16-gpu-iql-her-porting.md

refactor(cuda): eliminate all CPU fallbacks — CUDA mandatory across ML stack

2026-03-16 21:01:28 +01:00

2026-03-17-candle-hard-refactor.md

refactor(cuda): eliminate candle from ml-core, ml-ppo, and 4 thin crates

2026-03-17 22:27:56 +01:00

2026-03-17-cuda-candle-elimination.md

feat(cuda): complete Candle elimination + cudarc 0.19.3 upgrade

2026-03-17 15:13:04 +01:00

2026-03-18-full-gpu-forward-passes.md

audit: annotate all remaining to_vec/memcpy_dtoh — categorized 158 sites

2026-03-18 19:36:41 +01:00

2026-03-18-gpu-dedup-consolidation.md

fix(ml-supervised): resolve all 104 compile errors — clean build

2026-03-18 07:26:56 +01:00

2026-03-18-gpu-resident-pipeline.md

feat: WORKSPACE COMPILES — zero candle, zero errors, GPU-native

2026-03-18 13:31:35 +01:00

2026-03-21-h100-epoch-optimization-phase1.md

docs: Phase 1 implementation plan — eliminate PER CPU roundtrips

2026-03-21 10:26:11 +01:00

2026-03-21-h100-epoch-optimization-phase2.md

docs: Phase 2 plan — cuBLAS batched GEMM + kernel fusion (10.7ms → <1ms/step)

2026-03-21 11:23:09 +01:00

2026-03-21-h100-epoch-optimization-phase3.md

perf: cuBLAS SGEMM pipeline + dead code elimination — 37s → 86ms/epoch (430x)

2026-03-21 15:33:00 +01:00

2026-03-21-h100-epoch-optimization-phase4.md

perf: keep GPU training data resident across epochs — init 73ms → 0ms

2026-03-21 16:29:32 +01:00

2026-03-21-training-profiles.md

docs: training profile configuration system — spec + plan

2026-03-21 23:22:58 +01:00

2026-03-22-gpu-composite-reward.md

plan: GPU composite reward implementation — 7 tasks

2026-03-22 20:35:57 +01:00

2026-03-22-gpu-segment-tree.md

docs: GPU segment tree spec + plan for O(log n) PER sampling

2026-03-22 00:39:34 +01:00

2026-03-23-9-exposure-dual-distributional.md

plan: 9-exposure action space + dual C51/IQN distributional architecture

2026-03-23 09:03:19 +01:00

2026-03-23-stable-distributional-loss.md

plan: three-layer stable distributional loss architecture

2026-03-23 20:42:08 +01:00

2026-03-24-c51-loss-convergence.md

feat: comprehensive DQN training pipeline overhaul — 16 bug fixes, MSE warmup, financial metrics

2026-03-24 21:48:38 +01:00

2026-03-24-financial-metrics-pipeline.md

fix: segment-based trade lifecycle + reversal P&L computation

2026-03-24 23:57:04 +01:00

2026-03-24-master-trading-agent.md

docs: master plan — maximize all 25 DQN features for profitable ES trading

2026-03-24 22:00:45 +01:00

2026-03-24-remaining-gpu-features.md

fix: segment-based trade lifecycle + reversal P&L computation

2026-03-24 23:57:04 +01:00

2026-03-25-boundary-trade-counting.md

fix: backtest evaluator — correct annualization, window sizing, objective calibration

2026-03-25 19:43:26 +01:00

2026-03-25-config-unification.md

docs: config unification plan — single source of truth, 8 tasks

2026-03-25 09:01:12 +01:00

2026-03-25-eliminate-nvrtc.md

docs: plan to eliminate NVRTC — precompile all 25 CUDA kernels at build time

2026-03-25 23:58:30 +01:00

2026-03-25-reward-v6-fixes.md

docs: reward v6 fixes plan

2026-03-25 01:49:53 +01:00

2026-03-25-trade-lifecycle-fix.md

docs: trade lifecycle fix plan — address review feedback

2026-03-25 23:02:39 +01:00

2026-03-26-eval-cleanup.md

docs: eval cleanup spec + plan — remove dead 5-action path, fix train/eval mismatch

2026-03-26 00:43:58 +01:00

2026-03-26-final-fixes-before-h100.md

docs: final fixes plan — 6 issues blocking H100 deployment

2026-03-26 01:06:28 +01:00

2026-03-27-bf16-native-completion.md

docs: BF16 native completion plan — zero casts, zero nvrtc, zero GpuTensor

2026-03-27 23:39:43 +01:00

2026-03-27-bug-fixes.md

feat: gradient stability + 9-agent audit bug fixes

2026-03-27 19:52:44 +01:00

2026-03-27-full-bf16-rewrite.md

docs: full BF16 rewrite spec + plan — zero F32 on GPU

2026-03-27 20:42:37 +01:00

2026-03-27-gpu-hot-path-performance.md

feat: gradient stability + 9-agent audit bug fixes

2026-03-27 19:52:44 +01:00

2026-03-27-gradient-stability.md

feat: gradient stability + 9-agent audit bug fixes

2026-03-27 19:52:44 +01:00

2026-03-27-hive-findings-fixes.md

fix: hive findings — capital floor, dueling centering, metrics, CVaR objective

2026-03-27 07:56:50 +01:00

2026-03-27-trade-physics-refactor.md

fix: shared trade physics, IQN arg swap, trailing stop, win_rate

2026-03-27 00:47:34 +01:00

2026-03-27-training-kernel-unification.md

refactor: training kernel fully unified with trade_physics.cuh

2026-03-27 08:16:51 +01:00

2026-03-29-drawdown-aware-training.md

feat(training): continuous drawdown penalty — smooth ramp from dd_threshold to floor

2026-03-29 20:23:51 +02:00

2026-03-29-generalization-gap.md

docs: generalization gap action plan (domain randomization + CQL + multi-fold)

2026-03-29 23:24:10 +02:00

2026-03-29-novel-generalization.md

docs: 8 novel generalization techniques plan (domain rand + CQL + adversarial + curiosity Q-penalty + counterfactual + regime-adversarial + feature filtering)

2026-03-29 23:32:12 +02:00

2026-03-30-dqn-gems-implementation.md

feat: wire OFI features through TOML profile + CUDA kernel guard tests

2026-03-31 01:35:04 +02:00

2026-03-30-gems-and-pearls-generalization.md

feat(generalization): complete all 34 tasks — 7 final CUDA kernels

2026-03-30 12:38:33 +02:00

2026-03-31-databento-download-workflow.md

refactor: remove entire CPU training path — 5,307 lines of dead code

2026-04-02 09:06:23 +02:00

2026-03-31-feature-precompute-pipeline.md

docs: feature precompute pipeline implementation plan

2026-03-31 23:35:47 +02:00

2026-03-31-imbalance-bars-and-cleanup.md

cleanup: remove ALL 32 enable_* feature flags — all features unconditional

2026-03-31 10:56:20 +02:00

2026-04-01-h100-gpu-optimization.md

docs: H100 GPU optimization implementation plan — 10 tasks

2026-04-01 21:41:21 +02:00

2026-04-02-autobatchsizer-vram-budget.md

fix: restore batch_size to GPU profiles — profile defaults, not auto-scaling

2026-04-02 21:26:24 +02:00

2026-04-02-mega-graph-sub10s-epochs.md

refactor: remove entire CPU training path — 5,307 lines of dead code

2026-04-02 09:06:23 +02:00

2026-04-02-mega-graph-training-loop-refactor.md

perf: graph_aux CUDA graph capture + fix 3 kernel ABI mismatches

2026-04-02 13:49:26 +02:00

2026-04-02-zero-copy-fxcache-pipeline.md

fix: restore cached_n_episodes field lost in merge + batch_size=128 for RTX 3050

2026-04-02 23:29:04 +02:00

2026-04-02-zero-copy-walk-forward.md

fix: restore cached_n_episodes field lost in merge + batch_size=128 for RTX 3050

2026-04-02 23:29:04 +02:00

2026-04-03-argo-fxcache-infrastructure-fix.md

feat: add symbol field to DQNHyperparameters + training profile

2026-04-04 08:56:44 +02:00

2026-04-03-feature-extraction-pipeline-fix.md

feat: add symbol field to DQNHyperparameters + training profile

2026-04-04 08:56:44 +02:00

2026-04-03-hyperopt-fxcache-migration.md

fix: vol_normalizer from raw targets + relax fxcache smoketest NaN check

2026-04-03 08:46:30 +02:00

2026-04-04-c51-loss-accumulator-fix.md

docs: spec + plan for C51 loss accumulator fix (separate MSE/C51 buffers)

2026-04-04 11:49:13 +02:00

2026-04-04-cuda-kernel-safety.md

docs: implementation plan for CUDA kernel safety fix

2026-04-04 11:12:47 +02:00

2026-04-04-feature-extraction-pipeline-fix.md

docs: implementation plan for feature extraction pipeline fix

2026-04-04 09:36:10 +02:00

2026-04-04-gpu-test-fixes.md

docs: implementation plan for GPU test fixes + OFI pipeline fix

2026-04-04 01:02:00 +02:00

2026-04-05-gpu-regime-and-training-hang-fix.md

docs: implementation plan for GPU regime + H100 hang fix

2026-04-05 21:45:51 +02:00

2026-04-06-h100-maximum-performance.md

docs: H100 maximum performance spec and implementation plan

2026-04-06 11:15:24 +02:00

2026-04-06-per-prefix-sum.md

fix: promote entire backtest pipeline from bf16 to f32

2026-04-07 13:58:11 +02:00

2026-04-06-per-tree-fix-and-zero-sync-readback.md

docs: H100 maximum performance spec and implementation plan

2026-04-06 11:15:24 +02:00

2026-04-07-reward-v7-gems-pearls.md

docs: reward v8 master plan — 18-item comprehensive training overhaul

2026-04-07 23:14:40 +02:00

2026-04-07-reward-v7.md

docs: reward v8 master plan — 18-item comprehensive training overhaul

2026-04-07 23:14:40 +02:00

2026-04-07-reward-v8-master.md

feat(v8): wire 5 dead CUDA kernels + config fixes

2026-04-08 01:26:09 +02:00

2026-04-07-reward-v71-fixes.md

docs: reward v8 master plan — 18-item comprehensive training overhaul

2026-04-07 23:14:40 +02:00

2026-04-08-advantage-standardization.md

fix(critical): disable mirror universe — only negates 4/42 features but inverts direction

2026-04-08 20:05:22 +02:00

2026-04-08-h100-epoch-optimization-v2.md

fix: update loss threshold for v8 batch_size=16384 + wire 5 kernels

2026-04-08 02:25:58 +02:00

2026-04-08-h100-graph-optimizations.md

fix: aux GEMM gradient explosion — scale 0.5→0.01 + clip after backward

2026-04-08 10:03:54 +02:00

2026-04-08-hierarchical-4branch.md

docs: Phase 2 hierarchical 4-branch implementation plan

2026-04-08 14:12:12 +02:00

2026-04-08-magnitude-branch-fix.md

fix(critical): disable mirror universe — only negates 4/42 features but inverts direction

2026-04-08 20:05:22 +02:00

2026-04-08-per-branch-loss-weighting.md

feat(magnitude): per-branch loss weighting — 0.2× C51 + 4.0× MSE for magnitude

2026-04-08 22:23:19 +02:00

2026-04-08-volume-bar-pipeline.md

docs: volume bar pipeline spec + implementation plan

2026-04-08 20:26:40 +02:00

2026-04-09-adv-logits-layout-fix.md

fix(critical): branch-major d_adv_logits layout + revert diagnostic hacks

2026-04-09 22:05:46 +02:00

2026-04-09-h100-nan-fix.md

fix: re-enable IQN + remove diagnostics for H100 baseline

2026-04-09 20:04:01 +02:00

2026-04-10-argo-pipeline-redesign.md

feat: unified Argo train workflow — replace 7 templates with 1

2026-04-10 20:36:03 +02:00

2026-04-10-bf16-elimination.md

refactor: eliminate ALL bf16 from entire workspace — pure f32/TF32 pipeline

2026-04-10 18:52:21 +02:00

2026-04-10-cublaslt-migration.md

fix: use CUBLAS_COMPUTE_32F for cublasLtMatmul (SM86 compat)

2026-04-10 10:33:36 +02:00

2026-04-10-dead-code-gpu-profiling.md

plan: dead code cleanup + GPU per-phase CUDA event profiling (6 tasks, 32 steps)

2026-04-10 22:23:16 +02:00

2026-04-11-abi-complete.md

plan: ABI Complete — per-branch epsilon, IQN anti-collapse, hindsight relabeling, vaccine, ensemble, bottleneck (8 tasks)

2026-04-11 12:27:56 +02:00

2026-04-11-abi-foundation.md

plan: ABI Foundation — 7-level exposure + PopArt + tau coupling + PER clear (6 tasks, 27 steps)

2026-04-11 11:28:59 +02:00

2026-04-11-action-diversity-iqn4branch.md

fix: IQN 4-branch runtime + order counterfactual + per-branch entropy + histogram

2026-04-11 22:40:47 +02:00

2026-04-11-exp-collector-zero-copy.md

refactor: exp collector zero-copy — direct pointer into trainer params_buf

2026-04-11 23:10:25 +02:00

2026-04-11-varstore-elimination.md

plan: VarStore elimination — zero-copy pointer views into flat params_buf (8 tasks)

2026-04-11 13:35:15 +02:00

2026-04-12-adaptive-c51-vrange.md

feat: adaptive C51 v_range — Q-values 0.008→0.525 (65× improvement)

2026-04-12 07:58:02 +02:00

2026-04-12-dsr-reward-restructure.md

feat: v9 Differential Sharpe Ratio reward — optimize for consistency, not PnL

2026-04-12 14:33:56 +02:00

2026-04-13-iql-full-integration.md

docs: IQL full integration implementation plan — 9 tasks

2026-04-13 14:56:27 +02:00

2026-04-13-single-cublas-handle.md

docs: implementation plan for single shared cuBLAS handle

2026-04-13 11:32:23 +02:00

2026-04-14-adaptive-training-dynamics.md

plan: Adaptive Training Dynamics Layers 1+2 implementation plan

2026-04-14 21:42:57 +02:00

2026-04-14-cross-pollinated-branch-intelligence.md

plan: Cross-Pollinated Branch Intelligence — 9 tasks, 64 steps

2026-04-14 23:26:13 +02:00

2026-04-14-direction-conditioned-magnitude.md

fix: Layer 3 plan — add Q_dir normalization by delta_z

2026-04-14 22:30:29 +02:00

2026-04-15-oos-generalization-enhancement.md

plan: OOS Generalization Enhancement — 9 tasks, 10 components

2026-04-15 20:33:03 +02:00

2026-04-15-oos-performance-enhancement.md

plan: add Tasks 9d-9e — Q-mean drift regularization + new component LR warmup

2026-04-15 19:43:50 +02:00

2026-04-15-supervised-architecture-transfer.md

plan: add Tasks 10-11 — Q-mean centering + atom entropy recovery (execute first)

2026-04-15 08:40:09 +02:00

2026-04-16-introspective-state-vector.md

plan: Introspective State Vector — 9 tasks, pure GPU implementation

2026-04-16 22:46:54 +02:00

2026-04-16-learned-risk-management.md

plan: Learned Risk Management — 6 tasks, 5th branch risk_budget [0,1]

2026-04-16 01:32:16 +02:00

2026-04-16-self-improving-training-loop.md

plan: Self-Improving Training Loop — 6 tasks, 9 enrichments

2026-04-16 17:34:14 +02:00

2026-04-16-tick-microstructure-intelligence.md

plan: Tick Microstructure Intelligence — 4 tasks, OFI_DIM 8→20

2026-04-17 00:02:02 +02:00

2026-04-16-trade-plan-head.md

plan: Trade Plan Head — 5 tasks, hierarchical plan-based trading

2026-04-17 00:48:18 +02:00

2026-04-16-training-environment-alignment.md

plan: Training Environment Alignment — 5 tasks, 3 phases

2026-04-16 21:49:19 +02:00

2026-04-17-unified-cublas-training-graph.md

plan: rewrite Task 9 for child graph architecture

2026-04-17 23:58:32 +02:00

2026-04-18-multi-stream-phase2.md

plan: Phase 2 multi-stream parallelism — 7 tasks

2026-04-18 16:07:06 +02:00

2026-04-18-unified-single-graph.md

plan: unified single-graph — 5 tasks, zero ungraphed kernel launches

2026-04-18 21:26:40 +02:00

2026-04-19-kernel-optimization-phase3.md

feat: target_dim 4→6 + spec v5 with pearls (bar duration, book CoM, retrospective hold)

2026-04-19 23:47:04 +02:00

2026-04-19-kernel-optimization.md

plan: kernel optimization — 8 tasks, 7 kernels → cuBLAS + batch-parallel

2026-04-19 12:15:04 +02:00

2026-04-19-ofi-momentum-dense-reward.md

plan: OFI momentum + dense micro-reward — 9 tasks, full gradient flow

2026-04-19 23:21:24 +02:00

2026-04-19-true-single-graph.md

plan: true single-graph — 7 tasks, zero CPU on hot path

2026-04-19 00:06:05 +02:00

2026-04-20-adaptive-learning-dynamics.md

plan: adaptive learning dynamics — 19 tasks across 5 phases

2026-04-20 19:12:41 +02:00

2026-04-20-unified-state-layout.md

plan: unified state layout implementation — 8 tasks

2026-04-20 14:36:03 +02:00

2026-04-21-policy-quality-phase2.md

plan(policy-quality): reframe DELETE tasks per no-functionality-removal rule

2026-04-22 11:18:28 +02:00

2026-04-21-policy-quality.md

docs(policy-quality): main-branch workflow — cut feat/wip branches

2026-04-21 21:06:33 +02:00

2026-04-24-dqn-v2-plan-1-substrate.md

plan(dqn-v2): Plan 1 substrate + refactor complete (Tasks 9-17)

2026-04-24 18:29:34 +02:00

2026-04-24-dqn-v2-plan-2-temporal-core.md

plan(dqn-v2): Plan 2 revision — GPU-drives architecture + Mamba2 validation scope

2026-04-24 19:04:56 +02:00

2026-04-24-dqn-v2-plan-3-behavioural.md

plan(dqn-v2): mark Tasks 8+9 LANDED; 11/12 done

2026-04-25 09:15:43 +02:00

2026-04-24-dqn-v2-plan-4-supervised-concepts.md

plan(dqn-v2): Plan 4 Task 2c GRN — further decomposed into 2c.1/2c.2/2c.3+4/2c.5

2026-04-25 11:53:10 +02:00

2026-04-24-dqn-v2-plan-5-validation.md

plan(dqn-v2): Plan 5 — validation substrate + final pass implementation plan

2026-04-24 10:06:25 +02:00

2026-04-27-distributional-rl-thompson-plan-A-phase-0.md

plan(dqn): Thompson sampling rollout — Plans A+B+C+D (4 sequential phases)

2026-04-27 08:10:42 +02:00

2026-04-27-distributional-rl-thompson-plan-B-phase-1.md

plan(dqn): Thompson sampling rollout — Plans A+B+C+D (4 sequential phases)

2026-04-27 08:10:42 +02:00

2026-04-27-distributional-rl-thompson-plan-C-phase-2.md

plan(dqn): Thompson sampling rollout — Plans A+B+C+D (4 sequential phases)

2026-04-27 08:10:42 +02:00

2026-04-27-distributional-rl-thompson-plan-D-phase-3.md

plan(dqn): Thompson sampling rollout — Plans A+B+C+D (4 sequential phases)

2026-04-27 08:10:42 +02:00