jgrusewski c2825a7928 spec: GPU-native Hindsight Experience Replay for wave-exit timing ...

Backward-looking HER: on trade close, compute peak unrealized PnL
during the trade's lifetime. If peak >> actual, inject synthetic
transition with peak_pnl and boosted priority. Teaches the agent
optimal wave-exit timing 10× faster than sparse reward alone.

Two kernels: rl_hindsight_track (per-step mid accumulation) and
rl_hindsight_inject (synthetic push on done with coordination).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-26 00:35:18 +02:00

..

2026-03-10-zero-cpu-hotpath-design.md

…

2026-03-11-dqn-factored-actions-design.md

…

2026-03-13-gpu-test-workflow-design.md

…

2026-03-14-kanidm-sso-design.md

…

2026-03-15-fused-cuda-training-design.md

…

2026-03-21-h100-epoch-optimization-design.md

…

2026-03-21-h100-epoch-optimization-phase3-design.md

…

2026-03-21-training-profiles-design.md

…

2026-03-22-gpu-composite-reward-design.md

…

2026-03-22-gpu-segment-tree-design.md

…

2026-03-22-two-phase-hyperopt-design.md

…

2026-03-23-9-exposure-action-space-design.md

…

2026-03-23-dual-distributional-c51-iqn-design.md

diag+docs(dqn): trunk-slice grad decomposition + stale IQN-trunk doc fix

2026-04-22 22:03:20 +02:00

2026-03-23-profitability-roadmap.md

…

2026-03-23-trading-realism-conformance.md

…

2026-03-25-boundary-trade-counting-design.md

…

2026-03-25-trade-lifecycle-fix-design.md

…

2026-03-26-eval-cleanup-design.md

…

2026-03-27-bf16-performance-design.md

…

2026-03-27-bug-fixes-design.md

…

2026-03-27-code-hygiene-design.md

…

2026-03-27-full-bf16-rewrite-design.md

…

2026-03-31-feature-precompute-pipeline-design.md

…

2026-04-01-h100-gpu-optimization-design.md

…

2026-04-04-c51-loss-accumulator-fix-design.md

docs: spec + plan for C51 loss accumulator fix (separate MSE/C51 buffers)

2026-04-04 11:49:13 +02:00

2026-04-04-cuda-kernel-safety-design.md

docs: spec for CUDA kernel safety fix (__restrict__ aliasing NaN bug)

2026-04-04 11:07:47 +02:00

2026-04-04-feature-extraction-pipeline-fix-design.md

docs: spec for feature extraction pipeline fix (strict cache, symbol scoping)

2026-04-04 09:28:42 +02:00

2026-04-04-gpu-test-fixes-design.md

docs: spec for fixing 14 GPU test failures + OFI pipeline issue

2026-04-04 00:57:49 +02:00

2026-04-04-q-value-explosion-investigation.md

docs: spec for Q-value explosion / NaN investigation

2026-04-04 23:00:16 +02:00

2026-04-04-training-step-performance-design.md

docs: spec for training step performance + Q-value explosion investigation

2026-04-04 17:30:18 +02:00

2026-04-05-gpu-regime-and-training-hang-fix.md

docs: spec for GPU regime + H100 training hang fix

2026-04-05 21:25:05 +02:00

2026-04-06-gpu-native-validation-design.md

spec: GPU-native validation + H100 performance optimizations

2026-04-06 22:48:07 +02:00

2026-04-06-h100-maximum-performance.md

docs: H100 maximum performance spec and implementation plan

2026-04-06 11:15:24 +02:00

2026-04-06-per-prefix-sum-design.md

fix: promote entire backtest pipeline from bf16 to f32

2026-04-07 13:58:11 +02:00

2026-04-06-per-tree-fix-and-zero-sync-readback.md

docs: H100 maximum performance spec and implementation plan

2026-04-06 11:15:24 +02:00

2026-04-07-f32-backtest-pipeline-design.md

spec: f32 backtest pipeline — eliminate bf16 financial arithmetic

2026-04-07 13:43:04 +02:00

2026-04-07-reward-v7-design.md

feat(v8): wire 5 dead CUDA kernels + config fixes

2026-04-08 01:26:09 +02:00

2026-04-07-reward-v8-master-plan.md

docs: reward v8 master plan — 18-item comprehensive training overhaul

2026-04-07 23:14:40 +02:00

2026-04-07-reward-v71-fixes-design.md

docs: reward v7.1 spec — 6 critical fixes for exposure degeneracy and training quality

2026-04-07 22:21:11 +02:00

2026-04-08-advantage-standardization-design.md

fix(critical): disable mirror universe — only negates 4/42 features but inverts direction

2026-04-08 20:05:22 +02:00

2026-04-08-h100-epoch-optimization-v2.md

docs: H100 epoch optimization v2 spec — 47s → 19s (2.5x speedup)

2026-04-08 01:49:01 +02:00

2026-04-08-h100-graph-optimizations.md

docs: H100 graph-level optimization spec — 4 deferred fixes (~13s/epoch)

2026-04-08 08:48:36 +02:00

2026-04-08-hierarchical-4branch-design.md

docs: Phase 2 hierarchical 4-branch DQN spec — direction(3) × magnitude(3) × order(3) × urgency(3)

2026-04-08 12:04:23 +02:00

2026-04-08-per-branch-loss-weighting-design.md

feat(magnitude): per-branch loss weighting — 0.2× C51 + 4.0× MSE for magnitude

2026-04-08 22:23:19 +02:00

2026-04-08-volume-bar-pipeline-design.md

docs: volume bar pipeline spec + implementation plan

2026-04-08 20:26:40 +02:00

2026-04-10-argo-training-pipeline-redesign.md

spec: Argo training pipeline redesign — one workflow, smart caching

2026-04-10 20:26:55 +02:00

2026-04-10-dead-code-cleanup-gpu-profiling.md

spec: dead code cleanup + GPU per-phase CUDA event profiling

2026-04-10 22:16:24 +02:00

2026-04-11-autonomous-branch-intelligence-design.md

spec: Autonomous Branch Intelligence (ABI) — fix DQN action collapse

2026-04-11 11:23:23 +02:00

2026-04-11-candle-varstore-elimination.md

spec: Candle GpuVarStore elimination — 40+ call sites, 15 files, blocks bottleneck

2026-04-11 13:30:12 +02:00

2026-04-13-deterministic-forward-design.md

docs: spec for deterministic eval via training graph replay

2026-04-13 09:26:14 +02:00

2026-04-13-iql-full-integration-design.md

docs: fix 6 issues in IQL integration spec from code review

2026-04-13 14:51:30 +02:00

2026-04-13-iql-h100-hardening-design.md

docs: IQL H100 hardening spec — drift readiness, p5 floor, buffer-relative staleness

2026-04-13 20:13:39 +02:00

2026-04-13-single-cublas-handle-design.md

docs: spec for single shared cuBLAS handle refactor

2026-04-13 11:29:05 +02:00

2026-04-14-adaptive-training-dynamics-design.md

spec: Adaptive Training Dynamics v2 — 4-layer design for breaking Sharpe plateau

2026-04-14 21:40:03 +02:00

2026-04-14-cross-pollinated-branch-intelligence-design.md

spec: Component 6 early termination — PLATEAU_EXHAUSTED after 12 attempts

2026-04-14 23:46:27 +02:00

2026-04-14-direction-conditioned-magnitude-design.md

spec: direction-conditioned magnitude head (Layer 3)

2026-04-14 22:14:49 +02:00

2026-04-15-oos-generalization-enhancement-design.md

spec: add G11-G15 — Q-anchoring, predictive coding, Sharpe reward, confidence replay, commitment

2026-04-15 20:46:05 +02:00

2026-04-15-oos-performance-enhancement-design.md

spec: OOS Performance Enhancement — 8 components for closing val/OOS gap

2026-04-15 18:44:18 +02:00

2026-04-15-supervised-architecture-transfer-design.md

spec: Component 13 + full train-vpb4w results + backtracking bug analysis

2026-04-15 08:52:17 +02:00

2026-04-16-homeostatic-regularization-design.md

spec: trade-level reward attribution + exploration risk budget + homeostatic regularization

2026-04-16 01:24:03 +02:00

2026-04-16-introspective-state-vector-design.md

spec: Phase 4 adaptive minimum hold time — prevent coin-flip exits

2026-04-16 23:51:45 +02:00

2026-04-16-learned-risk-management-design.md

feat: trade-level reward attribution + learned risk management spec

2026-04-16 01:29:11 +02:00

2026-04-16-self-improving-training-loop-design.md

spec: Self-Improving Training Loop — 9 enrichments, eval-as-training

2026-04-16 17:30:01 +02:00

2026-04-16-tick-microstructure-intelligence-design.md

spec: fix 4 critical issues from review — OFI_DIM cascade, deploy atomicity

2026-04-16 23:43:55 +02:00

2026-04-16-trade-level-reward-exploration-design.md

spec: trade-level reward attribution + exploration risk budget + homeostatic regularization

2026-04-16 01:24:03 +02:00

2026-04-16-trade-plan-head-design.md

spec: Plan Phase 2 — 6 pearls/gems/novels for adaptive planning

2026-04-17 09:01:53 +02:00

2026-04-16-training-environment-alignment-design.md

spec: adapt components instead of removing — DSR on trade P&L, rank non-zeros only

2026-04-16 21:44:35 +02:00

2026-04-17-unified-cublas-training-graph-design.md

docs: update unified graph spec with Phase 1 H100 measurements

2026-04-18 14:59:57 +02:00

2026-04-18-multi-stream-phase2-design.md

spec: Phase 2 multi-stream parallelism design

2026-04-18 16:03:45 +02:00

2026-04-18-true-single-graph-design.md

spec: true single-graph — zero CPU on hot path, zero ungraphed launches

2026-04-18 23:59:35 +02:00

2026-04-18-unified-single-graph-design.md

spec: add Phase 3 performance optimizations (8 items, prioritized)

2026-04-18 22:04:41 +02:00

2026-04-19-kernel-optimization-design.md

spec: kernel optimization — cuBLAS projections + batch-parallel scans

2026-04-19 12:10:49 +02:00

2026-04-19-ofi-momentum-dense-reward-design.md

feat: target_dim 4→6 + spec v5 with pearls (bar duration, book CoM, retrospective hold)

2026-04-19 23:47:04 +02:00

2026-04-20-adaptive-learning-dynamics-design.md

spec: review fixes — labels, dedup, P2 as 7th component, WinRate 55%

2026-04-20 18:58:35 +02:00

2026-04-20-bitonic-sort-integration-design.md

feat: wire config weights to kernel + revert C51 alpha + atom warm-start spec

2026-04-20 08:51:42 +02:00

2026-04-20-data-pipeline-gaps-design.md

cleanup: remove legacy Sequential q_network + tune config for dense reward

2026-04-20 08:17:34 +02:00

2026-04-20-dqn-cleanup-ralph-loop-design.md

docs(cleanup): ralph-loop prompt + scoreboard for DQN code cleanup sweep

2026-04-20 22:03:14 +02:00

2026-04-20-unified-state-layout-design.md

spec: unified state layout — fix train/validation mismatch

2026-04-20 14:29:53 +02:00

2026-04-21-phase1-reward-inventory.md

diag+gate: TD-propagation smoke test + entropy→plasticity gate + tx_cost doc

2026-04-21 02:05:28 +02:00

2026-04-21-policy-quality-baseline-metrics.md

docs(policy-quality): Task 0.17 baseline metrics — captured (Phase 0 complete)

2026-04-22 08:42:50 +02:00

2026-04-21-policy-quality-design.md

docs(policy-quality): main-branch workflow — cut feat/wip branches

2026-04-21 21:06:33 +02:00

2026-04-21-policy-quality-task2x-scoping.md

docs(policy-quality): Task 2.X scoping — per-bin variance + Q-spread regularization

2026-04-22 11:43:17 +02:00

2026-04-21-policy-quality-track1-triage.md

docs(policy-quality): Track 1 magnitude triage (Phase 1) — H4 + H10 CONFIRMED

2026-04-22 08:51:51 +02:00

2026-04-21-policy-quality-track2-triage.md

cleanup+fix(reward): Task 2.4 R6 relocation + Task 2.5 Bug #6 docstring

2026-04-22 16:56:36 +02:00

2026-04-21-policy-quality-track3-triage.md

docs(policy-quality): Track 3 controllers audit (Phase 1) — 0 load-bearing, 6 diagnostic, 1 candidate-for-delete

2026-04-22 09:09:09 +02:00

2026-04-21-policy-quality-track4-triage.md

docs(policy-quality): Track 2 reward audit (Phase 1) — 2 DELETE, 3 KEEP

2026-04-22 09:06:50 +02:00

2026-04-21-unified-train-val-env-design.md

phase3(env-unification): full unification — all 3 kernels now call unified_env_step_core

2026-04-21 11:04:51 +02:00

2026-04-23-isv-signal-quality-audit.md

docs(isv): signal quality audit for DQN ISV bus — L40S oscillation diagnosis

2026-04-23 10:15:16 +02:00

2026-04-23-isv-v-range-unification.md

feat(dqn): ISV-unified per-branch Q-support range (spec 2026-04-23)

2026-04-23 21:18:19 +02:00

2026-04-23-tlob-feature-integration.md

docs(tlob): feature-diff doc for TLOB 51 vs OFI 20 — Phase A

2026-04-23 09:12:01 +02:00

2026-04-24-dqn-v2-unified-design.md

spec(dqn-v2): D.8 TLOB pivot — cuBLAS port using existing DQN infra, no ONNX, no pretraining

2026-04-24 21:03:57 +02:00

2026-04-26-distributional-rl-aggregation-design.md

spec(dqn): Plan C Phase 2 ACTIVE — UCB asymmetry regression triggers Thompson resumption

2026-04-29 16:02:53 +02:00

2026-04-27-moe-regime-redesign-design.md

spec+plan(moe): correct gate input dim 42 → STATE_DIM=128

2026-04-27 18:22:40 +02:00

2026-04-29-numerical-stability-investigation-design.md

spec(dqn): SP1 numerical stability — revise per critical review

2026-04-29 22:59:00 +02:00

2026-04-29-numerical-stability-sp2-sp3-design.md

spec(dqn): SP2 + SP3 combined — F0 regression fix + Q-learning structural stability

2026-04-30 08:36:32 +02:00

2026-04-30-sp4-signal-driven-magnitude-control-design.md

fix(sp4): migrate fast/slow grad-norm EMA update to GPU per feedback_no_cpu_compute_strict

2026-05-01 14:07:10 +02:00

2026-05-01-sp5-magnitude-differentiation-and-eval-collapse-design.md

docs(sp5): remove duplicate Layer A close-out section (superseded by Layer D)

2026-05-01 19:54:02 +02:00

2026-05-02-sp6-per-branch-consumer-infrastructure-refactor.md

docs(sp6): brainstorm spec — per-branch consumer infrastructure refactor

2026-05-02 01:36:48 +02:00

2026-05-03-sp7-loss-balance-controller-design.md

spec(sp7): v2 — flatness-gated targets + Wiener α + sentinel-bootstrap

2026-05-03 00:11:26 +02:00

2026-05-03-sp9-kelly-cold-start-isv-driven.md

spec(sp9): Kelly cold-start warmup floor — fully ISV-driven design

2026-05-03 20:33:36 +02:00

2026-05-03-sp10-unconditional-thompson-selector-isv-temperature.md

spec(sp10): unconditional Thompson selector + ISV-driven temperature

2026-05-03 22:24:14 +02:00

2026-05-04-sp11-reward-as-controlled-subsystem.md

spec(sp11): amend B1b smoke-recovery — z-score normalization for mag-ratio canary

2026-05-04 11:30:52 +02:00

2026-05-04-sp12-quality-over-quantity-reward-shaping.md

spec(sp12): per-trade event-driven reward composition (v3)

2026-05-04 18:41:57 +02:00

2026-05-04-sp13-redefine-success-for-predictive-skill.md

docs(sp13): v3 spec + plan — Hold-pricing replaces Hold-elimination

2026-05-04 23:18:58 +02:00

2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md

docs(sp14): spec v3 — K=4 direction + ISV-adaptive β rate limiter

2026-05-05 17:21:17 +02:00

2026-05-06-trader-discipline-and-recovery-design.md

docs(sp15): amend spec v2 — 10 fixes from second critical review

2026-05-06 08:58:28 +02:00

2026-05-08-sp18-reward-shape-hold-attractor-design.md

docs(sp18): copy spec + plan to feat/sp18-combined branch

2026-05-09 00:25:06 +02:00

2026-05-09-sp19-20-wr-first-design.md

feat(sp20): Phase 1.4 (Path C) — kernel-arg refactor + aggregation kernel + wire-up

2026-05-09 20:49:07 +02:00

2026-05-10-foxhunt-specialized-agents-design.md

docs(claude): foxhunt-aware agents & skills design — phase 1 (12 items)

2026-05-10 22:19:28 +02:00

2026-05-15-mamba2-grn-integration-notes.md

docs(phase-e-4-a): mamba2 + grn integration research — use ml-alpha Mamba2Block

2026-05-15 20:32:47 +02:00

2026-05-15-phase-e-temporal-encoder-design.md

docs(phase-e): E.4 temporal encoder design + E.4.A implementation plan

2026-05-15 20:42:35 +02:00

2026-05-15-t10-mamba2-backward-from-h-enriched.md

docs(phase-e-4-a): execution status + T10 backward_from_h_enriched patch sketch

2026-05-15 22:08:31 +02:00

2026-05-16-ml-alpha-cfc-ppo-design.md

refactor(ml-alpha): drop competitive-gate machinery — stacked is default

2026-05-16 23:17:33 +02:00

2026-05-17-kloop-parallelization-design.md

spec(perf): revise K-loop parallelization design — full critical pass

2026-05-17 23:13:32 +02:00

2026-05-18-auc-regret-controller-design.md

spec(ml-alpha): mark AUC-regret controller SUPERSEDED — empirically falsified

2026-05-18 23:24:01 +02:00

2026-05-18-ml-alpha-v2-multi-horizon-design.md

spec(ml-alpha): v2 multi-horizon redesign (A+B+C+D+E integrated)

2026-05-18 13:20:26 +02:00

2026-05-18-per-horizon-attention-pool-design.md

spec(ml-alpha): per-horizon attention pool design (C20)

2026-05-18 10:20:38 +02:00

2026-05-18-real-lob-integration-design.md

spec(ml-backtesting): real-LOB integration design

2026-05-18 01:51:11 +02:00

2026-05-19-cbsw-cold-start-aggregator-design.md

refactor(ml-backtesting): delete StopRules + use_cold_start_stopgap atomically

2026-05-20 00:40:14 +02:00

2026-05-19-deployability-sweep-parallelism-design.md

spec(ml-backtesting): apply 9 critical-review fixes to parallelism spec

2026-05-19 16:27:38 +02:00

2026-05-19-isv-driven-stop-controller-design.md

feat(ml-backtesting): max-hold + exit_px defensive + spec criteria revision

2026-05-20 08:10:00 +02:00

2026-05-19-ml-alpha-v2-deployability-validation-design.md

spec(ml-alpha): trunk-grows refactor + deployability validation (supersedes prior)

2026-05-19 00:54:45 +02:00

2026-05-19-ml-alpha-v2-trunk-grows-and-deployability-design.md

spec(ml-alpha): trunk-grows refactor + deployability validation (supersedes prior)

2026-05-19 00:54:45 +02:00

2026-05-20-continuous-reasoning-trader-design.md

spec(crt): v3 architecture reset — collapse Phase A+B into CRT.1 unified controller

2026-05-20 20:15:28 +02:00

2026-05-20-crt-train-output-smoothness-design.md

spec(crt-train): output smoothness retraining intervention (intervention A)

2026-05-20 22:59:49 +02:00

2026-05-21-crt-train-intervention-b-multi-timescale-readout.md

docs(per-horizon-cfc): kickoff — spec + plan + historical record

2026-05-21 15:05:28 +02:00

2026-05-21-crt-train-isv-driven-lambda-controller.md

docs(per-horizon-cfc): kickoff — spec + plan + historical record

2026-05-21 15:05:28 +02:00

2026-05-21-gpu-log-ring-design.md

docs(per-horizon-cfc): kickoff — spec + plan + historical record

2026-05-21 15:05:28 +02:00

2026-05-21-per-horizon-cfc-inference-design.md

docs(per-horizon-cfc): kickoff — spec + plan + historical record

2026-05-21 15:05:28 +02:00

2026-05-24-sp20-trader-grade-trade-management-design.md

docs(sp20): trader-grade trade management spec + audit infrastructure

2026-05-24 16:28:39 +02:00

2026-05-25-cuda-graph-step-pipeline.md

spec: update CUDA graph spec with all session findings and implementation plan

2026-05-25 21:41:10 +02:00

2026-05-25-cuda-performance-roadmap.md

refactor(rl): pre-allocate 56 replay-step gradient buffers for Graph C

2026-05-25 23:00:40 +02:00

2026-05-25-gpu-per-async-diag.md

spec: GPU-resident PER + async non-blocking diag streaming

2026-05-26 00:00:00 +02:00

2026-05-25-q-learning-improvements.md

spec: add mandatory constraints section — GPU-only, ISV-driven, TDD

2026-05-25 20:09:14 +02:00

2026-05-26-gpu-hindsight-replay.md

spec: GPU-native Hindsight Experience Replay for wave-exit timing

2026-05-26 00:35:18 +02:00