foxhunt

Files

jgrusewski 3c61ee52ce plan(dqn): Thompson sampling rollout — Plans A+B+C+D (4 sequential phases)

Implementation plans for the distributional-RL aggregation spec
(docs/superpowers/specs/2026-04-26-distributional-rl-aggregation-design.md).

Plan A — Phase 0: TDD hypothesis verification (8 tasks)
  - Standalone GPU test kernel: sample_c51_inverse_cdf,
    sample_iqn_quantile_interp, compute_e_c51, compute_e_iqn,
    thompson_direction_test, argmax_eq_test
  - 6 unit tests (5 GPU + 1 CPU synthetic edge)
  - GPU integration on converged checkpoint (Test 0.F)

Plan B — Phase 1: existing-lever audit (8 tasks)
  - 6 unit tests covering B.2/CF/PopArt/Q-target audit fixture
  - Exit gate: all 6 PASS = no reward-shaping bug; Phase 2 unblocked

Plan C — Phase 2: Thompson sampling integration (11 tasks)
  - Direction branch of experience_action_select rewritten:
    eps-greedy + Boltzmann → Thompson (training) + argmax E[Q] (eval)
  - C51 + IQN buffers wired to gpu_dqn_trainer + gpu_backtest_evaluator
  - train_active_frac HEALTH_DIAG instrumentation
  - 4 GPU-direct tests against production kernel
  - Aggregation Contract table + memory pearl

Plan D — Phase 3: long verification + dead-code cleanup (8 tasks)
  - Test 3.A: regression anchor against original C51 Flat bias
  - L4: 1 seed × 6 folds × 30 epoch (~1 hour)
  - L5: 5 seed × 6 fold matrix per Plan 5 Task 5 (Tier 1+2+3 PASS)
  - Direction-only eps_dir adaptive boost + Boltzmann tau-floor cleanup
    (per feedback_no_partial_refactor.md)
  - nsys regression check; final architecture/spec footer

Plans gated sequentially: each phase exit gate must pass before next.

2026-04-27 08:10:42 +02:00

infra

feat(infra): GPU training launcher with local/cloud routing

2026-02-23 11:19:58 +01:00

plans

feat(dqn): enable Branching DQN with 45 factored actions (5×3×3)

2026-03-11 22:00:13 +01:00

superpowers

plan(dqn): Thompson sampling rollout — Plans A+B+C+D (4 sequential phases)

2026-04-27 08:10:42 +02:00

dqn-gpu-hot-path-audit.md

feat(dqn): Plan 1 Task 9 — atoms_update GPU kernel + AtomsMonitor + delete CPU paths