gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

plan(dqn-v2): Plan 4 reality reconciliation

Browse Source 
Plan 4 was drafted 2026-04-24 against an earlier Plan 3 design. After
Plan 3 landed (commits 44539d8f4..3cb083f18), the pre-plan gate
referenced slot names that don't exist in the landed implementation:

- PLAN_PARAMS_0_EMA_INDEX → became READINESS_EMA_INDEX (Task 4)
- STATE_KL_THRESHOLD_EMA_INDEX → eliminated; Task 7 uses kernel-internal
  trailing-EMA-of-self pattern (no separate threshold slot)
- TEMPORAL_REWARD_{PERSIST,REGIME_SHIFT,CONSISTENCY}_EMA_INDEX → consolidated
  into rc[5] → ISV[68] REWARD_BONUS_EMA_INDEX via Plan 3 Task 6a/6b/6c

Updated:
- Pre-plan slot-existence check matches landed slot names
- Validation-doc gate softened: Plan 3 Argo Tier 1 PASS becomes OPTIONAL;
  local 5-epoch multi-fold smoke is the de-facto gate (user choice
  to proceed without burning Argo cycles)
- Status table at top shows landed Plan 3 baseline (ISV_TOTAL_DIM=87,
  PS_STRIDE=43, fingerprint at [85,86]) and per-task difficulty estimate
- Reality-reconciliation note documents the rename trail

No task body changes; subsequent commits will revise individual task
specs as they become next-up for execution.
This commit is contained in:
jgrusewski
2026-04-25 09:21:36 +02:00
parent e1626876af
commit 3bfbcd7137
1 changed files with 54 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

71
docs/superpowers/plans/2026-04-24-dqn-v2-plan-4-supervised-concepts.md
View File

@@ -4,46 +4,83 @@
**Goal:** Wire the six IN concepts from Part E of the DQN v2 spec into the DQN training pipeline: full TFT Variable Selection Network, Gated Residual Network as a trunk building block (if absent), multi-quantile IQN decomposition, explicit encoder-decoder separation, ISV-exposed attention interpretability, and auxiliary regression/classification heads. Each concept lands wired, diagnostic, and completely finished — no "evaluate later" (Invariant 9).
**Architecture:** Most adoption is additive: a TFT VSN layer in front of the existing trunk, attention-weight ISV plumbing from Mamba2 + TLOB + trunk heads into new diagnostic slots `[63..72)`, auxiliary heads branching off the trunk output via a lightweight two-layer MLP. Encoder-decoder separation is a refactor-not-rewrite: the existing trunk is renamed `StateEncoder`, the four Q-heads become `ValueDecoder`s with an explicit interface boundary. Multi-quantile IQN extends the existing IQN branch — no new branch needed.
**Status (2026-04-25, before any Plan 4 task lands):** Plan 3 substrate fully landed (11/12 tasks; Task 10 Argo validation pending but not blocking). `ISV_TOTAL_DIM = 87`, `PS_STRIDE = 43`, fingerprint at ISV [85, 86]. Local smoke baseline shows healthy convergence with 4 reward-shaping signals + state-KL amplification + GPU-only replay seed.
| Task | Scope | Adds parameters? | Difficulty |
|---|---|---|---|
| 1 — E.1 Full TFT VSN | extend `vsn_mag`/`vsn_dir` masks to all 6 feature groups | yes (per-group VSN params) | medium-heavy |
| 2 — E.2 GRN | audit-then-decide; if GRN absent in trunk, adopt; else canonicalise | conditional | medium or no-op |
| 3 — E.3 Multi-quantile IQN | extend IQN output from 1 to 5 quantiles (5/25/50/75/95) | yes (head expansion 1→5) | medium-heavy |
| 4 — E.4 Encoder-decoder separation | refactor — `StateEncoder` + 4× `ValueDecoder` with explicit interface | no | medium (cross-cutting) |
| 5 — E.5 Attention-weight interpretability | ISV-expose attention weights from Mamba2 + TLOB | no | **light — ISV-only** |
| 6 — E.6 Multi-task auxiliary heads | 2 new auxiliary heads (next-return regression + regime classification) | yes (2 small MLPs) | medium |
| 7 — Part E audit close-out | docs only; close E rows to "complete" | no | trivial |
| 8 — Plan 4 validation run | Argo L40S multi-seed × 60 epochs | no | infrastructure |
**Architecture:** Most adoption is additive: a TFT VSN layer in front of the existing trunk, attention-weight ISV plumbing from Mamba2 + TLOB + trunk heads into new diagnostic slots tail-appended after Plan 3's slot 86, auxiliary heads branching off the trunk output via a lightweight two-layer MLP. Encoder-decoder separation is a refactor-not-rewrite: the existing trunk is renamed `StateEncoder`, the four Q-heads become `ValueDecoder`s with an explicit interface boundary. Multi-quantile IQN extends the existing IQN branch — no new branch needed.
**Tech Stack:** Rust workspace + CUDA C++ via nvcc, cudarc 0.19, sccache with `CARGO_INCREMENTAL=0`. One new CUDA kernel: `vsn_feature_selection_kernel.cu`. Attention-weight plumbing reuses existing ISV/pinned-memory infrastructure.
**Authority:** `docs/superpowers/specs/2026-04-24-dqn-v2-unified-design.md` §4.E. All 9 invariants apply.
**Dependencies on prior plans:**
- Plan 1: `StateResetRegistry`, `AdaptiveController` trait, audit docs, named-dimension registry.
- Plan 1: `StateResetRegistry`, read-only `AdaptiveMonitor` trait (post-2026-04-24 GPU-drives revision; was `AdaptiveController`), audit docs, named-dimension registry.
- Plan 2: Per-branch γ, Q-quantile atoms, Mamba2 backward wired.
- Plan 3: Behavioural reward shaping + replay-seed warm-start. Multi-quantile IQN (E.3) can interact with C.4/D.4 temporal rewards.
- Part E audit at `docs/ml-supervised-to-dqn-concept-audit.md` must list E.2 GRN decision (IN-adopt vs IN-canonicalise) before Task 2 starts.
**Reality reconciliation note (2026-04-25):** Plan 4 was drafted 2026-04-24 against an earlier Plan 3 design. After Plan 3 landed, several pre-plan checks reference slot names that don't exist (`PLAN_PARAMS_0_EMA_INDEX` → became `READINESS_EMA_INDEX`; `STATE_KL_THRESHOLD_EMA_INDEX` → eliminated, kernel uses internal trailing EMA; `TEMPORAL_REWARD_*_EMA_INDEX` → consolidated into rc[5] / ISV[68] `REWARD_BONUS_EMA_INDEX`). The pre-plan slot-existence check below is updated to match landed reality. Validation-doc gate softened: Argo Plan 3 Tier 1 PASS becomes `OPTIONAL` — local smoke baseline is sufficient if the user wants to proceed without burning Argo cycles for the Plan 3 gate.
---
## Pre-plan: verify Plan 3 state
- [ ] **Step 0.1: Verify Plan 3 exit criteria satisfied**
- [ ] **Step 0.1: Verify Plan 3 substrate landed (post-2026-04-25 revision)**
```bash
# All 15 Plan 3 ISV slots must exist.
# Plan 3 ISV slots that actually landed (not the original-plan names).
# Reward-component EMAs (Task 1):
for name in REWARD_POPART_EMA_INDEX REWARD_CF_EMA_INDEX REWARD_TRAIL_EMA_INDEX \
REWARD_MICRO_EMA_INDEX REWARD_OPP_COST_EMA_INDEX REWARD_BONUS_EMA_INDEX \
TRADE_ATTEMPT_RATE_EMA_INDEX TRADE_TARGET_RATE_INDEX \
PLAN_PARAMS_0_EMA_INDEX \
STATE_KL_TRAIN_VAL_EMA_INDEX STATE_KL_THRESHOLD_EMA_INDEX STATE_KL_AMPLIFICATION_INDEX \
TEMPORAL_REWARD_PERSIST_EMA_INDEX TEMPORAL_REWARD_REGIME_SHIFT_EMA_INDEX \
TEMPORAL_REWARD_CONSISTENCY_EMA_INDEX; do
REWARD_MICRO_EMA_INDEX REWARD_OPP_COST_EMA_INDEX REWARD_BONUS_EMA_INDEX; do
grep -q "pub const $name" crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs || {
echo "Plan 3 ISV slot missing: $name"; exit 1
echo "Plan 3 Task 1 slot missing: $name"; exit 1
}
done
# Trade-attempt novelty (Task 3):
for name in TRADE_ATTEMPT_RATE_EMA_INDEX TRADE_TARGET_RATE_INDEX; do
grep -q "pub const $name" crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs || {
echo "Plan 3 Task 3 slot missing: $name"; exit 1
}
done
# Adaptive plan-threshold (Task 4 — readiness EMA at 75; PLAN_THRESHOLD_INDEX=49 reused):
grep -q "pub const READINESS_EMA_INDEX" crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs \
|| { echo "Plan 3 Task 4 slot missing: READINESS_EMA_INDEX"; exit 1; }
# State-KL (Task 7 — note: only TRAIN_VAL + AMPLIFICATION; threshold is kernel-internal):
for name in STATE_KL_TRAIN_VAL_EMA_INDEX STATE_KL_AMPLIFICATION_INDEX; do
grep -q "pub const $name" crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs || {
echo "Plan 3 Task 7 slot missing: $name"; exit 1
}
done
# Replay seed + CQL ramp (Task 8/9):
for name in SEED_STEPS_TARGET_INDEX SEED_STEPS_DONE_INDEX SEED_FRAC_EMA_INDEX; do
grep -q "pub const $name" crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs || {
echo "Plan 3 Task 8/9 slot missing: $name"; exit 1
}
done
# Temporal-reward suite consolidated into rc[5] / ISV[68] REWARD_BONUS_EMA — no separate
# TEMPORAL_REWARD_PERSIST/REGIME_SHIFT/CONSISTENCY slots; this is the post-revision design.
# C.4 timing-bonus uses PS-state PEAK_PNL_BAR (slot 38) — verify PS_STRIDE grew accordingly:
grep -q "PS_STRIDE *43\|PS_STRIDE *= *43" crates/ml/src/cuda_pipeline/state_layout.cuh \
|| { echo "PS_STRIDE not at expected 43 — Tasks 5+6a+6b+6c may not all be landed"; exit 1; }
# ISV total dim should be 87 post-Tasks 8+9:
grep -q "ISV_TOTAL_DIM: usize = 87" crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs \
|| { echo "ISV_TOTAL_DIM not 87 — Plan 3 not fully landed"; exit 1; }
# Plan 3 validation doc must exist with a "Tier 2 (behavioural subset) PASS" marker.
grep -q "Tier 1.*PASS" docs/dqn-v2-plan-3-validation.md || {
echo "Plan 3 Tier 1 gate not documented as passing"; exit 1
}
echo "Plan 3 exit criteria satisfied. Plan 4 may begin."
echo "Plan 3 substrate verified. Plan 4 may begin."
```
**Note on validation gate (softened):** The original plan required `docs/dqn-v2-plan-3-validation.md` with a "Tier 1 PASS" marker before Plan 4 starts. That gate was an Argo cloud deploy requirement. As of 2026-04-25 the user explicitly chose to proceed with Plan 4 without first burning Argo cycles for Plan 3 validation. Local 5-epoch multi-fold smoke baseline (`SQLX_OFFLINE=true cargo test -p ml --lib -- multi_fold_convergence --ignored`) is the de-facto gate. If a Plan 4 task regresses this smoke, fall back and investigate before continuing.
- [ ] **Step 0.2: Verify baseline compiles**
```bash