From c0d3d7b2ff0b24549ed448cf9fc1dfd981248e30 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Sat, 25 Apr 2026 11:11:13 +0200 Subject: [PATCH] =?UTF-8?q?plan(dqn-v2):=20Plan=204=20Task=202=20GRN=20?= =?UTF-8?q?=E2=80=94=20decomposed=20into=202a/2b/2c=20with=20reality=20che?= =?UTF-8?q?cks?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit After a first dispatch attempt of monolithic Task 2 (GRN ADOPT) was correctly refused with a thorough scope assessment, this revision records the decomposition and the structural facts that drove it: 1. layout_fingerprint_seed() only fingerprints ISV slots, not param-tensor layout. The "fingerprint will auto-update" assumption in the original Task 2 spec was wrong for param-tensor reshuffles. 2. compute_param_sizes has 86 tensors (docstring saying 42 is stale). Inserting GRN's 9 sub-tensors shifts 82 downstream tensors and 98 padded_byte_offset call sites. 3. At least 12 kernels consume h_s2 expecting ReLU non-negativity. GRN's LayerNorm output is zero-mean (signed). Per-consumer verification needed before swap. 4. crates/ml-supervised::tft::gated_residual is incompatible as a port: different formula (sigmoid gate, not GLU split) and incompatible tensor abstraction. New CUDA kernels from scratch. Decomposition: - Task 2a (research, no code): audit h_s2 consumers for ReLU vs LayerNorm semantics. Output: per-consumer table. - Task 2b (small, checkpoint break): extend fingerprint seed to include param-tensor names + sizes. Pearl-aligned: complete fingerprint coverage rather than partial. - Task 2c (large, checkpoint break): GRN kernels + 98 call-site migration + h_s2 consumer shims (per 2a). Blocked on 2a+2b. Recommended order updated to thread 2a → 2b → 2c. Tasks 1, 6, 3 remain independent of 2c and can land in parallel where useful. Pearl rules section added documenting the safety constraints applied throughout the plan. No code changes. Plan-doc only. --- ...04-24-dqn-v2-plan-4-supervised-concepts.md | 73 ++++++++++++++++--- 1 file changed, 62 insertions(+), 11 deletions(-) diff --git a/docs/superpowers/plans/2026-04-24-dqn-v2-plan-4-supervised-concepts.md b/docs/superpowers/plans/2026-04-24-dqn-v2-plan-4-supervised-concepts.md index d0ea94067..4b2d20f68 100644 --- a/docs/superpowers/plans/2026-04-24-dqn-v2-plan-4-supervised-concepts.md +++ b/docs/superpowers/plans/2026-04-24-dqn-v2-plan-4-supervised-concepts.md @@ -125,15 +125,24 @@ digraph plan4_deps { } ``` -**Recommended order:** -1. **Task 5** — ISV-only attention diagnostics (light, no checkpoint break, gives telemetry for the heavier tasks) -2. **Task 4** — Encoder/decoder Rust-API split (refactor only, no kernel changes, no checkpoint break — sets clean boundaries for 1/3/6) -3. **Task 2** — GRN ADOPT (real architectural change, but localized to the 2 trunk Linear blocks; checkpoint break) -4. **Task 1** — Full VSN over feature groups (depends on E/D boundary; checkpoint break) -5. **Task 6** — Aux heads (depends on E/D boundary; checkpoint break) -6. **Task 3** — Multi-quantile IQN (most invasive — touches all IQN consumers; checkpoint break) -7. **Task 7** — Audit close-out (pure docs, after 1-6) -8. **Task 8** — Argo validation run (after 1-7 land) +**Recommended order (revised 2026-04-25 after Task 2 decomposition):** +1. **Task 5 Mode A** — ISV-only attention diagnostics — ✅ landed `cfc4ccb72` +2. **Task 4** — Encoder/decoder Rust-API split — ✅ landed `fbc299fa2` +3. **Legacy CPU-DtoH cleanup** — `per_branch_vsn_mean` + `per_branch_target_drift` deleted — ✅ landed `c247d9cab` +4. **Task 2a** — Audit h_s2 consumers for ReLU vs LayerNorm semantics (research, no code) — **next up** +5. **Task 2b** — Extend layout_fingerprint_seed to cover param-tensor layout (small commit, checkpoint break) +6. **Task 2c** — GRN ADOPT proper (large commit, checkpoint break) — blocked on 2a + 2b +7. **Task 1** — Full VSN over feature groups (depends on 2c trunk shape; checkpoint break) +8. **Task 6** — Aux heads (independent of 2c — can land in parallel; checkpoint break) +9. **Task 3** — Multi-quantile IQN (independent of trunk; checkpoint break) +10. **Task 7** — Audit close-out (pure docs, after 1/2c/3/6) +11. **Task 8** — Argo validation run (terminal) + +**Pearl-derived safety rules applied throughout:** +- `pearl_cold_path_no_exception_to_gpu_drives.md` — every producer (mean, EMA, RMS, etc.) goes through a GPU kernel even on cold path. No CPU DtoH+loop helpers. +- `pearl_one_unbounded_signal_per_reward.md` — exactly one unbounded multiplicand per reward term; bound everything else to [0,1]. +- `feedback_no_partial_refactor.md` — shared-contract changes (param-tensor layout, ISV layout) migrate every consumer in the same commit. +- `feedback_no_legacy_aliases.md` — no compat shims for renamed/removed code paths. **Checkpoint break consolidation:** Tasks 2, 1, 6, 3 each break checkpoint compatibility. Land them in sequence with NO Argo run between (one fingerprint shift per commit; final Argo run at Task 8 amortizes the retraining cost across all changes). Revert to a known-good commit if any single task regresses the local smoke beyond ~25-point Sharpe drop. @@ -383,9 +392,51 @@ git commit -m "plan4(task1C): upgrade E.1 VSN to per-feature weights (cond. on 1 --- -## Task 2: E.2 Gated Residual Network — Branch A (ADOPT) confirmed +## Task 2: E.2 Gated Residual Network — Branch A (ADOPT), DECOMPOSED into 2a/2b/2c -**Reality reconciliation (2026-04-25).** GRN audit completed: GRN exists in `crates/ml-supervised/src/tft/gated_residual.rs` (Rust + supervised pretraining only) and `crates/ml/src/transformers/temporal_fusion.rs` (TFT inference path). It is **NOT** wired into the DQN cuda_pipeline trunk. The DQN trunk in `batched_forward.rs::forward_online_raw` is `Linear→ReLU→Linear` with no LayerNorm, no ELU, no GLU. **Branch A (ADOPT) applies.** Branch B is moot. +**Reality reconciliation (2026-04-25, second iteration).** GRN audit completed: GRN exists in `crates/ml-supervised/src/tft/gated_residual.rs` (Rust + supervised pretraining only) and `crates/ml/src/transformers/temporal_fusion.rs` (TFT inference path). It is **NOT** wired into the DQN cuda_pipeline trunk. The DQN trunk in `batched_forward.rs::forward_online_raw` is `Linear→ReLU→Linear` with no LayerNorm, no ELU, no GLU. **Branch A (ADOPT) applies.** Branch B is moot. + +### Why this decomposed (post-2026-04-25 dispatch attempt) + +A first dispatch of monolithic Task 2 was refused by the implementer with a thorough scope assessment. Three structural facts changed the calculus: + +1. **`layout_fingerprint_seed()` only covers ISV slots, not param-tensor layout.** The current scheme can detect ISV-shape changes but param-tensor reshuffles (which GRN will cause) pass silently through checkpoint load. The "fingerprint will auto-update" assumption was wrong. +2. **`compute_param_sizes` has 86 tensors** (the docstring saying 42 is stale). Inserting GRN's 9 sub-tensors at trunk position shifts 82 downstream tensors and **98 `padded_byte_offset` call sites** across the cuda_pipeline. Single-commit migration is required (per `feedback_no_partial_refactor.md`) but per-site verification is non-trivial. +3. **At least 12 kernels consume `h_s2`** (VSN gates, Mamba2 history, attention, plan head, KAN spline, ISV feature gate, ISV temporal route, regime dropout, prediction loss, selectivity head, conf head, value/branch decoders). All currently see `ReLU(...)` output (non-negative); GRN's LayerNorm output is zero-mean and contains negatives. Whether each consumer's downstream arithmetic tolerates negative inputs is unverified. + +The existing supervised GRN at `crates/ml-supervised::tft::gated_residual` is **incompatible** as a port: different formula (sigmoid gate, not GLU split) and different tensor abstraction (no cuBLAS). New CUDA kernels must be written from scratch. + +### Decomposition + +#### Task 2a: Audit h_s2 consumers for ReLU vs LayerNorm semantics (research only, no code changes) + +Inventory every kernel/site that reads the trunk's `h_s2` buffer. For each: (i) document what arithmetic the consumer applies, (ii) flag whether it implicitly assumes `h_s2 ≥ 0`, (iii) propose mitigation if so (e.g., `relu(h_s2)` shim at the consumption site, or design change to handle signed input). Output: a table appended to `docs/ml-supervised-to-dqn-concept-audit.md` (or a new audit doc), one row per consumer. + +This is the unblock for Task 2c — without per-consumer signoff, the trunk swap risks silent breakage. + +#### Task 2b: Extend `layout_fingerprint_seed()` to cover param-tensor layout (small commit, checkpoint break) + +The current seed string fingerprints ISV slot names + indices. Extend it to also include param-tensor names + sizes (e.g., `W_S1=86,SH1xSD;b_S1=87,SH1;…`). Re-bake `LAYOUT_FINGERPRINT_CURRENT`. Add the param-layout assertion to checkpoint load alongside the ISV fingerprint check. With this in place, **any** param-tensor reshuffle will trigger fail-fast on stale checkpoints — including Task 2c's GRN insertion. + +This task itself is a one-commit checkpoint break (the seed string changes, fingerprint changes, all current checkpoints fail load — but no model behavior changes, so the smoke is just "does training still complete from cold start"). + +Pearl: same family as `pearl_cold_path_no_exception_to_gpu_drives.md` — the fingerprint contract should be **complete** for what it claims to protect, not partial. Partial coverage is worse than no coverage because it implies safety where there isn't. + +#### Task 2c: GRN ADOPT proper (large commit, checkpoint break) + +With 2a's per-consumer signoff and 2b's structural fingerprint in place, the actual ADOPT becomes mechanical: + +- New `grn_kernel.cu` (forward + backward, composed from cuBLAS GEMM + small custom kernels: `grn_elu_inplace`, `grn_glu_split`, `grn_layernorm_forward`, `grn_layernorm_backward`, `grn_residual_add`) +- New `gpu_grn.rs` (Rust wrapper, Adam state, init paths) +- Modify `compute_param_sizes`: insert GRN_h_s1 (7 tensors: W_a, b_a, W_b, b_b, W_residual, γ, β) and GRN_h_s2 (6 tensors: same minus W_residual since SH1==SH2). Total +9 trunk tensors. +- Migrate **all 98 `padded_byte_offset` call sites** that index ≥ trunk-end position (per `feedback_no_partial_refactor.md`) +- Update `batched_forward.rs::encoder_forward_only` and `batched_backward.rs` trunk path +- Apply 2a-derived shims at h_s2 consumers if any required non-negativity +- Update audit docs + +**No CPU compute introduced.** All GRN ops on GPU (cuBLAS GEMMs + custom element-wise kernels). The pearl `pearl_cold_path_no_exception_to_gpu_drives.md` applies: even diagnostic-side reductions of GRN params (if any) go through the same multi-block GPU kernel pattern Tasks 5 and the legacy-cleanup follow-up established. + +### Original task body (pre-decomposition reference, kept for design context) **Implementation surface (concrete):** - **Trunk site**: `batched_forward.rs::forward_online_raw` lines 471–481 (h_s1 + h_s2 GEMMs). Replace each `Linear→ReLU→Linear` block with a GRN block.