spec(dqn): add Invariant 9 — no deferred work, no stubs, concrete E decisions

User-mandated addition:

Invariant 9 — No deferred work, no stubs, no TODO/FIXME
Every commit lands complete. No stubs (placeholder return values), no
TODO/FIXME/XXX/HACK/TBD markers, no half-finished implementations. If
it can't finish in this commit, it doesn't start. Stubs and deferrals
train the network on semantic emptiness — invisible in convergence
metrics but burn GPU time optimising against partly-fake signal.
Authority: feedback_no_stubs.md, feedback_no_todo_fixme.md,
feedback_no_quickfixes.md elevated to first-class spec enforcement.
Pre-commit hook greps for forbidden markers.

Part E decisions made concrete (per Invariant 9):
- E.1 TFT VSN: IN (full VSN extension)
- E.2 GRN: IN if A.5 audit finds absent (otherwise mark existing as canonical)
- E.3 Multi-quantile heads: IN (5/25/50/75/95 quantile decomposition)
- E.4 Encoder-decoder: IN (extends D.3 to full trunk/head separation)
- E.5 Interpretability: IN (attention-weight ISV diagnostics)
- E.6 Auxiliary heads: IN (next-return + regime-classification)
- xLSTM: OUT (redundant with Mamba2+TLOB; YAGNI)
- KAN: OUT (function approx not a bottleneck)
No "evaluate later" items. Every concept either lands or explicitly
doesn't, with rationale.

§8 decisions tightened:
- D.8 TLOB mode: decision criterion = per-step cost benchmark ≤ 5ms
- C.6 controller order: fixed in spec (atoms→gamma→Kelly→cql_alpha→
  tau→epsilon→conviction_floor→plan_threshold→balancer)
- A.3 resource plan: auto-switch L40S→H100 at 12 GPU-hour threshold
- A.4 metric band init: [mean − 3σ, mean + 3σ] from last 3 good runs

No discretionary "we'll see" remain. All decisions data/order/budget/
statistic-driven.

Counter updates: "seven invariants" → "nine" in §3 header and §5
cleanup contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-24 09:32:22 +02:00
parent 3a62e23056
commit 336ee40b9d

View File

@@ -51,7 +51,7 @@ Each milestone has explicit exit criteria. Implementation does not declare "done
## 3. Cross-cutting invariants (non-negotiable)
Every feature added by this spec obeys all eight invariants. Violations are caught by pre-commit hooks and smoke tests.
Every feature added by this spec obeys all nine invariants. Violations are caught by pre-commit hooks and smoke tests.
### Invariant 1 — ISV-driven adaptive bounds
@@ -120,6 +120,20 @@ Rationale: raw indices are the other half of the "hardcoded tuned constant" prob
Enforcement: audit pass during A.1 (State Reset Registry) and A.2 (ISV Contract) picks up and names every raw index. New constants added to `docs/dqn-named-dims.md` — a simple table of name → index → purpose — and lint via grep: any `*.rs` or `*.cu` file that accesses a slot by raw index outside the definition module flags a pre-commit warning.
### Invariant 9 — No deferred work, no stubs, no TODO/FIXME
Every commit lands complete. No stubs (functions that return placeholder values instead of doing the work), no `TODO`, `FIXME`, `XXX`, `HACK`, or `TBD` markers in production code, no "we'll fix this later" deferrals, no half-finished implementations.
**If it can't be finished in this commit, it doesn't start in this commit.** Either the feature is done (behaviourally correct, wired, tested, diagnostic-emitting) or the feature doesn't land. No partial state.
Why this matters for a GPU RL trainer specifically: stubs that silently return placeholder Q-values, TODO-marked reward components that return zero, FIXME'd gradient paths that skip computation — these **train the network on semantic emptiness**. The model "learns" that certain inputs don't matter because the broken path reinforces ignoring them. Hours of L40S GPU time are burned optimising against a training signal that's partly fake. The bug is often invisible in metrics because training "converges" to something — just not to the policy we wanted.
Authority: `feedback_no_stubs.md`, `feedback_no_todo_fixme.md`, `feedback_no_quickfixes.md`. This invariant elevates those three memory rules to first-class spec enforcement.
Enforcement: pre-commit hook greps added code for `TODO|FIXME|XXX|HACK|TBD|unimplemented!|todo!` — any match rejects the commit. Stub detection is partly linted by the existing hook (`grep -n "return 0.0\|return 1.0\|return 0.5"` with `// ok: <reason>` suppression); extended to cover new fallback patterns as they emerge.
**Applies to spec implementation commits only.** Legit open questions in the spec itself (§8) are fine — they are design-time decisions documented with their decision criteria, not implementation-time stubs.
---
## 4. Architecture
@@ -503,26 +517,28 @@ Wiring:
State-layout impact: adds D_tlob dimensions. Schema version bump (A.2) + state_layout.cuh coordinated change with D.6, D.3.
### Part E — Supervised→DQN Concept Audit
### Part E — Supervised→DQN Concept Adoption
Systematic review of ml-supervised concepts the project has already built, with classification for DQN adoption.
Concrete decisions (per Invariant 9, every E item either lands or does not — no "evaluate later"):
| Concept | Current DQN use | Adoption in spec |
|---|---|---|
| TFT Variable Selection Network (VSN) | Partial (`vsn_mag`, `vsn_dir` masks) | E.1: full TFT VSN over state features |
| Gated Residual Network (GRN) | Audit needed | E.2: if not present, adopt as trunk building block |
| TFT multi-quantile heads | Partial (IQN CVaR) | E.3: full quantile decomposition for risk branch |
| TLOB attention encoder | Orphan to DQN | D.8 (wired) |
| Mamba2 forward | Wired | — |
| Mamba2 backward | Partial | D.1 (wired) |
| Liquid Time-constant | Unclear | D.7 (audited) |
| xLSTM | Orphan | E.4: evaluate for long-horizon memory augmentation |
| KAN | Orphan | E.5: evaluate for function-approximation in small-data regime |
| Encoder-decoder separation | Single trunk | E.6: evaluate separating state-encoder from value-decoder |
| Attention-weight interpretability | Partial | E.7: ISV-expose attention diagnostics from all temporal heads |
| Multi-task auxiliary heads | None | E.8: add Q + next-return-prediction + regime-classification heads |
| Concept | Current DQN use | Decision | Action |
|---|---|---|---|
| TFT Variable Selection Network (VSN) | Partial (`vsn_mag`, `vsn_dir` masks) | **IN** | E.1 — extend existing masks to full TFT VSN over all state feature groups |
| Gated Residual Network (GRN) | Audit result TBD during A.5 | **IN if absent** | E.2 if A.5 finds GRN not used in trunk, adopt as trunk building block; if present, mark existing impl as canonical in the audit |
| TFT multi-quantile heads | Partial (IQN CVaR only) | **IN** | E.3 — extend IQN branch to explicit multi-quantile decomposition (5/25/50/75/95 quantiles) |
| TLOB attention encoder | Orphan to DQN | **IN** | D.8 wires to DQN trunk |
| Mamba2 forward | Wired | — | already wired |
| Mamba2 backward | Partial | **IN** | D.1 completes |
| Liquid Time-constant | Unclear | **Audit-then-IN-or-DELETE** | D.7 decides; no third path |
| xLSTM | Orphan | **OUT** | Redundant with Mamba2 (state-space) + TLOB (attention); adding a third temporal encoder violates YAGNI. Leaving the supervised-crate impl as-is; no DQN wiring. |
| KAN | Orphan | **OUT** | Function-approximation is not a current bottleneck; Q-head dueling + C51 atoms already give sufficient expressivity. Leaving the supervised-crate impl as-is. |
| Encoder-decoder separation | Single trunk | **IN** | E.4 — D.3 (horizon-decomposed V) is a form of this; E.4 extends by separating the shared trunk into "state encoder" + per-branch "value decoders" with explicit interface |
| Attention-weight interpretability | Partial | **IN** | E.5 — ISV-expose attention weights from Mamba2, TLOB, and trunk heads; new diagnostic slots `ATTN_*_FOCUS_EMA` |
| Multi-task auxiliary heads | None | **IN** | E.6 — add auxiliary regression heads to trunk: next-bar return prediction + 5-bar regime classification. Trained via joint loss; gradients flow to trunk only (heads themselves are lightweight) |
Some E items ("evaluate") are investigation sub-tasks producing short design notes that feed follow-up PRs. Others ("wired") are concrete engineering work landing in this spec's rollout.
**Explicit OUT items (xLSTM, KAN):** the supervised-crate implementations remain in place for ml-supervised use. They are NOT orphans to this spec's scope because they serve their supervised consumers. The Wire-It-Up invariant applies to consumer paths per crate; xLSTM and KAN having zero DQN consumers is the correct state.
E.1, E.2, E.3, E.4, E.5, E.6 are all in scope for this spec. Not "evaluate" — concrete engineering. Each lands wired, diagnostic, and fully finished per Invariant 9.
Deliverable: `docs/ml-supervised-to-dqn-concept-audit.md`.
@@ -539,7 +555,7 @@ Deliverable: `docs/ml-supervised-to-dqn-concept-audit.md`.
### Cleanup contract
- Every commit landed on main preserves all seven invariants.
- Every commit landed on main preserves all nine invariants.
- Worktrees are removed after merge. No abandoned worktrees.
- Orphan modules discovered during implementation are resolved in the same commit that discovers them (wired or deleted).
@@ -629,12 +645,19 @@ Pre-commit hook rejects component-adding commits that don't touch the relevant a
---
## 8. Open questions (to resolve during implementation)
## 8. Decisions deferred to implementation (with decision criteria)
- D.8: TLOB end-to-end trainable vs frozen-with-side-stream retraining. Decide based on per-step cost benchmark during D.8 impl.
- E.4, E.5, E.6: xLSTM / KAN / encoder-decoder — each produces a short design note; adoption decided per-note, not in this spec.
- C.6: ordering of individual `AdaptiveController` migrations — atoms-first or gamma-first? Decide at the start of C.6's internal commit series based on which has the simplest ISV surface area.
- A.3 multi-seed: N = 5 seeds may be too expensive for L40S budget; may need to run the production-readiness validation on H100 pool instead. Resource plan finalised with A.3 impl.
Per Invariant 9, this spec has no "evaluate later" work items. The items below are **bounded decisions** — the design path is fixed, only the parameter point is chosen at implementation time with an explicit decision criterion written into the spec.
- **D.8 TLOB training mode** — decision criterion: benchmark a single training step with TLOB end-to-end vs TLOB frozen. If end-to-end cost per step ≤ 5ms incremental over baseline, use end-to-end. If > 5ms, use frozen-with-side-stream (retrain TLOB via supervised job every N epochs on future-return prediction). No third option. Decision made in the D.8 commit message based on measured benchmark data.
- **C.6 controller migration order** — decision: migrate in order of increasing ISV surface area (number of slots the controller reads/writes). Concretely: `atoms``gamma``Kelly cap``cql_alpha``tau``epsilon``conviction_floor``plan_threshold``balancer`. This order is now IN the spec; not deferred.
- **A.3 multi-seed resource plan** — decision: default to L40S pool. If `N=5, K=6` total budget exceeds 12 GPU-hours (measured on the first multi-seed run), migrate to H100 pool (2-3× faster). The switch is automated via `argo-train.sh` based on measured budget. No human intervention.
- **A.4 metric bands initial population** — decision: populate from the most recent 3 successful training runs' epoch-by-epoch metric distributions. Band = `[mean 3σ, mean + 3σ]` per metric. Re-fit after every promotion of a new baseline run.
All four decisions are concrete enough that implementation agents execute them without further design input. The decision criterion is data-driven (D.8), order-based (C.6), budget-based (A.3), or statistical (A.4) — no discretionary "we'll see" choices.
---