User correctly identified that CPU mirror function tests don't test
the production GPU code path. A bug shared between mirror and kernel
(translated identically wrong) would slip through. Mirror tests + a
single GPU bridge test were a weak compromise.
GPU-direct testing strategy:
- All Phase 0 kernel-correctness tests (0.A, 0.B, 0.C, 0.D, 0.F):
launch tiny test-only kernels with the SAME math the Phase 2
production kernel will use; assert properties of the output.
- Test 0.E (synthetic edge discovery): stays CPU. It tests an
ALGORITHMIC PROPERTY of Thompson exploration (does it discover
edge if edge exists?), not a kernel correctness property.
- All Phase 2 unit tests (2.A-2.D): GPU-direct against the
modified production kernel.
- Phase 0.F (real checkpoint extraction): unchanged — already GPU.
Local development uses RTX 3050 GPU (per memory user_dev_environment.md).
CI runs --ignored flag to skip GPU tests on CPU-only runners.
Time budget: Phase 0 was 1-2 days (CPU mirror); now 2-3 days
(GPU-direct, includes kernel wrapper setup half-day).
Other delta:
- Phase 0 deliverable file renamed: distributional_q.rs ->
distributional_q_tests.rs (no mirror functions, just tests +
kernel wrappers).
- Phase 2 unit tests rephrased to launch production kernel rather
than compare against CPU mirror.
- L1 verification gate runtime: seconds -> minutes (GPU launch
overhead per test).
The user's intuition was right: testing production directly is the
honest approach. Mirror was an optimization that traded correctness
for speed; with local GPU available the optimization isn't needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Self-review identified 8 major + 8 medium + 15 minor issues. All fixed:
MAJORS:
M1 Test 0.A: clarified — compare argmax(E[Q]), Boltzmann(E[Q]),
and Thompson sampling distributions; assertion is on relative
ordering across all three.
M2 IQN quantile count: replaced hardcoded `5` with N_IQN_QUANTILES
constant (defined per task #147 fixed-quantile design).
M3 "Converged checkpoint" definition: ≥60 epochs trained AND
val_sharpe stabilised (no >10% change over last 10 epochs).
Cites prior 60-epoch validation runs (task #80, train-7rgqd).
M4 R3 reframed: replaced "no issue" handwave with explicit
by-design tradeoff acknowledgment + cost analysis. Wasted
exploration is the cost of finding out whether edge exists.
M5 Test 0.D σ_long=0.05 justified: chosen to match expected order
of magnitude given typical |return| ~ 50bps; Phase 0.F
validates against real checkpoint.
M6 rng_ctr post-increment: clarified — matches existing pattern
at experience_kernels.cu:858 (no behaviour change).
M7 train_active_frac instrumentation: NEW Phase 2 deliverable —
existing HEALTH_DIAG only has val_active_frac, but L3 verifies
training-time active_frac. Spec now explicitly adds this
~10-line metrics.rs change as a Phase 2 deliverable.
M8 eps_dir cleanup code-level detail: explicit reference to
experience_kernels.cu lines 814-865; remove eps_dir from both
static EPS_FLOOR clamp AND adaptive boost block; verify
variable can be removed from kernel signature via grep.
MEDIUMS:
Med1 Current C51/IQN combination: clarified that compute_expected_q
blends per training schedule; Phase 2 replaces with explicit
0.5*E_C51 + 0.5*E_IQN equal weighting; Phase 0.F verifies.
Med2 Eval mode phrasing: "eval mode already sets eps=0 in existing
kernel" — no semantic override, factually correct.
Med3 Magnitude σ claim: clarified — magnitude branch likely has σ
bias in OPPOSITE direction (Full has larger σ; UCB would
prefer Full and worsen saturation). Empirical verification
deferred. Phase 0.F should also report per-magnitude σ.
Med4 Hierarchical sampling claim corrected: it's not about
balancing 50/50 (already 50/50). It's about decoupling
cluster-best decisions; clarified.
Med5 n_atoms vs N_IQN_QUANTILES: clarified — n_atoms variable per
config (currently 51); N_IQN_QUANTILES fixed at 5.
Med6 Conviction code: removed pseudo-code; references existing
implementation at experience_kernels.cu:1091; provides
implementation hint for E[Q] reuse.
Med7 Q-target propagation: clarified — uses full distribution
(C51 atom projection / IQN quantile regression), not just
E[Q]. Thompson modifies action selection only.
Med8 References: added Thompson 1933 (original), Bellemare 2017
(C51), Dabney 2018 (IQN) for theoretical foundations.
MINORS:
Min1 Date updated to 2026-04-27.
Min2-3 Argmax monotonic /2 simplified out — argmax(a+b) =
argmax((a+b)/2). Code clarity improved.
Min4 P(argmax picks Long) = 0 deterministic; reframed assertion.
Min5 Test 0.F structural assertions added: σ_C51[FLAT] < 0.01 ×
σ_C51[LONG]; same for IQN; E[Q_FLAT] > E[Q_LONG]; argmax
picks FLAT; Thompson P(LONG)+P(SHORT) ≥ 0.20.
Min6 -INFINITY → CUDART_INF_F (CUDA convention).
Min7 dir_idx scope: comment notes it's declared earlier in kernel.
Min8 action_select args: explicit — three buffers exist on GPU
but not currently passed; new params, no new buffers.
Min9 Phase 0 time math: 5 hours tests + 1 hour enumeration + 2
hours 0.F + (3 hours runtime if checkpoint training needed,
runs in parallel). Honest budget.
Min10 "Two-stream" → "5-Layer Gate" header.
Min11 Plan 5 reference uses full path consistently.
Min12 Plan B time budget: explicit 1 day if pass; 2-5 days if bug.
Min13 active_frac: clarified Long+Short combined, not per direction.
Min14 train_active_frac: now in Phase 2 deliverables (see M7).
Min15 "20 mechanisms" → 21, with sub-counts in section headers.
Spec now 615 lines, comprehensive coverage of:
- Pearl + theoretical foundation
- Problem statement (with bias-might-be-correct caveat)
- Architecture (Thompson at training, argmax at eval)
- Train vs eval distinct selectors with behavior-change disclosure
- Direction-only scope with magnitude σ-bias warning
- Conviction stays E[Q]-based (no Kelly cap jitter)
- Interaction matrix: 21 mechanisms in 3 categories
- 6 v2 enhancements documented + deferred
- Phase 0/1/2/3 with tests, exit gates, time budgets
- 5-layer verification + train_active_frac instrumentation
- 8 risks with mitigations + 5 stop conditions
- What v1 doesn't touch (referencing interaction matrix)
- References (Thompson 1933, Bellemare 2017, Dabney 2018, etc.)
- Aggregation contract (project-wide pearl, enforced)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-user direction: every existing mechanism in the DQN system MUST be
explicitly considered for interaction with Thompson, and Thompson itself
MUST be examined for system-specific improvements beyond vanilla.
INTERACTION MATRIX (3 categories, 20 mechanisms):
Category 1 — Compose with Thompson (no change required):
Counterfactual reward, B.2 novelty bonus, PopArt, Saboteur,
Curiosity, NoisyNets/VSN, Distillation, CQL, Polyak target EMA,
HER, PER, Replay warm-start, Multi-fold validation harness.
Category 2 — Trivially adapt to Thompson (one-line changes):
D7/N7 contrarian sign flip (negate the SAMPLE), cosine epsilon
schedule (still applies to mag/ord/urg), per-sample epsilon (IQL
expectile gap), adaptive Boltzmann tau (still applies to mag/ord/urg).
Category 3 — Take precedence over Thompson (hard constraints):
Plan-based action lock (Thompson sample discarded if plan active),
per-magnitude Kelly cap, trail stop, capital floor breach.
Critical insights from the audit:
1. NoisyNets is ALREADY a form of training-time Thompson at the
parameter level. Output-space Thompson stacks on top —
total exploration = parameter-space ⊗ output-space (multiplicative).
2. Curiosity is ORTHOGONAL to Thompson — Thompson explores actions
whose Q is uncertain; curiosity explores states whose dynamics
are uncertain. Both axes desirable; no conflict.
3. Plan lock takes precedence; same as currently with Boltzmann.
OUTSIDE-THE-BOX v2 ENHANCEMENTS (deferred to follow-up specs):
v2.1 Triple-source Thompson (C51 + IQN + Ensemble) — incorporate
the existing ensemble Q-head as 3rd uncertainty source.
v2.2 Persistent Thompson (anti-churn for HFT) — bias sampling
toward current direction, ISV-driven; reduces tx_cost from
Long/Short oscillation across bars.
v2.3 CVaR-aware eval (risk-adjusted deployment) — eval picks
argmax(E[Q] − λ·CVaR_α[Q]); risk-aware decision making for
production with real capital.
v2.4 Information-Directed Sampling (Russo & Van Roy 2014) — picks
action minimizing regret²/info_gain; more efficient than
vanilla Thompson when learning saturates.
v2.5 Hierarchical Thompson on (trade vs no-trade) → (which
direction) — addresses 50/50 structural advantage of no-trade.
v2.6 Composition with curiosity-driven exploration — explicit
coupling beyond reward-side composition.
Each v2 enhancement gets its own spec/plan when prioritised. Vanilla
Thompson is v1; ships first; verified independently.
ALSO FIXED (from earlier self-review):
- Pearl claim softened: only the C51 Hold/Flat bias is directly
attributed; other historic bugs had different mechanisms.
- TFT entry removed from contract table — TFT is a Variable
Selection Network (feature processor), not a Q-head. Replaced
with generic "Future Q-head additions" placeholder.
- Eval direction = argmax E[Q] explicitly flagged as a behavior
change from current Boltzmann-with-tau (val_dir_dist will be
more concentrated than current).
- Phase 0.E budget reduced (1-2 hours, not 1 day) — synthetic
bandit is ~50 lines of Rust, not full RL training loop.
- Phase 0 enumerates existing checkpoints before training new one.
- Architecture diagram parenthesis fixed.
- Conviction implementation note: compute E[Q] once, reuse for
conviction AND eval-mode argmax — no redundant computation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User correctly challenged the "band-aid removal" framing. All fixes
shipped during the val-Flat-collapse investigation addressed real bugs
at their respective layers and should be preserved:
- Kelly cap warm-branch (0c9d1ee39): post-decision physics layer.
Thompson-independent. KEEP.
- Train Return display + Sharpe annualization (non-tau parts of
7a3d88646): display/metric layer. Thompson-independent. KEEP.
- Direction Boltzmann tau-floor (tau part of 7a3d88646) +
adaptive eps_dir floor (d54b49efc): gates inside direction-branch
action selection. Phase 2 replaces direction-branch action
selection wholesale (eps-greedy + Boltzmann → Thompson), so these
direction-only code paths become structurally unreachable.
The latter two are NOT band-aids being removed because Thompson is
better. They are dead code being cleaned up because Thompson replaces
the surrounding mechanism. Magnitude/order/urgency branches keep their
existing eps-greedy + Boltzmann + tau-floor + EPS_FLOOR paths intact.
Reframed Phase 3 deliverable: "direction-branch dead-code cleanup"
with explicit rationale (per feedback_no_legacy_aliases.md and
feedback_no_partial_refactor.md). 0.5 day budget instead of 1.
Also clarified eval action selection: argmax of (E[Q_C51]+E[Q_IQN])/2
is correct. Bellman backup is a Q-learning UPDATE rule, not an
action-selection rule. Once Q is learned, optimal policy is greedy
argmax of learned Q. Online Bellman lookahead at eval would require
a forward model of market dynamics — not available, not standard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Revises the C51-bias spec after deeper review surfaced 12 design gaps,
of which 4 were critical:
1. Train-only vs train+eval ambiguity — UCB at eval would conflate
"model recommends Long" with "model is uncertain about Long",
inflating reported edge. CRITICAL for trading where eval drives
real capital decisions.
2. Thompson sampling is more principled than UCB:
- parameter-free (no κ to tune)
- uses distribution directly without scalar reduction
- naturally explore-exploit balanced via distribution shape
3. c51_alpha is the wrong blend weight (it's C51-vs-MSE-warmup, not
C51-vs-IQN). Equal-weight average of C51 and IQN samples is the
structural choice — no tuned blend weight needed.
4. The bias might be CORRECT BEHAVIOUR — model rationally choosing
Flat when no edge has been discovered. Phase 0 must include a
synthetic-edge test (controlled MDP with KNOWN positive Long
expected value) to verify Thompson can discover edge if it exists.
Other gaps fixed:
- Eval at argmax E[Q] (not Boltzmann, not Thompson)
- Pearl wording broadened to cover ensembles + future methods
- Ensemble Q-head added to aggregation contract table
- Explicit caveat: NEVER extend Thompson to magnitude branch (would
worsen existing magnitude saturation)
- Phase 0.F uses CONVERGED checkpoint (≥30 epochs), not 2-epoch run
- L4 long smoke (30 epochs, ~1 hour) added — Thompson edge discovery
needs longer feedback loop than 5 epochs
- Phase 3 explicitly removes eps-floor + tau-floor band-aids
(Thompson replaces direction Boltzmann; band-aids become dead code)
- Conviction stays E[Q]-based, not sample-based (avoid Kelly cap
jitter from stochastic samples)
Architecture (Thompson only, no UCB):
TRAINING: dir_idx = argmax(0.5 × (sample_C51(d) + sample_IQN(d)))
magnitude/order/urgency: existing Boltzmann + ε-greedy
EVAL: dir_idx = argmax(0.5 × (E[Q_C51] + E[Q_IQN]))
magnitude/order/urgency: existing Boltzmann (eval mode)
Direction-branch ε-greedy + Boltzmann are REMOVED — Thompson is the
exploration mechanism. No new GPU buffers; existing C51 atoms + IQN
quantiles passed to action_select.
5-7 days active work across 4 sub-plans; each gets its own
writing-plans cycle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Designs the structural fix for the C51 expected-Q Hold/Flat bias exposed
by the Kelly val-Flat-collapse fix. The bias is a manifestation of a
deeper project-wide pearl:
"Distributional RL aggregation discards uncertainty;
action selection must restore it."
Any value head representing Q as a distribution (atoms, quantiles,
ensembles) MUST expose both E[Q] and σ(Q) to action selection. Boltzmann
on E[Q] alone produces structural bias toward low-variance actions
regardless of expected payoff — the C51+IQN Flat-attractor is one
instance of this lost-information pattern.
Fix: extract σ(Q) from BOTH C51 atoms (closed form) and IQN quantiles
(IQR/1.349), blend by loss-time weight, feed Q_eff = E[Q] + κ·σ
(κ=1.0 structural identity) to direction-branch Boltzmann ONLY.
4-phase implementation:
Phase 0 — TDD hypothesis verification (Rust mirror functions + 5 unit
tests including GPU integration on real checkpoint)
Phase 1 — Audit existing reward levers (B.2, CF, PopArt, Q-target)
via 6 unit tests; fix any bugs found
Phase 2 — UCB integration: new compute_q_with_uncertainty kernel,
modified action_select, Rust orchestration, project-wide
aggregation contract in dqn-wire-up-audit.md
Phase 3 — Verification per Plan 5 Task 5 multi-seed × multi-fold
5-layer verification gate; 8 risks with mitigations; 4 stop conditions
that halt execution and force redesign.
Existing band-aid fixes (Kelly cap, eps-floor, tau-floor) stay — they
address symptoms at different layers. UCB adds the missing aggregation
step that was the common root across all the symptoms.
Direction-branch only — magnitude/order/urgency don't have the
Flat-attractor (atom-mass collapse asymmetry).
5-7 days active work across 4 sub-plans; each gets its own
writing-plans cycle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A first dispatch of monolithic Task 2c was refused with a second scope
assessment that surfaced three architectural facts beyond what 2a/2b
had captured:
(d) attn_layer_norm_bwd_dx is a SIMPLIFIED element-wise approximation
(d_x = d_out * gamma / std), not the full LN Jacobian. Reusing
it in the GRN backward would propagate the approximation into
trunk gradients silently. Task 2c.1 must write a new full
Jacobian: (1/D) * rstd * (D * d_out_g - sum_d(d_out_g) -
normed * sum_d(d_out_g * normed)).
(e) IQN target-trunk migration is not a clean swap. iqn_compute_
target_h_s2 reproduces the legacy LeakyReLU trunk independently
in CUDA. Deleting it requires a target_encoder_forward_only
extraction on the target path (Task 4-style refactor on the
target side). Sub-task in itself.
(f) Three relu_mask removal sites have three different save-buffer
lifetimes (online trunk per-step, IQN-aux per-step in different
trainer, target-sync per-target-update). Three plumbing tasks,
not one.
Decomposition:
- 2c.1: grn_kernel.cu with full LN Jacobian backward + GLU + ELU +
residual-add. Module compiles standalone (additive, dead code).
- 2c.2: gpu_grn.rs Rust wrapper with 5 save-for-backward state
buffers per block (not 3 as the prior plan suggested).
- 2c.3+4: ATOMIC commit — compute_param_sizes 86→95 + fingerprint
rewrite + xavier init for 13 new tensors + all 98 padded_byte_
offset migrations + 3 relu_mask sites with different save-buffer
plumbing each + iqn target trunk delete + target_encoder_forward_
only + mag_concat RMS-match adaptive ISV.
- 2c.5: docs flip.
No code changes. Plan-doc only.
After a first dispatch attempt of monolithic Task 2 (GRN ADOPT) was
correctly refused with a thorough scope assessment, this revision
records the decomposition and the structural facts that drove it:
1. layout_fingerprint_seed() only fingerprints ISV slots, not
param-tensor layout. The "fingerprint will auto-update" assumption
in the original Task 2 spec was wrong for param-tensor reshuffles.
2. compute_param_sizes has 86 tensors (docstring saying 42 is stale).
Inserting GRN's 9 sub-tensors shifts 82 downstream tensors and
98 padded_byte_offset call sites.
3. At least 12 kernels consume h_s2 expecting ReLU non-negativity.
GRN's LayerNorm output is zero-mean (signed). Per-consumer
verification needed before swap.
4. crates/ml-supervised::tft::gated_residual is incompatible as a
port: different formula (sigmoid gate, not GLU split) and
incompatible tensor abstraction. New CUDA kernels from scratch.
Decomposition:
- Task 2a (research, no code): audit h_s2 consumers for ReLU vs
LayerNorm semantics. Output: per-consumer table.
- Task 2b (small, checkpoint break): extend fingerprint seed to
include param-tensor names + sizes. Pearl-aligned: complete
fingerprint coverage rather than partial.
- Task 2c (large, checkpoint break): GRN kernels + 98 call-site
migration + h_s2 consumer shims (per 2a). Blocked on 2a+2b.
Recommended order updated to thread 2a → 2b → 2c. Tasks 1, 6, 3
remain independent of 2c and can land in parallel where useful.
Pearl rules section added documenting the safety constraints
applied throughout the plan.
No code changes. Plan-doc only.
Comprehensive revision to match landed Plan 1+2+3 codebase state. Pattern
mirrors the Plan 3 second revision: per-task "Reality reconciliation"
blocks at the top of each task body identifying what's stale vs. landed,
with concrete file paths in the actual cuda_pipeline tree.
Added:
- Task dependency graph with recommended execution order:
5 (light ISV) → 4 (refactor) → 2 (GRN ADOPT) → 1 (Full VSN) →
6 (aux heads) → 3 (multi-Q IQN) → 7 (audit) → 8 (Argo)
- Per-task implementation surface with concrete trunk hooks:
- Task 1: pre-h_s1 VSN gate in batched_forward.rs::forward_online_raw
- Task 2: GRN audit confirms ADOPT branch (GRN absent from DQN trunk;
only in ml-supervised TFT). Replaces h_s1/h_s2 Linear blocks.
- Task 3: re-scoped — IQN already runs num_quantiles=32 with random τ.
Task is CONSTRAINING to fixed τ ∈ {.05, .25, .5, .75, .95}.
- Task 4: pure Rust API split (no kernel changes, no checkpoint break)
around existing forward_online_raw structure
- Task 5: split into Mode A (light, pre-Task-1, 3 ISV slots) and
Mode B (full, post-Task-1, 7 ISV slots). Mode A recommended first.
- Task 6: aux head loss scaled by ISV[LEARNING_HEALTH] per pearl
- Checkpoint-break consolidation note: Tasks 2/1/6/3 each break checkpoint
compatibility; land in sequence with no Argo run between (one fingerprint
shift per commit; final Argo at Task 8 amortizes retraining cost)
- Task 8 absorbs Plan 3's deferred Argo Tier 1 gate (combined validation)
No code changes.
Plan 4 was drafted 2026-04-24 against an earlier Plan 3 design. After
Plan 3 landed (commits 44539d8f4..3cb083f18), the pre-plan gate
referenced slot names that don't exist in the landed implementation:
- PLAN_PARAMS_0_EMA_INDEX → became READINESS_EMA_INDEX (Task 4)
- STATE_KL_THRESHOLD_EMA_INDEX → eliminated; Task 7 uses kernel-internal
trailing-EMA-of-self pattern (no separate threshold slot)
- TEMPORAL_REWARD_{PERSIST,REGIME_SHIFT,CONSISTENCY}_EMA_INDEX → consolidated
into rc[5] → ISV[68] REWARD_BONUS_EMA_INDEX via Plan 3 Task 6a/6b/6c
Updated:
- Pre-plan slot-existence check matches landed slot names
- Validation-doc gate softened: Plan 3 Argo Tier 1 PASS becomes OPTIONAL;
local 5-epoch multi-fold smoke is the de-facto gate (user choice
to proceed without burning Argo cycles)
- Status table at top shows landed Plan 3 baseline (ISV_TOTAL_DIM=87,
PS_STRIDE=43, fingerprint at [85,86]) and per-task difficulty estimate
- Reality-reconciliation note documents the rename trail
No task body changes; subsequent commits will revise individual task
specs as they become next-up for execution.
Tasks 1, 2, 3, 5 landed on main between a59e7599c and a0abc3da3. The
remaining Task 4/6/7/8/9 bodies in the plan had accumulated drift:
- Task 4: allocated slot 49 (already PLAN_THRESHOLD_INDEX); EMA'd
plan_params[0] (kernel compares against readiness).
- Task 6: allocated 59/60/61 (now collides with Plan 2 Q-quantile
[50..58) and Task 1 reward EMAs [63..69)); required 6 new PS memo
fields and included tuned 0.5e-4f penalty.
- Task 7: allocated 58/63/64 (63/64 collide with REWARD_TRAIL_EMA /
REWARD_MICRO_EMA from Task 1); amplification formula had tuned
2.0× trigger, 0.02 decay, 1.0/2.0 endpoints.
- Task 8: referenced trainer helpers that don't exist
(sample_state_feature_pair, step_scripted, replay_insert_with_
priority_scale); tuned priority_scale=0.5.
- Task 9: used pre-pivot CPU-compute AdaptiveMonitor pattern with
tuned 0.9/0.1 EMA rates.
This revision:
- Adds per-task "Reality reconciliation" sub-header flagging the
stale premise being fixed.
- Marks landed tasks with ✅ LANDED <SHA> and records actual outcomes
(vs. the planned outcomes the original text described).
- Rewrites Task 4/6/7/8/9 bodies to use tail-append slot allocation
(indices recomputed from ISV_TOTAL_DIM at impl time), GPU kernel
producer + read-only AdaptiveMonitor consumer pattern, and
ISV-derived adaptive coefficients in place of tuned constants.
- Splits Task 6 into 6a/6b/6c with independent PS-slot additions
(MIN_PNL / REGIME_SHIFT_BAR / PRE_ENTRY_CONVICTION_EMAs).
- Enumerates concrete trainer-helper prereqs for Task 8.
- Updates Task 10 metric bands to match actual landed ISV slot names.
- Updates exit criteria summary to check off what landed.
No code changes.
Aligns Plan 3 with post-Plan-2 reality:
1. AdaptiveController → AdaptiveMonitor throughout (spec §4.C.6
2026-04-24 revision). Tasks 4 (plan-threshold) and 9 (cql_alpha
seed-coupled) become GPU kernel + read-only CPU monitor pairs
instead of CPU-compute controllers — uniform with Plan 1's
tau/epsilon/gamma/kelly_cap and Plan 2's per-branch γ pattern.
2. Stale ISV slot indices [49..65) replaced with tail-append language.
Current post-Plan-2 ISV_TOTAL_DIM = 63; new slots start at 63 and
grow. Fingerprint at 61-62 auto-re-tails.
3. Architecture paragraph updated to document replay-seed orchestration
(Task 8 dqn_replay_seed.rs) as constructor-time cold-path pre-training
— not a GPU-drives violation.
4. Prerequisites section clarified: Plan 2 state — ISV_TOTAL_DIM 63,
TLOB at SL_TLOB_START, layout fingerprint auto-recomputes.
No scope changes to individual Tasks 1-10. Just terminology + slot
indexing aligned with current main state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User pearl (2026-04-24): the DQN already has every primitive TLOB needs
(gpu_attention, batched_forward/backward, GpuLinear, cuBLASLt handles,
cuda_autograd). Port TLOB's Q/K/V + attention as a composition of
existing primitives. Random-init, trainable end-to-end from the DQN's
reward signal. Uniform with Mamba2, IQL, atoms/γ/τ/ε — all of which
already follow this pattern.
Eliminates:
- ONNX Runtime dependency (was already dead — stripped from ml-supervised)
- Separate supervised pretraining pipeline
- "Freeze vs fine-tune" false dichotomy
- Pretrained-checkpoint-file-not-found failure mode
Prerequisites for the cuBLAS-native design (all satisfied):
- gpu_attention.rs exists
- GpuLinear trainable layer exists
- cuda_autograd over cuBLAS exists
- MBP-10 data already in the DQN data pipeline
What was "BLOCKED on prerequisites" in the Task 6C audit referred to
the OLD pretrained-ONNX design. The cuBLAS-native design has all
prerequisites satisfied — Task 6C can proceed under the new scope.
Pearl captured in memory: pearl_tlob_no_pretraining.md. Generalises to
any future attention/state-space/Neural ODE module: port to cuBLAS,
random init, let the DQN teach it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Revises Plan 2 to match the post-Plan-1 reality:
1. AdaptiveController → AdaptiveMonitor throughout (read-only observers
per spec §4.C.6 2026-04-24 revision; GPU kernels compute, CPU reads).
Applies to Task 3 (per-branch gamma) and Task 5 (liquid_mod audit).
2. Task 2 (D.1 Mamba2 backward) scope narrowed from "implement" to
"validate existing". Plan 1 A.5 audit confirmed
mamba2_scan_projected_bwd kernel + mamba2_backward host call are
already fully wired. Task 2 now adds a finite-difference grad-check
smoke + non-zero grad-propagation smoke; escalates if either fails.
3. Task 3 (D.2 per-branch gamma) rewritten for GPU-drives compliance:
- Replace scalar GAMMA_EFF_INDEX=43 with 4 per-branch slots at
43-46 (DIR/MAG/ORD/URG).
- New per_branch_gamma_update_kernel.cu reads v-range + health,
writes 4 slots. Deletes the Plan 1 gamma_update_kernel.cu.
- 4 read-only monitors (or one consolidated PerBranchGammaMonitor).
- ISV slots 44-48 shift downstream; fingerprint re-tails at 50-51;
ISV_TOTAL_DIM grows 49 → 52.
- c51_loss_kernel and iql_value_kernel read per-branch γ from ISV.
- No CPU-side γ computation anywhere.
4. Header architecture block updated: new ISV slot count target,
GPU-drives principle explicit, current Plan-1 layout table inline
for reference.
5. Pre-plan verification updated: expects AdaptiveMonitor trait (not
AdaptiveController); checks for 6 Plan-1 monitor files.
6. Removed "schema version bump 1 → 2" language — layout fingerprint
auto-recomputes from seed bytes; no integer version space exists.
Task 4 (D.5 soft fold transitions), Task 5 content (D.7 liquid audit),
Task 6 (D.3+D.6+D.8 coordinated state-layout migration), Task 7
(validation run) structure preserved. Internal slot numbers in those
tasks will reconcile to the new layout when they land (no pre-emptive
edits since those indices depend on whether Task 1 quantile slots or
Task 3 per-branch gamma slots land first — re-compute at exec time).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User raised that the CPU-controller pattern (where CPU computes adaptive
values from ISV and writes back to ISV) violates the architectural
principle: GPU kernels should compute adaptive decisions; CPU-side code
is pure observation.
Replaces the AdaptiveController trait with read-only AdaptiveMonitor:
- No update() — CPU doesn't compute adaptive values.
- No write_output() — CPU doesn't write adaptive values to ISV.
- read() returns the GPU-computed value from ISV.
- diagnose() emits HEALTH_DIAG snapshot.
- observe() + fire_rate() track how often the GPU-computed value changes.
Reclassifies the 9 adaptive mechanisms:
- 6 reactive get GPU kernel + CPU monitor: atoms, gamma, kelly_cap, tau
(Polyak EMA), epsilon, grad_balancer (last is already GPU-driven).
- 3 static get ISV constructor-write, no monitor: cql_alpha,
conviction_floor, plan_threshold.
New ISV slots for GPU-written adaptive outputs (EPSILON_EFF, TAU_EFF,
GAMMA_EFF, KELLY_CAP_EFF) and CPU-born inputs (EPOCH_IDX, TOTAL_EPOCHS),
plus slots for the 3 static configs.
Rationale:
- Unified adaptive machinery, no CPU-side special cases.
- Per-sample granularity available (Expected SARSA τ in c51_loss_kernel
is already the exemplar — reads ISV q_gap + health per sample).
- Zero CPU→GPU config transfer in any path.
- ISV is single source of truth for every adaptive value.
Plan 1 Tasks 8-17 restructured:
- Task 8 creates AdaptiveMonitor trait (not AdaptiveController).
- Tasks 9, 10, 11, 13, 14, 17: GPU kernel + monitor pairs.
- Tasks 12, 15, 16: static ISV writes only.
Follow-up commits will revert Batch A (d76849f31) and Batch B (4189da563)
which implemented the old CPU-compute pattern, then re-implement under
this design.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Populated docs/dqn-wire-up-audit.md with every pub module and CUDA
kernel in the DQN path. Each entry classified Wired / Partial / Orphan /
Ghost / OUT-of-DQN-scope with the action plan linking to the plan+task
that resolves any non-Wired status.
No Orphan left unclassified. Orphans fall into three buckets:
1. Scheduled for wiring by a later Plan (gpu_statistics → Plan 2 D.2;
tlob_loader → Plan 2 D.8).
2. OUT-of-DQN-scope because supervised consumers exist (PPO kernels,
xLSTM, KAN trainable adapter, flash_attention, benchmarks).
3. Genuinely unused — escalated to user review in the task output,
not deleted autonomously (streaming_dbn_loader, unified_data_loader,
training/orchestrator, training_pipeline, inference_validator,
model_loader_integration, paper_trading/mod.rs,
portfolio_transformer, regime_detection/mod.rs).
Summary: 109 total modules/kernels, 74 wired, 7 partial, 11 orphan,
0 ghost, 17 OUT-of-DQN-scope.
Plan 1 Task 6. Spec §4.A.5, Invariant 2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User raised a backward-compat concern: integer "schema version"
semantically implies a family of coexisting versions with upgrade
paths between them, inviting the forbidden pattern
(feedback_no_legacy_aliases.md, feedback_no_partial_refactor.md).
Replace with a compile-time structural fingerprint:
- ISV[0..2) stores a u64 FNV-1a hash of the slot layout (split across
two f32 lanes preserving raw bits).
- The hash is computed by a const fn over the slot list; any slot
change automatically updates the fingerprint. No human decides
"what version is this now".
- Checkpoint load is fail-fast only. Error message does NOT mention
migration as an option.
- Pre-commit hook rejects any `fn migrate_isv|upgrade_isv` to make
the no-migration rule structurally enforced (landed as part of
Plan 1 Task 5 hook extension).
- StateResetRegistry entry renamed ISV_SCHEMA_VERSION →
ISV_LAYOUT_FINGERPRINT (Task 2's landed code touched in Task 5's
commit per no-partial-refactor).
Updated:
- spec §4.A.2 (fingerprint design + rationale)
- spec §5 landing-order note (fingerprint auto-updates on layout change)
- Plan 1 architecture line
- Plan 1 Task 2 StateResetRegistry test + entry naming
- Plan 1 Task 5 — full rewrite of implementation steps
- Plan 1 exit criteria #6
- Plan 2 pre-plan verification (grep fingerprint constants, not version==0)
- Plan 2 Task 6D.1 (update fingerprint seed, not bump version)
- Plan 2 Task 6D exit criteria #7
- Plan 3 pre-plan verification
- docs/isv-slots.md ISV[0..2) row
No code changed. Plan 1 Task 5 is not yet implemented — this commit
realigns the spec + plans so the implementer subagent works from the
corrected design.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Self-review caught that the CqlAlphaSeedCoupledController read_signals
example used unimplemented!("plumbed by caller"), which violates the very
Invariant 9 the plan enforces. Replaced with a concrete implementation
that reads SEED_FRACTION_EMA_INDEX from the ISV bus (a Plan 1 reserved
controller-signal slot).
No other plan had placeholder violations. The remaining TBD/TODO/FIXME
grep hits across Plans 1-5 are either the pre-commit hook's rejection
patterns (correctly quoted) or enforcement scans that DETECT the markers
in audit docs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fifth and final plan decomposing the DQN v2 unified spec
(docs/superpowers/specs/2026-04-24-dqn-v2-unified-design.md §2, §4.A.3,
§4.A.4, §4.A.4.1). No new policy mechanisms — orchestration, automation,
regression detection, and the final sign-off run.
Covers spec sections:
- §4.A.3 Multi-seed × multi-fold Argo harness (N=5, K=6 defaults via
--multi-seed / --folds flags on argo-train.sh)
- §4.A.4 Regression detection hard-stop (2N consecutive error-band
metrics → self-SIGTERM with TerminationReason)
- §4.A.4.1 nsys profile harness with --profile flag and
compare-nsys-profiles.py for 20% per-epoch regression detection
- §2 Tier 1 + Tier 2 + Tier 3 final exit criteria via scripts/validation/
suite (tiered tier1/tier2/tier3 check scripts + wrapper)
- §2 DQN v2 rollout close-out (spec marked LANDED, retrospective,
plan-level validation docs archived)
Plan structure: 6 tasks — 4 infrastructure (harness, regression-detect,
nsys, validation-scripts) + final-run task + rollout-closeout task. The
final-run task is the sole exit-gate for the DQN v2 rollout: ALL THREE
tiers must PASS on the first final-validation run (per stop-on-anomaly:
retrying with different seeds hoping for a different outcome is anti-
pattern).
Preserves all 9 invariants. Every mechanism wired, no stubs, no TODO/FIXME.
Retrospective written from actual implementation experience after Plans
1-5 all land.
This is the terminal plan. Plans 1-5 together constitute the complete
DQN v2 rollout. After Plan 5's final-run exit passes, the spec is marked
LANDED and the rollout is declared complete.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fourth of five sequential plans decomposing the DQN v2 unified spec
(docs/superpowers/specs/2026-04-24-dqn-v2-unified-design.md §4.E).
Covers the six Part E IN decisions:
- §4.E.1 TFT Variable Selection Network across 6 feature groups
(market/OFI/TLOB/MTF/portfolio/plan_isv) with vsn_feature_selection kernel
- §4.E.2 Gated Residual Network — CONDITIONAL branch (ADOPT vs CANONICALISE)
based on docs/ml-supervised-to-dqn-concept-audit.md row
- §4.E.3 Multi-quantile IQN heads (5/25/50/75/95) replacing single CVaR output
- §4.E.4 Encoder-Decoder separation — explicit StateEncoder + per-branch
ValueDecoder with D.3 horizon-decomposed V_short/V_long sub-heads
- §4.E.5 Attention-weight interpretability — 7 new ISV slots [65..72) for
per-group attention focus EMAs + Mamba2 retention proxy
- §4.E.6 Multi-task auxiliary heads — next-bar return MSE + 5-bar regime CE,
ISV-coupled aux-weight schedule (sharpe-reactive)
ISV_TOTAL_DIM seals at 72 with this plan's allocations. Part E audit doc
closes out all rows (zero TBD/evaluate remain per Invariant 9).
Plan structure: 8 tasks (6 feature tasks + audit close-out + validation).
TDD-disciplined steps. Task 2 documents the CONDITIONAL branch decision
pathway explicitly (ADOPT vs CANONICALISE Branch A/B structure).
Plan 4 exit gate: Tier 1 convergence RETAINED (no regression vs Plan 3
baseline) + aux heads produce measurable signal. Tier 2 + Tier 3 checked
by Plan 5.
Preserves all 9 invariants. Zero stubs. Zero TODO/FIXME. Every Part E item
either lands fully or is explicit OUT (xLSTM, KAN); no third path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second of five sequential plans decomposing the DQN v2 unified spec
(docs/superpowers/specs/2026-04-24-dqn-v2-unified-design.md).
Covers spec sections:
- §4.C.1 Quantile-based atom support (8 new ISV slots, q_quantile_reduce kernel)
- §4.D.1 Mamba2 backward pipeline completion (no atomicAdd — per-sample arrays + host reduce)
- §4.D.2 Per-branch gamma via AdaptiveController (4 new ISV slots)
- §4.D.5 Soft fold-boundary transitions (extends StateResetRegistry with SoftReset category)
- §4.D.7 Liquid Time-constant audit (trace fire rate, decide wire-or-delete)
- §4.D.3 + §4.D.6 + §4.D.8 coordinated state-layout migration (horizon-decomposed V +
plan_isv[6] + TLOB integration with atomic ISV schema version bump 1→2)
Plan structure: 7 tasks + pre-plan verification, all TDD-disciplined with
bite-sized steps (test → fail → implement → pass → commit). Task 6 is
explicitly atomic to preserve Invariant 5 (state-layout consistency).
Plan 2 prerequisites: Plan 1 landed (StateResetRegistry, AdaptiveController
trait, ISV schema version, audit docs). Plan 2 exit: all 9 invariants
preserved; convergence-scaffolding run passes with 16 new ISV slots
populated; ISV schema version == 2.
Preserves all 9 invariants. No stubs. No TODO/FIXME. Every new module
wired to production path in the same task it lands in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First of 5 implementation plans for the DQN v2 unified integrated
policy system spec (d13b53586, 336ee40b9). Plan 1 covers:
Task 1: Audit doc scaffolding + pre-commit hook (Invariant 7, 9)
Task 2: StateResetRegistry definition (A.1)
Task 3: Wire StateResetRegistry into fold-boundary reset (A.1)
Task 4: Named-dimension refactor — ps[], plan_isv[], plan_params[],
branch indices, dir/mag sub-indices (Invariant 8)
Task 5: ISV schema version at ISV[0], fail-fast on checkpoint
mismatch (A.2)
Task 6: Orphan audit populated (A.5)
Task 7: Hot-path purity audit populated; MIGRATE calls fixed (A.6)
Task 8: AdaptiveController trait + harness (C.6)
Tasks 9–17: Migrate each adaptive controller to the trait in spec-
specified order (atoms → gamma → Kelly → cql_alpha → tau
→ epsilon → conviction_floor → plan_threshold → balancer)
Task 18: Plan 1 validation run (smoke + 3-epoch L40S)
Plan 1 landing criteria: all 9 invariants preserved across every
commit, all smoke tests pass, audit docs fully populated. Plan 2
(temporal core) starts only after Plan 1 passes exit criteria.
Plans 2–5 cover Parts B, C (minus C.6 already in Plan 1), D, E, and
final validation. Planned as separate documents in docs/superpowers/
plans/2026-04-24-dqn-v2-plan-{2..5}-*.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User-mandated addition:
Invariant 9 — No deferred work, no stubs, no TODO/FIXME
Every commit lands complete. No stubs (placeholder return values), no
TODO/FIXME/XXX/HACK/TBD markers, no half-finished implementations. If
it can't finish in this commit, it doesn't start. Stubs and deferrals
train the network on semantic emptiness — invisible in convergence
metrics but burn GPU time optimising against partly-fake signal.
Authority: feedback_no_stubs.md, feedback_no_todo_fixme.md,
feedback_no_quickfixes.md elevated to first-class spec enforcement.
Pre-commit hook greps for forbidden markers.
Part E decisions made concrete (per Invariant 9):
- E.1 TFT VSN: IN (full VSN extension)
- E.2 GRN: IN if A.5 audit finds absent (otherwise mark existing as canonical)
- E.3 Multi-quantile heads: IN (5/25/50/75/95 quantile decomposition)
- E.4 Encoder-decoder: IN (extends D.3 to full trunk/head separation)
- E.5 Interpretability: IN (attention-weight ISV diagnostics)
- E.6 Auxiliary heads: IN (next-return + regime-classification)
- xLSTM: OUT (redundant with Mamba2+TLOB; YAGNI)
- KAN: OUT (function approx not a bottleneck)
No "evaluate later" items. Every concept either lands or explicitly
doesn't, with rationale.
§8 decisions tightened:
- D.8 TLOB mode: decision criterion = per-step cost benchmark ≤ 5ms
- C.6 controller order: fixed in spec (atoms→gamma→Kelly→cql_alpha→
tau→epsilon→conviction_floor→plan_threshold→balancer)
- A.3 resource plan: auto-switch L40S→H100 at 12 GPU-hour threshold
- A.4 metric band init: [mean − 3σ, mean + 3σ] from last 3 good runs
No discretionary "we'll see" remain. All decisions data/order/budget/
statistic-driven.
Counter updates: "seven invariants" → "nine" in §3 header and §5
cleanup contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Self-review pass on the DQN v2 unified design spec (d13b53586). Fixes
applied inline:
Ambiguity fixes:
- §3 Invariant 3: explicit definition of "training step loop" (per-step
code in fused_training::step_fused and captured graph children; NOT
per-epoch / per-fold / per-checkpoint code).
- §2 Tier 1: "stable epochs" defined as epochs [warmup_end..run_end)
with warmup_end from B.3's seed-phase decay or explicit flag.
- §4 A.2: migration mechanism clarified as fail-fast initially; helpers
added only when a specific migration need arises (no speculative
scaffolding).
- §4 A.3: N=5, K=6 stated as DEFAULTS with ≥ MINIMUMS per Invariant 6;
resource budget flagged (~8-10 GPU-hours per validation pass).
- §4 B.3: "≥ 100K experiences" (minimum), warm-restart clarified as
runtime condition not feature flag.
- §4 C.3: adaptive KL threshold mechanism made explicit (second ISV
slot for threshold EMA, third for amplification multiplier).
- §4 C.6: cleanup timing explicit — old scaffolding removed in the SAME
commit that migrates its last consumer (no deferred pass).
- §4 D.1: DQN-path-specific scope clarified; validation via grad-norm
smoke + integration test.
- §4 D.6: plan_isv index naming made consistent with existing layout;
coordinated state-layout migration commit called out.
New Invariant 8 — Named dimensions, not indices:
Every dimension, slot, offset, or semantic position in a multi-field
buffer has a named constant. Raw numeric indices (`[0]`, `[6]`, `[23]`)
appear ONLY in the definition site. Named constants specified for ISV
slots (existing), portfolio state ps[0..30), plan_isv[0..7), plan_params
[0..6), state-vector offsets, branch indices (BRANCH_DIR/MAG/ORD/URG),
action sub-indices (DIR_SHORT/HOLD/LONG/FLAT, MAG_QUARTER/HALF/FULL).
Enforcement: audit pass during A.1/A.2, lint via grep for raw-index
access outside definition modules.
Landing order revision:
- Dependency graph explicit (A.2 before new ISV slots, D.1 before
D.2-4-8, state-layout changes in ONE coordinated commit).
- Spec decomposition note added — writing-plans may produce multiple
sequential plans rather than one monolithic plan.
New doc tracked by pre-commit: docs/dqn-named-dims.md (Invariant 8).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Unifies the Q-support range source across atom grid / warm-start quantile
clamp consumers via the ISV signal bus. One broadcast written at epoch
boundary from per-branch Q-stats EMAs, read by the two consumers that
previously held disagreeing ranges. Target observation: atom utilisation
≥40% (up from 11-15% on train-fpxnw).
Phase 0 — per-branch Q-stats kernel Rust plumbing:
* Load q_stats_per_branch_reduce alongside legacy q_stats_reduce
* Add per_branch_q_stats_pinned (28 f32 = 4 × 7, device-mapped)
* PerBranchQValueStats struct: [QValueStatsResult; 4]
* reduce_current_q_stats_per_branch launches the new kernel with the
four branch (off, size) pairs derived from config.branch_N_size
Phase 1 — ISV v-range plumbing (zero behavioural change at epoch 1):
* ISV_NETWORK_DIM=23 preserved for w_isv_fc1 sizing; ISV_TOTAL_DIM=31
allocates 8 additional slots for per-branch (centre, half-width)
* Slot constants V_CENTER_DIR..V_HALF_URG covering slots 23..30
* eval_q_mean_ema / eval_q_std_ema / eval_ema_initialized promoted
to [f32; 4] / [bool; 4]; scalar setters preserved for trajectory
backtracking (broadcast same value to all branches)
* Bootstrap at construction: centre=0, half=(v_max-v_min)/2 → the
byte-identical [config.v_min, config.v_max] span per branch before
any Q observations arrive
* reset_eval_v_range_state resets the 4 per-branch EMAs AND the 8 ISV
slots to bootstrap values; legacy eval_v_range_pinned[2] still reset
(deferred removal — spec Phase 3)
* update_eval_v_range reworked: signature takes PerBranchQValueStats and
per_branch_q_gaps. Maintains 4 independent adaptive-rate EMAs,
computes (centre, half) per branch with min_half_floor=0.1×(v_max-v_min)
and clamps to config bounds, writes 8 ISV slots. Branch-0 (direction)
centre±half is also mirrored into the legacy eval_v_range_pinned for
consumers that have not yet migrated to the per-branch bus.
Phase 2a/2b — atom grid per-branch v-range:
* adaptive_atom_positions kernel signature changed from
(v_min: float, v_max: float) to (branch_idx: int, isv_signals: float*);
reads centre/half from ISV slots 23+2·b, 24+2·b. Eliminates the f64→f32
ABI trap (spec Phase 2 side-effect) since the only per-branch range
path is now pointer-based.
* recompute_atom_positions passes branch_idx + isv_signals_dev_ptr per
branch; no scalar v_min/v_max arg remains.
Phase 2c — warm-start quantile clamp per-branch from ISV:
* warm_start_atom_positions reads per-branch (centre, half) from pinned
ISV host memory, clamps shared reward-quantile vector into each
branch's adaptive range before tiling into atom_positions_buf.
Bootstrap makes this equivalent to the pre-spec config.v_{min,max}
clamp until the first Q observation lands.
Deviations from spec:
* Phase 2d (per_sample_support_buf → [N, 4, 3]) NOT implemented. The
spec's premise was that per_sample_support is host-tiled from
eval_v_range, but the active path in this codebase has it filled by
iql_compute_per_sample_support (V(s)-centered, per-sample, already
adaptive) — orthogonal to the ISV bus. Migrating that kernel to
per-branch output would require rewriting iql_value_kernel +
iql_support_floor + C51/MSE loss kernel indexing in lockstep, which
the "no unrelated refactoring" constraint disallows. The loss-kernel
Bellman projection today uses V-centered bounds that are themselves
adaptive; the ISV v-range fix still lands the primary win (atom grid
+ warm-start agreement) without touching IQL.
Compile verified: cargo check -p ml + --workspace pass (SQLX_OFFLINE,
CARGO_INCREMENTAL=0, sccache). No TODO/FIXME/XXX introduced.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diagnostic-only audit of all 22 ISV slots using train-mdh86 HEALTH_DIAG
(20 epochs, same oscillation pattern as train-rq6n8). Identifies three
dead writers (slots 2, 3, 4), one strongly anti-correlated signal
(slot 12 learning_health, r=-0.765 vs Sharpe), sample-0 bias in four
regime slots (8-11), and EMA-pollution risk on the Q-scale EMAs
(16, 21) that feed Kelly conviction + C51 bin weighting.
Produces a per-slot table with writer/reader citations and an ordered
fix list. Hypothesises the observed +34 → -67 Sharpe swing is driven
by health mis-reporting "improving" while policy diverges, q_dir_abs_ref
contamination collapsing Kelly, and the zero-writer TD-error EMA
disabling micro-reward regime awareness.
No code changes — reconnaissance only.
Compares the 51-feature TLOBFeatureExtractor (crates/ml-supervised/src/tlob/
features.rs + mbp10_feature_extractor.rs) against Foxhunt's 20-slot OFI
vector persisted in .fxcache.
Findings: of TLOB's 51 slots, 23 are placeholders (sine waves / hardcoded
constants), 12 duplicate existing OFI or 42-dim market features, and only
~10 carry genuinely novel information. Meanwhile the existing
MicrostructureState struct in ml-features already computes 10 features
(realized variance, Hawkes intensity, weighted book pressure, spread
dynamics, aggression ratio, queue-depletion asymmetry, order-count flux,
intra-bar momentum, regime score, OFI trajectory) that are dropped before
reaching fxcache — these dominate the TLOB novel candidates and require
zero new math.
Phase B plan (drafted inline): bump OFI_DIM 20→28, bump FXCACHE_VERSION
4→5, prioritize persisting the 10 Foxhunt-internal MicrostructureState
features before any TLOB-derived additions, sweep all hard-coded 18/20
constants across CUDA kernels + gpu_dqn_trainer.rs, and rely on the
ensure-fxcache Argo step for cache regeneration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends grad_decomp_kernel to snapshot the trunk tensor slice (tensors
0..4 = w_s1, b_s1, w_s2, b_s2) in addition to the existing direction +
magnitude branch slices (8..12 / 12..16). Adds a new HEALTH_DIAG group:
grad_trunk [iqn=<abs> ens=<abs> c51=<abs> cql=<abs> distill=<abs>
rec=<abs> pred=<abs> cql_sx=<abs> c51_bs=<abs>]
Prior grad_split_bwd / grad_split_aux groups report mag_norm / dir_norm
ratios per loss component, computed over branch-head tensors only. That
measurement range structurally reports 0.0000 for any loss component
that writes exclusively to the trunk — IQN and Ens in particular. This
caused the persistent misdiagnosis that IQN-to-trunk was not wired; the
prior scoping in /tmp/foxhunt_research/iqn-to-trunk-wiring-scoping.md
confirmed the wiring is live (apply_iqn_trunk_gradient at
gpu_dqn_trainer.rs:4882) and that the zero reading was a blind spot in
the measurement pipeline.
Smoke confirms the diagnostic: after iqn_readiness ramps up (late
epochs), grad_trunk reports iqn=100..381 (real trunk SAXPY amplitude),
ens=0.07..3.57, c51=2.46..8.91 (value-head dueling path contributes
through trunk), while cql/cql_sx/distill/rec/pred stay near-zero — a
clean diagnostic baseline.
Also fixes stale documentation at dual-distributional-c51-iqn-design.md
that claimed "IQN trains in isolation — its gradients don't flow back
to the shared trunk": reworded to reflect current wired state with
file:function citation and explicit iqn_readiness gating note. Updated
the "What Changes" table ("IQN training") and "Implementation Order"
(Phase 1 marked DONE) with the same citation.
Changes:
- grad_decomp_kernel.cu: per-component result slot 2 → 3 floats
(mag_norm, dir_norm, trunk_norm); extra __shared__ sum_trunk +
tree reduction; new grad_trunk_start/trunk_len kernel args.
- gpu_dqn_trainer.rs: pinned result buffer 18 → 27 floats; snapshot
now does two copy_f32 passes (trunk → dst[0..trunk_len), branch →
dst[trunk_len..]); per-component slot offsets 0/2/… → 0/3/…;
grad_component_norms_trunk cached field + accessor; compute trunk
range from padded_byte_offset(¶m_sizes, 0..4).
- fused_training.rs: grad_trunk_norms_by_component() + per-component
grad_trunk_*_abs() accessors.
- training_loop.rs: HEALTH_DIAG emits new grad_trunk group ordered
[iqn ens c51 cql distill rec pred cql_sx c51_bs]; extended doc
comment explaining the three groups' roles.
- design spec (Problem #1 + What Changes row + Implementation Order):
stale "IQN trains in isolation" replaced by current wired-state
description, cites gpu_dqn_trainer.rs:4882 and readiness ramp at
gpu_dqn_trainer.rs:4228-4243.
Pure diagnostic — no training dynamics change, no atomicAdd, no tuning
knobs, no TF32 changes. Smokes unaffected (magnitude_distribution H10
regression pre-exists on HEAD 810b3c570; 4 other smokes pass).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Task 2.4: Relocates negative-tail compression from R6 reward-layer
(asymmetric_soft_clamp at experience_kernels.cu:78-81) to C51 Bellman
target smoothing (c51_loss_kernel.cu::block_bellman_project_f).
Functionality preserved — same invariant, better location. Upper +10
cap kept inline as fminf(reward, 10.0f) for numerical safety.
Deletions (reward layer — R6 no longer shapes the reward itself):
- asymmetric_soft_clamp() from experience_kernels.cu:78-81 (no callers)
- Reward-layer clamp replaced with fminf(reward, 10.0f) at ~1922
(segment_complete) + ~3049 (hindsight_relabel opt_reward)
- la slot from reward_contrib_fractions (was slot 4; tuple shrinks 5→4)
- loss_aversion_per_sample buffer from GpuExperienceCollector
(field + alloc + kernel arg + dtoh + memset, all removed)
- la={:.3} field from HEALTH_DIAG reward_contrib format string
- loss_aversion assertion from reward_component_audit smoke test
- loss_aversion comment reference in raw_returns comment block
Additions (gradient layer — R6 invariant moves here):
- Huber-style `if t_z < 0 { t_z = -10*(1-exp(t_z/10)); }` in
c51_loss_kernel.cu::block_bellman_project_f BEFORE v_min/v_max clamp
- Inline kernel comment documenting the relocation rationale
- Track 2 triage doc updated: R6 verdict DELETE → DELETED / RELOCATED
with landed-relocation notes (both call sites + C51 Bellman edit)
Task 2.5 Bug #6: Stale `patience_mult` docstring at
experience_kernels.cu:1144 referenced the defunct R7 V8 reward (deleted
in Task 0.8). Rewrote the reward-shape docstring to reflect current
post-V7 / Task 0.8 reality (sparse = 2.0 * vol_normalized_return, capped
inline) and notes the R6 relocation. Per feedback_trust_code_not_docs.md.
Per feedback_no_functionality_removal.md: R6's invariant is RELOCATED,
not deleted. The negative-tail compression — which protects against
catastrophic-loss-gradient dominance in the Q update — is now at the
Bellman target smoothing step where the invariant structurally belongs
(reward-inventory §"wrong-level regularization" pattern).
Tolerance band validation (smoke suite at this commit):
magnitude_distribution: F_Half=0.150 F_Full=0.237 (≥0.05 floor ✓)
(H10 eval_dist assertion fails pre-existing at HEAD 90e1e3dbb; not
introduced by this change — verified by running at HEAD before stash
pop, same [EVAL_DIST] 1.000/0.000/0.000 collapse.)
reward_component_audit: cf_flip=0.584 trail=0.304 (cf_flip≥0.1 ✓, PASS)
controller_activity: [CTRL_FIRE] anti_lr=0.000 tau=0.000 gamma=0.000
clip=0.400 cql=0.000 cost=0.000 (PASS)
exploration_coverage: entropy @ep5=0.988 @ep20=0.985 (PASS)
multi_fold_convergence: Best Sharpe 81.54/38.82/84.18 (≥20 floor ✓)
best_val_metric 0.043/0.024/0.049 (baseline was 0.028/0.018/0.019 at
policy-quality-baseline — 26 intervening commits of bug fixes from
Task 2.5 bugs #1–#7 would account for persistent drift; within
run-to-run variance of HEAD-pre-change)
Scoping output from the Full=0% investigation (post-Task-2.2 smoke showed
eval_dist [eq=0.580 eh=0.420 ef=0.000] — Quarter+Half recovered,
Full still 0%).
Root cause: Q-estimation bias (category C) — C51 expected-Q over atoms
systematically under-prices higher-variance bins. Full has 4x Quarter's
PnL variance per experience_kernels.cu risk scaling. Bias is already
documented at experience_kernels.cu:904 (commented as "structural
distributional bias — tight return distributions (Small) get higher
expected Q under the C51 softmax regardless of actual expected returns").
Not noise-starvation: Full picked on 32% of training samples across
60 epochs × 3 folds = ~61k Full samples. Ample data.
Proposed Task 2.X = Rank 1 + Rank 2 combined (~90 LOC):
Rank 1: per-bin variance weighting in C51 branch loss — amplify
Full's gradient by sqrt(var_f / mean_var)
Rank 2: Q-spread regularization — lambda * ReLU(target_spread -
Q_spread)^2 penalty preventing degenerate fixpoint
Alternative fixes ranked but not recommended:
Rank 3 per-magnitude reward shaping (~80 LOC) — only if data-disfavor
is confirmed at L40S
Rank 4 magnitude curriculum (~20 LOC) — rejected (noise-starvation ruled out)
Rank 5 state-vector enrichment (~150 LOC) — premature
Prerequisite: ~40 LOC of per-magnitude win-rate + realized-variance
instrumentation to confirm category C vs A at L40S scale.
L40S gating recommendation was:
ef >= 0.15 at L40S → Task 2.X not needed (smoke artefact)
0.05 <= ef < 0.15 → Task 2.X scoped but optional
ef < 0.05 → Task 2.X required
Per feedback_fix_aggressively.md (landed this session): we ship Task 2.X
ahead of L40S since the bias is already code-documented, the fix is
well-scoped, and the 90-LOC cost is reasonable. L40S validation (Task
2.8) will verify the fixed state end-to-end instead of the un-fixed one.
No deletions anywhere per feedback_no_functionality_removal.md.
Per feedback_no_functionality_removal.md: R5 was originally scoped as
DELETE in the Phase 2 plan and the Track 2 triage because
reward_contrib[3] = 0.000 across 60 / 60 smoke epochs and
dqn-smoketest.toml sets micro_reward_scale = 0.0. Re-examination during
Phase 2 Task 2.3 rejected the DELETE path:
- dqn-production.toml already sets micro_reward_scale = 0.1, so R5 is
load-bearing in production, not dead code. The 0.0 value in smoke is
deliberate test-isolation (td_propagation / magnitude_distribution /
reward_component_audit all want the sparse-reward TD path isolated).
- The state-vector OFI block at state[SL_OFI_START..SL_OFI_START+SL_OFI_DIM)
= [42..62) provides representation features for the encoder (policy
side). R5 is a per-bar reward gradient on the critic (critic side).
Different mechanisms — production deploys both together.
- R5 also reads PREV_MID (retrospective hold quality), which is NOT in
the state vector. That signal exists only in the kernel branch.
Changes — pure documentation, no behavior change:
- experience_kernels.cu: ~30-line comment block at the R5 wiring site
(~L1915) documenting the parameter-not-flag status, production vs
smoke values, why state-vector OFI is complementary not redundant,
and the feedback_no_functionality_removal.md seal.
- experience_kernels.cu: fix stale kernel-signature comment that claimed
OFI was at state[66..74). Correct range is [42..62) per state_layout.cuh.
- config.rs: extend DQNHyperparameters::micro_reward_scale docstring and
add a comment at the Default impl pointing back at the kernel site.
- gpu_experience_collector.rs: extend reward_contrib_fractions docstring
to mark the micro=0.000 slot as a SEMANTIC value when the loaded profile
has micro_reward_scale=0.0, not a wiring regression.
- track2-triage.md: R5 verdict changed from DELETE to FIX-documented-disable
with the rationale above; "Proposed Phase 2 changes" section 1 and
"Next Track 2 steps" updated accordingly.
Smoke tests: 3 / 4 pass (reward_component_audit, controller_activity,
exploration_coverage). magnitude_distribution is failing on baseline
HEAD c0fee5a9b as well — pre-existing H10 magnitude-head collapse
regression (eval dist_mag ef=0.000, eh=0.000), unrelated to R5.
Verified via git-stash round-trip: baseline fails, changes do not
introduce new failures.
Closes Track 2 R5 verdict (originally DELETE, now FIX-documented-disable).
User directive: "we don't remove functionality at all, we fix what's
broken". Standing rule captured in
~/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_no_functionality_removal.md.
Phase 2 plan reframed:
* Task 2.3 (R5 micro-reward): was DELETE; now FIX — either (a) set a
non-zero scale that delivers real signal, or (b) document why it's
intentionally disabled while preserving the code path.
* Task 2.4 (R6 loss-aversion): unchanged — the relocation to C51
Bellman target is consolidation, not removal. Invariant preserved.
* Task 2.6 (E4 entropy-reg): was DELETE-CANDIDATE; now TUNE — if the
signal doesn't reach magnitude, re-route it rather than remove.
* Task 2.7 (C4 grad-clip): was DELETE-CANDIDATE; now
KEEP-AND-DOCUMENT — gradient clipping is a safety feature; run the
ablation for diagnostic, not for removal.
* H9 delete-magnitude-branch fallback: rejected permanently. On
magnitude convergence failure the fallback is per-magnitude reward
shaping / per-bin advantage weighting / curriculum / state enrichment.
Task 2.5 wiring-bug sweep unchanged (correctness fixes, not removal).
This commit updates only the planning narrative. Task execution hasn't
started on 2.3/2.6/2.7 yet, so no code changes required here. When
those tasks run, they will implement the fix-paths, not the
delete-paths.
Task 2.0 instrumentation (commits d60e5375a / 980f3b07f / 41b0c559c)
revealed two silent bugs in Task 0.4's grad_ratio_mag_dir accessor:
Bug A — readback size mismatch: pinned buffer allocated at
total_params, but grad_buf length is total_params+cutlass_tile_pad,
so size check always failed → Err silently coerced to 0.0 by proxy.
Bug B — readback timing wipe: estimate_avg_q_value_with_early_stopping
in process_epoch_boundary replays the forward graph, which zeros
grad_buf. Any subsequent readback sees all zeros.
Both fixed in 41b0c559c; snapshot hoisted to top of process_epoch_boundary.
With those bugs fixed, the measured gradient ratio is NOT 0.0000 — it is
50–400× mag/dir across 60 epochs. Magnitude branch is over-fed, not
starved. Direction gradient is small but non-zero (~2e-2 to 7e0).
Direction policy is observably healthy (Short/Hold/Long/Flat 38/12/42/14%).
Track 1 triage's H4 CONFIRMED verdict was a measurement artefact
produced by Bugs A+B. H4 as originally defined is now REJECTED.
Plan changes:
- Add new "Task 2.0 findings" section after cross-cutting concerns,
documenting bug A, bug B, observed data, and revised strategy.
- Task 2.1 marked DEFERRED (not deleted — kept for reference).
- Task 2.2 promoted to PRIMARY fix.
- New fallback: H9 delete-magnitude-branch as replacement for Task 2.1
if Task 2.2 alone is insufficient.
- Task 2.0 inventory row updated with LANDED status and commit chain.
Net effect: Phase 2 simplifies. The hardest task (2.1 three-branch
architectural fix) is skipped; primary path is Task 2.2 (~25 LOC).
First Task 2.0 dispatch escalated BLOCKED: the four loss-component
backward kernels are captured inside the fused training graph, so
host-side snapshot-between-components isn't possible mid-graph without
a force-ungraphed diagnostic step (~210 LOC + cross-stream sync risk).
Revised approach (chosen after cost analysis):
cudaMemcpyAsync(device → pinned host) IS captureable in a CUDA graph.
Even better: DtoD into per-component scratch buffers, then an in-graph
reduction kernel computes per-component (mag_norm, dir_norm) and writes
8 floats to a pinned result slot. Only the 8-float result crosses
PCIe (at epoch boundary), keeping per-step PCIe traffic to zero.
Changes to the plan's Step 2 + Step 3 + Step 4:
- Step 2: added 4 device-side scratch buffers (one per component,
~10 MB each = 40 MB device) + 8-float pinned result slot + new
reduction kernel grad_decomp_kernel.cu spec'd out.
- Step 3: clarified that DtoD snapshot + backward + reduction kernel
are ALL captured in the graph; graph replays them every step;
no force-ungraphed dance needed.
- Step 4: added refresh_grad_component_norms() accessor that reads
the 8-float pinned slot at epoch boundary (zero-copy) and populates
the host-side cache.
Approach matches Task 0.4 pattern (commit bb42c9963) extended four-fold.
No atomicAdd (reduction uses shared-mem tree), no non-captured replay,
no cross-stream sync risk.
LOC estimate: ~90 (was ~210 for the rejected option).
Three polish items from the plan review:
1. Task 2.4 Step 5 — replaced vague "must not regress" with a numeric
tolerance-band table: multi_fold best_val_metric ±15% per fold,
Best Sharpe ≥ 20 floor, and hard F_Half/F_Full ≥ 0.05 + cf_flip ≥ 0.1
BLOCKERS. Anchors to the baseline metrics doc values captured at
policy-quality-baseline.
2. Task 2.6 — converted the ad-hoc text decision at Step 1 into the same
five-row decision table format Task 2.1 uses (DELETE / KEEP / KEEP-as-
safety-net / INCONCLUSIVE / DELETE-because-inseparable). Step 2
ablation got its own 4-row outcome table keyed to ent_mag delta,
multi-fold Sharpe regression, and NaN appearance.
3. Task 2.7 Step 1 — added "Repeat 3× with different seeds" clause and
explicit total wall-clock note: ~75 min (3 × 25 min) for the ablation
pass before the Step 2 decision. Aligns with the Cross-cutting concern
#6 about sample-noise rejection.
No task count change (still 11: 2.0–2.10). Net code-delta estimate
unchanged. Standing-rule compliance unchanged (no stubs / no atomics
/ no quickfixes / no hiding / no feature flags / no push-per-task).
V7 audit of the 7 adaptive controllers named in spec §5.3 against the
baseline controller_activity smoke run (3 folds × 20 epochs = 60 epochs,
intervention-based fire detection per commit ed4b30b49):
* C1 anti_lr: DIAGNOSTIC (0/60; wiring surprise — fire detector reads
base scheduler, not post-anti_lr LR, flagged for Phase 2 fix)
* C2 adaptive tau: DIAGNOSTIC (2/60; fires at fold boundaries only)
* C3 adaptive gamma: DIAGNOSTIC (1/60; pinned to floor by health-coupled
correction; re-measure at L40S when health is real)
* C4 adaptive grad_clip: CANDIDATE FOR DELETE pending ablation (12/60,
20% — above diagnostic, below load-bearing; needs Track 3 Step 2)
* C5 cql_alpha: DIAGNOSTIC (2/60; regime_stability gate uninformative
at smoke scale)
* C6 cost_anneal: DIAGNOSTIC by design (deterministic sigmoid; fire_cost
hard-coded false at training_loop.rs:2475)
* C7 base LR scheduler: DIAGNOSTIC by construction (pure open-loop, not
in the closed-loop controller battery)
Cross-controller finding: no controller fires in > 50% of epochs, so
the policy is not currently held on the rails by adaptive intervention
at smoke scale. However, C2 / C3 / C5 are structurally inert here —
their trigger signals (health, regime_stability) are degenerate at
smoke scale, so their 0-ish fire rates are lower bounds, not
representative. Promote to final after L40S validation.
Wiring surprise (anti_lr): fire_lr detection reads lr_scheduler.get_lr()
*before* the anti_lr multiplier applies. Under smoke Constant LR this
reports 0 correctly-for-the-wrong-reason; under L40S Cosine/Linear it
will false-positive on pure decay drift. Recommend detecting via the
anti_mult != 1.0 branch directly (training_loop.rs:2936–2942) for an
unambiguous intervention semantic matching C4/C6.
Follow-ups flagged for Phase 2:
- Fix C1 fire-detection wiring before L40S re-run
- Run C4 grad-clip ablation (25 min, gates the final C4 verdict)
- Optional: reset fire_counts per fold to remove C2/C5 boundary artefact
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Preliminary triage of spec §5.1 hypotheses H1–H10 using the Phase 0
baseline capture on RTX 3050 Ti. L40S validation pending per plan.
Verdicts:
H1 PENDING (needs forced-exploration instrumentation not yet wired)
H2 REJECTED var_scale=0.96 across 19/20 epochs; Var[Q] inactive at smoke scale
H3 INCONCLUSIVE kelly degenerate (insufficient win/loss counts at smoke scale)
H4 CONFIRMED grad_ratio_mag_dir=0.0000 across 20/20 epochs (threshold <0.1)
H5 REJECTED ent_mag stays ≥0.98 throughout; no bootstrap collapse
H6 REJECTED Full fire rate (0) is lower than Quarter fire rate, not higher
H7 REJECTED vsn and sigma symmetric between mag and dir branches
H8 REJECTED target-net drift equal (mag=dir=0.001)
H9 PENDING (same instrumentation gap as H1)
H10 CONFIRMED training ent_mag=0.98, eval F_Quarter=100%
Synthesis: H4 is the root cause. Magnitude branch receives ~0 gradient →
weights stay near init → three magnitude Q-values near-identical → argmax
picks bin 0 (Quarter) on ties → H10 manifests at eval time. H2, H5, H7,
H8 all ruled out as contributors.
Proposed Phase 2 priority: fix H4 (gradient-flow path into magnitude head
— likely per-component advantage weighting or direction-conditioning of
w_b1fc) + H10 (Q-margin argmax + stochastic eval rollouts as safety net).
Phase 1 next: validate preliminary verdicts on L40S, instrument
per-component gradient decomposition for magnitude, proceed with
Tracks 2/3/4 in parallel.
All <PENDING> markers replaced with real values from a clean 20-epoch
magnitude_distribution smoke run + 3-fold multi_fold_convergence run
on RTX 3050 Ti at HEAD 0472b9730.
All 5 Phase 0 smoke tests PASS:
magnitude_distribution — Quarter=0.596 Half=0.150 Full=0.255
reward_component_audit — cf_flip=0.619 trail=0.290 la=0.013
controller_activity — all 6 controllers < 20% firing
exploration_coverage — ent_mag @ep5=0.987 @ep20=0.985
multi_fold_convergence — 3/3 folds, Best Sharpe 51 / 39 / 87
This commit closes Phase 0. The next commit tags it as
policy-quality-baseline — the reference state against which Phase 2
interventions are measured.
Phase 0 instrumentation is complete (Tasks 0.4 / 0.5 / 0.8 / 0.10 / 0.16
shipped this session; 0.6 partial; 0.3 partial). This scaffold encodes
the full metric set with explicit <PENDING …> markers per row so the
GPU-run capture can fill in values without forgetting any HEALTH_DIAG
field.
The policy-quality-baseline tag is intentionally NOT applied yet —
tagging requires real captured values, not the scaffold. Runner workflow:
for t in magnitude_distribution reward_component_audit \\
controller_activity exploration_coverage multi_fold_convergence; do
SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 \\
FOXHUNT_TEST_DATA=test_data/futures-baseline \\
cargo test -p ml --release --lib -- \$t --ignored --nocapture 2>&1 | tee out_\$t.log
done
Then edit this doc, replace markers with values, amend, and tag.
User preference: everything lands on main, ~500 LOC is small enough
that feature-branch isolation isn't needed.
Spec changes:
* §2.3 outcome paths: drop "unmerged branch" vocabulary
* §3 architecture: main-branch timeline, no feat/, no wip/
* §4 Phase 0: commits on main, tag policy-quality-baseline at end
* §5 Phase 1: experiments are worktree edits reverted after, only
triage docs commit (to main)
* §6 Phase 2: commits land on main, author chooses granularity,
each leaves smokes green
* §7.3 outcome handling: iterate on main, baseline tag for rollback
* §7.4 closing: tag policy-quality-v1, update status
Plan changes:
* Task 0.1: verify-starting-state (not branch creation)
* Task 1.x: temp-edit-then-revert (not wip branches)
* Task 1.5: triage docs commit directly to main
* Phase 4: tag + close (not merge + cleanup)
* Rollback: one git reset --hard command
