Decisive smoke train-xrkb7 @ ebc7144434: WR=0.4346 with full Phase 3
mechanism + enlarged aux head. Statistically identical to all baselines
(0.4338-0.4346). Hypothesis falsified.
Root cause investigation:
- Aux head's 70% accuracy was mostly majority-class prediction on
imbalanced data (fold 0: 88% down labels)
- Fold 1 with balanced labels: aux accuracy dropped to 52% (near-random)
- Aux head has minimal per-bar discriminative power
Architectural insight: aux head predicts next-bar direction but the
system makes MULTI-BAR TRADE decisions with target_bars / profit_target
/ stop_loss / conviction plan parameters. The decision horizons mismatch.
This commit:
1. Revert mechanism to dormant (W=0, beta=0) - hypothesis falsified
2. Preserve infrastructure (kernels, NaN guards, enlarged aux head)
3. Write vNext spec: trade-outcome aux head with plan_params concat
input + K=3 outputs {Profit, Stop, Timeout} + 12-weight W matrix
vNext reuses ~80% of current infrastructure. ~2-3 days focused work.
See docs/plans/2026-05-14-sp22-h6-vNext-trade-outcome-aux.md.
Cargo check clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Derived the projection-arithmetic transformation that collapses per-(b, a)
atom-position shift into a SINGLE per-sample reward adjustment:
effective_reward = reward + gamma_eff * Delta_target * (1-done) - Delta_online
where
Delta_online = W[a_d] * state_121[b] (taken-action shift on online support)
Delta_target = W[a*] * next_state_121[b] (sampled-target shift on target dist)
The Bellman projection (block_bellman_project_f body) needs NO arithmetic
change — only the reward arg passed in. This dramatically reduces Step 7's
kernel surgery surface area.
For Step 8 backward, derived the gradient paths:
dL/dDelta_online = (1/dz) * sum_n p_target_n * (log_p_online[upper_n] - log_p_online[lower_n])
dL/dDelta_target = -gamma*(1-done) * dL/dDelta_online
dL/dW[a_d] += dL/dDelta_online * state_121
dL/dW[a*] += dL/dDelta_target * next_state_121
dL/dstate_121[b] += dL/dDelta_online * W[a_d]
dL/dnext_state_121[b] += dL/dDelta_target * W[a*]
Critical constraint documented: Steps 7+8+11 MUST land atomically. Splitting
Step 7 forward from Step 8 backward creates a silent gradient path through
the aux head (missing dDelta_online/dnetwork_weights). Splitting Step 8 from
Step 11 Adam accumulates dW without updating W — no learning. Per
feedback_no_partial_refactor.md.
The math now makes Step 7+8 a transcription job rather than exploration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Architectural finding during Phase 3c implementation attempt:
Original spec: α adds `W[a] * state_121[b]` to scalar Q values. Adam-
trained W via dloss/dQ gradient.
Actual codebase: C51 distributional Q-learning. Q is computed from a
probability distribution over fixed atom positions; C51 loss is KL
divergence over distributions, not MSE over scalars. Adding a constant
to logits is softmax-shift-invariant; adding to scalar expected_Q
happens post-loss-path. Either way, the KL loss is invariant under
scalar Q bias → W gradient = 0 → W never trains → α is mathematically
ineffective as originally specified.
Principled fix
──────────────
Per-(b, a) atom-position shift:
z_n_effective[b, a, n] = z_n_base + W[a] * state_121[b] for all n
Shifts atom positions state-dependently. C51 Bellman projection
re-projects target onto shifted positions → loss depends on
`W[a] * state_121[b]` smoothly → W gradient flows from projection
arithmetic.
Implementation cost grows from ~25-35 hr to ~40-60 hr because the
atom-shift threads through every kernel that uses atom positions
for the direction branch:
- compute_expected_q (action selection)
- c51_loss_kernel (Bellman projection)
- c51_grad_kernel (backward dW + dstate from projection arithmetic)
- mag_concat_qdir (magnitude-branch conditioning Q_dir compute)
- quantile_q_select / iqn_dual_head_kernel (investigate)
The Phase A `aux_to_q_dir_bias_kernel.cu` + `aux_to_q_dir_bias_backward_kernel.cu` become architecturally redundant under the revised
design. The dW + dstate gradient computations integrate into
c51_grad_kernel's existing projection backward. Phase A kernels
stay as compiled cubins (committed in 464bc5f7a); future cleanup
commit may delete them.
Initialization: structural prior `W_init = [-0.5, 0.0, +0.5, 0.0]`
reflects domain belief that aux conviction × position-sign biases
toward aligned trades. Adam refines from there.
state_121 ∈ [-1, +1] = recentered aux p_up. When +1 (aux up):
Q_short atoms shift DOWN by 0.5 → action selection avoids Short
Q_long atoms shift UP by 0.5 → action selection biases Long
Hold / Flat unchanged
Adam-vs-fixed-W: the atom-shift design subsumes the fixed-W case
as a config choice (set Adam LR for W = 0). Per user directive
"no shortcuts", the full version is the canonical Phase 3-final
spec.
β + SP11 controller + A2 eval-side + telemetry unchanged from
prior spec sections — those parts of Phase 3 are architecturally
sound. Only α needed correction.
Files touched
─────────────
- docs/plans/2026-05-12-sp22-h6-phase3-alpha-beta.md:
Replaced "α — post-encoder bypass-head" section with
"α — atom-position shift (architecturally revised 2026-05-13)".
Documents the C51 incompatibility, the per-(b, a) atom-shift
fix, the structural-prior initialization, the kernel inventory
for atom-shift threading, the Adam wireup, and the gradient
routing decision (option (i) unchanged).
- docs/dqn-wire-up-audit.md:
"Phase 3c — α architectural finding + spec revision (2026-05-13)"
entry documenting the math obstacle, the principled fix, the
Phase A kernel redundancy under the revised design, and the
scope-revision rationale.
Next session
────────────
Runbook (`docs/plans/2026-05-13-sp22-h6-phase3-alpha-beta-runbook.md`)
α tasks (B9 Steps 4-9, C1) describe the original scalar-bias
implementation and need rewriting for atom-shift threading before
execution can resume.
Refs
────
- pearl_audit_unboundedness_for_implicit_asymmetry (motivates
per-action W signs for the structural prior)
- C51 distributional Q math: Bellemare, Dabney, Munos 2017
- pearl_no_partial_refactor (atom-shift threading is atomic across
all consumer kernels)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folded A2 (eval-side α + β + aux trunk forward) into the Phase 3 spec
per user directive. Brings eval pipeline to production parity with
training-time rollout.
A2 components (6 sub-items)
───────────────────────────
A2.1 — Aux trunk weight loading at eval init (shared-source-of-truth
with trainer, mirroring existing GEMM weight pattern).
A2.2 — Per-window `prev_aux_dir_prob` buffer at eval ([n_windows] f32,
fill_f32(0.0) init + per-evaluate() reset).
A2.3 — Aux trunk forward at eval per-step (reuse `AuxHeadsForward`
orchestrator if API permits; fall back to thin eval-only
variant if too training-coupled).
A2.4 — α at eval (post-Q_dir bias launch, reuses training kernel).
A2.5 — β at eval (mirror of training producer in
`backtest_env_kernel.cu::segment_complete`; same NULL-fallback
semantics; reads scale_beta from training-emitted ISV slot).
A2.6 — Wire `prev_aux_dir_prob.raw_ptr()` (non-NULL) into all 3
backtest state-gather launchers.
7-component contract migration EXTENDED
───────────────────────────────────────
Added `backtest_env_kernel.cu` to the atomic 7-component migration
list (eval-side reward stride 6 → 7 alongside training-side). All
reward-component consumers across training AND eval paths migrate
in one commit per `feedback_no_partial_refactor`.
Scope estimate updated
──────────────────────
~34–47 hr engineering (5–6 working days), up from ~21–31 hr in the
within-phase-follow-ups version. Bulk of addition is A2.1–A2.3
greenfield work — the eval pipeline had no aux infrastructure
before.
Verification gates expanded (gate 7 added)
──────────────────────────────────────────
7. **A2 eval-side aux activity check**: post-cycle-1 validation eval
shows non-zero `r_aux_align` in eval-side WindowMetrics reward
decomposition. If eval r_aux_align ≈ 0 while training-side > 0 →
A2 wiring failure. gpu_backtest_validation tests should still
pass; potential tolerance adjustment for extended metrics
(CVaR/Omega) if β shifts numerics meaningfully; the four
directional tests remain bit-identical because constant_action_model
bypasses Q-network and β is no-op at test-time (sentinel-zero
scale_β since tests don't run SP11 controller).
Out of scope (sole remaining)
─────────────────────────────
Only the aux-trunk-gradient-flow-back-through-state[121] item, which
is a property statement (preserved by H6 design's stop-grad), not a
deferral.
Refs
────
- pearl_separate_aux_trunk_when_shared_starves (A2.1 aux trunk source-
of-truth pattern)
- pearl_no_partial_refactor (7-component migration includes eval-side
backtest_env_kernel atomically)
- pearl_no_deferrals_for_complementary_fixes (combined plan now
spans training + eval)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per the directive "no follow ups include in spec", expanded the
Phase 3 design to include three previously-out-of-scope items:
1. β as a separate 7th reward component (not added to r_trail)
────────────────────────────────────────────────────────────
Contract change from `reward_components_per_sample[6]` →
`reward_components_per_sample[7]`. Migrated atomically per
`feedback_no_partial_refactor` across every consumer:
- experience_kernels.cu (producer site at segment_complete)
- reward_component_ema_kernel.cu (iterate 0..7)
- reward_decomp_diag_kernel.cu (emit column 6)
- reward_component_mag_ratio_compute_kernel.cu (axis 6)
- reward_subsystem_controller_kernel.cu (7-weight output)
- gpu_experience_collector.rs + gpu_dqn_trainer.rs (buffer alloc
+ HEALTH_DIAG print format)
- reward_component_monitor.rs (7-component EMA reader)
- health_diag_kernel.cu (snap layout)
- training_loop.rs (sp11_reward + reward_split lines)
New ISV slot REWARD_AUX_ALIGN_EMA_INDEX for the component EMA.
2. SP11 controller emits scale_β adaptively
─────────────────────────────────────────
New ISV slot `SP22_AUX_ALIGN_SCALE_INDEX`. Producer:
reward_subsystem_controller_kernel extended to 7 weight outputs.
Anchor: REWARD_AUX_ALIGN_EMA_INDEX. Target: ~10% of total reward
magnitude. Bounds [0.05, 2.0]. Cold-start sentinel 0 per
`pearl_first_observation_bootstrap` — β no-op until first emit.
state_reset_registry registers both slots as FoldReset category.
3. Encoder-state[121] gradient routing — DECISION committed
─────────────────────────────────────────────────────────
Option (i) — both paths open. `dbatch_states[121] +=` flows
gradient into both α's W and the encoder's column for slot 121.
Long-term robust: encoder may eventually learn additively (belt
+ suspenders). Empirical observable: compare W magnitudes vs
encoder column-norm for state[121] in post-smoke telemetry.
Explicitly OUT of scope (with reasoning, not deferral)
──────────────────────────────────────────────────────
A2 (eval-side α + β + aux trunk forward at eval): the same
architectural deferral logic from H6 Phase 1. Adding aux
infrastructure to GpuBacktestEvaluator should follow positive
training-side evidence rather than precede it. A3 NULL-fallback at
eval makes both α and β no-ops at eval-time, isolating the verdict
to training-side. Becomes a separate spec if Phase 3 confirms.
Scope estimate updated
──────────────────────
~21–31 hr engineering (roughly 3-4 working days) — up from the
earlier "~10–16 hr" YAGNI estimate. The bulk of the addition is the
7-component contract migration touching ~8 kernel/source files
atomically and the SP11 controller extension.
Verification gates expanded
───────────────────────────
6. **7-component contract sanity** (smoke cycle 1): HEALTH_DIAG
sp11_reward shows 7 weights including w_aux; reward_split shows 7
components including aux_align; both non-zero within the first
epoch after the SP11 controller's first emit. If still zero,
either the controller emit is broken OR the β producer isn't
firing.
Refs
────
- pearl_no_deferrals_for_complementary_fixes (combined plan)
- pearl_no_partial_refactor (7-component atomic contract)
- pearl_controller_anchors_isv_driven (SP11 scale_β anchor)
- pearl_first_observation_bootstrap (cold-start sentinel 0)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bite-sized 12-task runbook for the recentering fix specified in
`docs/plans/2026-05-12-sp22-h6-phase2-recenter.md`. Tasks 1-5 are the
5 atomic edits (kernel write recentered to `2*p - 1`, sentinels 0.5
→ 0.0 across host + 3 NULL-fallback kernels, plus comment updates in
state_layout.rs / state_layout.cuh). Tasks 6-8 are the three
verification gates (cargo check / gpu_backtest_validation /
compute-sanitizer) identical to H6 Phase 1 — all required clean before
commit. Task 9 appends the H6 Phase 2 entry to the audit doc
(Invariant 7 satisfaction). Task 10 is the single atomic commit per
`feedback_no_partial_refactor` covering all 5 source files + audit
doc. Task 11 dispatches the 3-epoch baseline smoke on
ci-training-l40s. Task 12 monitors for the cycle-1 verdict using the
same single-channel grep-anchored bg waiter pattern as Phase 1
(per `feedback_no_redundant_monitor`).
Every task has exact file paths, complete code blocks for edits
(showing old + new strings for Edit-tool consumption), exact commands
with expected output, and explicit kill criteria for verification gate
failures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Post-mortem of the H6 Phase 1 smoke (WR = 50.21% verdict in
`docs/dqn-wire-up-audit.md`) traced an actual pearl violation in the
H6 implementation itself: state slot 121 uses the [0, 1] range
(`aux_softmax[env, 1] = p_up`) with sentinel 0.5, while every OTHER
state slot uses 0 as the "no signal" baseline (zero-padding,
feature_mask, ofi-missing, mtf-missing).
Per `pearl_first_observation_bootstrap`: "sentinel = 0; first
observation replaces directly." The H6 design violated this for
slot 121 alone. The encoder must learn TWO things about slot 121:
(1) the directional mapping AND (2) the appropriate bias offset for
the non-zero baseline — every other dim is single-step (mapping only).
Phase 2 is the simplest possible amplification fix: rewrite the bridge
to use the same convention as every other state slot. If the encoder
can't gradient-couple even after this fix, H6 is truly falsified and
we pivot to amplitude scaling or deeper hypothesis.
Change scope (atomic per `feedback_no_partial_refactor`):
- aux_softmax_to_per_env_kernel.cu: write `2*p_up - 1` instead of `p_up`
- gpu_experience_collector.rs: cold-start + FoldReset sentinel 0.5 → 0.0
- experience_kernels.cu: NULL-fallback in 3 state-gather kernels
0.5f → 0.0f
- state_layout.rs / state_layout.cuh: comment updates to reflect
[-1, +1] range and 0 sentinel
Estimated effort: ~45 min walltime (edits + verification gates) +
~25–40 min smoke wall-clock.
Verdict criteria (cycle 1 WR + a_var [d/m/o/u] in HEALTH_DIAG[0]):
- WR > 50.5% → recentering binding, H6 + Phase 2 sufficient → justify A2
- a_var for mag/ord/urg > 1e-3 → sub-branches learning under recentered signal
- WR pinned at 50.1–50.2% → Phase 2 falsified, pivot to amplitude
scaling or deeper hypothesis
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new plan docs to enable clean session handoff:
1. docs/plans/2026-05-12-sp22-h6-aux-policy-state-bridge.md
Detailed 10-step implementation runbook for H6 (aux→policy state
bridge). Each step has file paths, kernel signatures, expected
diff, and risk callouts. Estimated ~5hr for Phase 1 (training-side
+ A3 eval fallback). A2 (eval-side aux integration) is +1 day
follow-up if H6 confirmed.
2. docs/plans/2026-05-12-sp22-h6-next-session-prompt.md
Concise context-loading prompt to paste into next session.
Includes state of the world, runbook pointer, verification gates,
and start-here pointer.
Why staged: H6 implementation crosses kernel-level state-layout
contract (every consumer of STATE_DIM must migrate atomically per
feedback_no_partial_refactor). Doing it RIGHT requires careful
incremental verification with compute-sanitizer between steps —
not safely batched into a tail-end session.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
H1 result (commit 9adbca826, smoke train-5zmkr, reverted at e8814079d):
aux_dir_acc HD[2] = 78% with H=200 — aux head LEARNS direction at
longer horizon. But policy WR stayed pinned at 50.1-50.2% — the
learned aux signal does NOT propagate to policy (per
pearl_separate_aux_trunk_when_shared_starves: aux on separate
trunk with stop-grad to policy).
H6 design (synthesized from all v5-v11 + H1 evidence):
Wire the aux head's directional probability into the policy STATE
as an input feature.
- Policy gradient flows THROUGH the feature (uses it)
- Stop-grad blocks gradient BACK (aux trunk unaffected, pearl
preserved)
- Uses existing padding slot [121..128) in STATE_DIM=128 (no
layout growth)
Why this is the structural fix:
- Aux PROVED directional signal is in the features (78% at H=200)
- Policy PROVED it can't extract direction (WR=50% across all
v5-v11 conditions)
- Bridge connects the two without violating trunk separation
- Information-theoretic: gives policy a feature it provably
can't compute itself
Test outcome interpretation:
WR > 50.5% → Mechanism 1 was binding (trunk separation gap)
WR pinned → Mechanism 2 (reward density) or Mechanism 3 (V/A
unidentifiability) dominates → H3 or V/A fix next
Files changed:
- docs/plans/2026-05-12-sp22-wr-plateau-investigation.md:
H6 added as new primary hypothesis after H1; experiment order
revised
- docs/dqn-wire-up-audit.md: H6 design entry with three-mechanism
synthesis
Implementation scope (separate commit):
1. State layout: claim slot in padding [121..128) for aux_dir_prob
2. Aux trunk export: pull "up" probability per bar from aux forward
3. Experience collection: write aux_dir_prob into per-bar state
4. Stop-grad verification: confirm policy gradient blocked
5. Trade-open persistence: latch aux_dir_prob for trade duration
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SP21 T2.2 status: cascade wiring COMPLETE.
- All E1-E8 enrichment producers wired to consumers
- Local compute-sanitizer clean on eval-baseline closure path
- v9 cycle 1 confirms cascade live: win_conc=1.72, curric_conc=0.31,
hindsight_mag=1.47e-5
- Eval pipeline now hard-fails on GPU error, no CPU fallback,
factored-action branch sizes + default ISV all wired
v10 hypothesis test (10 epochs, killed at E8 with clear answer):
- val_Sharpe peaks at E3 (174), monotonically degrades to E8 (66, -62%)
- WR pinned at 50.1-50.2% across ALL 8 epochs (no improvement)
- PF pinned at 1.00-1.01
- Cascade controllers stable but cannot move policy off the plateau
- Verdict: 3-epoch baseline structure IS optimal; longer training
monotonically degrades. WR plateau is upstream of the cascade.
Implication: SP21 T2.2 cascade is necessary but insufficient for
project_goal_wr_55_pf_2 (WR≥55%, PF≥2.0). Need new SP arc to address
the WR=50% plateau at its source.
New plan: docs/plans/2026-05-12-sp22-wr-plateau-investigation.md
Hypotheses (priority order):
H1: Label horizon mismatch (cheapest, most likely)
H2: Action-space pathology (Hold hiding directional signal)
H3: Reward shape (no directional gradient)
H4: Feature representation gap (MTF features zero-padded)
H5: Bar resolution itself is too noisy (longshot)
Each hypothesis tested as atomic smoke with one variable changed
vs v9 baseline (peak val=174, WR=50.1%).
Phase 1 milestone: WR ≥ 51% on val for ≥ 1 fold.
Phase 2: WR ≥ 53% across 3 folds.
Phase 3: WR ≥ 55% AND PF ≥ 2.0 (SP20+ goal achieved).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the continuation plan for SP21 T2.2 Phase 1.5 (entry_q tracking
in portfolio_state slot 6) + Phase 2 (wire enrichment to real
per-trade tape from gpu_backtest_evaluator). Self-contained plan
that a fresh session can pick up cold:
- State-at-session-start summary with all 8 prior commits
- Phase 1.5 design (storage slot, capture site, plumbing,
EvalTrade extension)
- Phase 2 design (training_loop wire-up, enrichment refactor,
E5 alternate-signal options with recommendation)
- Hard rules carried from prior session (no NULLs, no stubs,
atomic per feedback_no_partial_refactor)
- Verification plan (5 GPU oracle test suites)
- Open design question (E5 alternate signal) with recommendation
- Multi-phase continuation map (Phases 3-7 + Phase 8)
- Final cascade verification (smoke run criteria)
Also amends the parent SP21 plan
(2026-05-10-sp21-train-eval-coherence-isv-defrost.md) with the
T2.2 multi-phase scope section that documents the full per-trade-tape
expansion the user committed to (option 3 — full wiring across
multiple sessions instead of the original recommended option (b)
aggregate-stats refactor).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the Phase 3.2 forward-reference loop in the SP20 aggregator.
Previously `out_inputs->per_bar_hold_reward = 0.0f` was hardcoded; the
per-bar Hold opportunity-cost producer existed
(experience_kernels.cu:3823 — `per_bar_opp_cost = -aux_conf × cost_scale`)
and wrote to `hold_baseline_buffer` and `r_micro` directly, but never
reached HOLD_REWARD_EMA. Result: HOLD_REWARD_EMA frozen at sentinel 0.0
across all observed epochs; the SP20 reward centering loop
(`r_micro += per_bar_opp_cost - HOLD_REWARD_EMA`) stayed uncentered,
biasing the policy's reward signal away from zero-mean.
T3.3 wireup mirrors the T2.2 alpha refactor: a per-env scratch buffer
that the producer writes at every step (alongside the existing
`hold_baseline_buffer` write), and the aggregator reads at the same
step. Sums opp_cost over Hold-direction envs only; emits the mean as
`per_bar_hold_reward`. The HOLD_REWARD_EMA's gate (`hold_fraction > 0.5f`
from T3.2) preserves the strict-majority semantic from before.
Atomic across producer site, kernel signature, aggregator, launcher,
collector, and tests (per feedback_no_partial_refactor):
- experience_kernels.cu — new `float* per_bar_opp_cost_per_env`
kernel arg, NULL-tolerant write at line ~3823.
- sp20_aggregate_inputs_kernel.cu — new arg, 6th shmem stripe
(`sh_opp_cost_sum`), per-thread Hold-gated accumulation, tree
reduction extended to 6 stripes, output `per_bar_hold_reward =
opp_cost_sum / hold_count` when hold_count > 0 else 0.
- sp20_aggregate_inputs.rs — launcher signature, dynamic_shmem_bytes
5→6 stripes, doc table, internal test.
- gpu_experience_collector.rs — new `per_bar_opp_cost_per_env:
CudaSlice<f32>` field, alloc, struct construction, kernel-arg
pass at both experience_env_step and sp20_aggregate_inputs
launches (raw_ptr).
- sp20_aggregate_inputs_test.rs — helper renamed
`run_kernel_with_is_win_and_opp_cost`, 4 existing sites pass
NULL, 2 NEW oracle tests verifying real producer + NULL fallback.
- sp20_phase1_4_wireup_test.rs — NULL fallback at the wireup site;
HOLD_REWARD_EMA-stays-at-zero assertion remains valid.
Verification:
- cargo check -p ml --tests: passes (warnings only)
- cargo test -p ml --test sp20_aggregate_inputs_test --features cuda
-- --ignored: 12/12 GPU oracle tests pass on RTX 3050 Ti, including
both new T3.3 tests (per_bar_hold_reward_means_over_hold_envs_only,
null_per_bar_opp_cost_emits_zero).
- cargo test -p ml --test sp20_phase1_4_wireup_test --features cuda
-- --ignored: 2/2 pass under the new NULL-tolerant contract.
Plan reference: docs/plans/2026-05-10-sp21-train-eval-coherence-isv-defrost.md
Tier 3 status: T3.1 ✓, T3.2 ✓, T3.3 ✓ (this commit), T3.4 withdrawn,
T3.5 cascade-pending, T3.6 withdrawn.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reverts the forced H=30 diagnostic in aux_horizon_update_kernel.cu
(commit c78c4766c) — the experiment confirmed the data ceiling
hypothesis (aux_dir_acc dropped 0.46→0.50 with pred_tanh collapsing
to ~0, opposite of the bootstrap-failure prediction). Restores the
adaptive Pearl-A + Wiener-α floor logic.
Adds SP21 plan: train/eval coherence + ISV defrost. Catalogs 26
findings across 5 tiers from a pair-audit of MinIO-archived training
logs (xmd6b 30 epochs, d7bj7 2 epochs).
Cross-cutting principle: every threshold or bound in training
control flow must be signal-driven from an ISV slot, not hardcoded
(per feedback_isv_for_adaptive_bounds + feedback_adaptive_not_tuned).
Canonical pattern documented: ISV[CURIOSITY_PRESSURE_INDEX=346]
(SP11 Fix 39) is the reference implementation.
Meta-finding: across SP3-SP20 we've been monitoring training-rollout
metrics (Thompson-noisy) instead of val metrics (deterministic
backtest). val_PF=1.18-1.33 with WR=46-48% across 30 epochs of
xmd6b shows the policy DOES extract asymmetric-payoff alpha — we
just couldn't see it through the meter inflation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restore 45-action factored space via Branching DQN (Tavakoli 2018),
outputting 11 Q-values (5+3+3) instead of 45. This was reduced to 5
exposure-only actions during debugging and was never intended as permanent.
- Enable use_branching: true by default in DQNConfig and DQNHyperparameters
- Add branching paths to select_action_with_confidence and select_action_inference
- Update agent.rs select_action_factored for branching-aware selection
- Expand CountBonus to per-branch tracking with bonuses_branched()
- Add order_type + urgency distribution tracking in monitoring
- Add DQN_ORDER_ACTIONS=3, DQN_URGENCY_ACTIONS=3, DQN_TOTAL_ACTIONS=45 to CUDA header
- Fix 7 pre-existing clippy doc_markdown errors in regime_conditional.rs
- Fix pre-existing cognitive_complexity in replay_buffer_type.rs (extract helpers)
- Fix flaky GPU test OOM under parallel execution (CPU fallback + test VRAM safety)
- Delete unused flash_attention submodules (block_sparse, causal_masking, etc.)
- Add GPU hot-path guard scripts and ensemble/hyperopt adapter improvements
Tests: ml-dqn 416/0, ml 905/0, clippy 0 errors on both crates
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
In candle-core (git 671de1d), min(D) and max(D) return Result<Tensor>,
not Result<(Tensor, Tensor)>. Use flatten_all() then min(0)/max(0) for
scalar reduction instead of the tuple destructuring pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Task 14: evaluate_baseline is at crates/ml/examples/evaluate_baseline.rs
(not bin/fxt/src/commands/), fix cargo check command and git add path
- Task 15: process_bar_factored takes (usize, &OHLCVBarF32, &FactoredAction)
not (f64, usize, f64, f64), fix test to use correct types
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
15-task plan covering Phase 1 (seal GPU→CPU roundtrips in training loop),
Phase 2 (vectorized GPU backtest environment for hyperopt), and Phase 3
(general GPU backtester integration).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three-phase plan to eliminate all GPU→CPU roundtrips from training:
- Phase 1: Seal training loop (persistent GPU epoch state, async monitoring)
- Phase 2: Vectorized CUDA backtest kernel for hyperopt evaluation
- Phase 3: General-purpose GPU backtester replacing CPU SIMD path
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bring Branching Dueling Q-Network (Tavakoli 2018) to full Rainbow parity
with the existing GPU hotpath. 3 independent advantage heads (exposure=5,
order=3, urgency=3) decompose the 45-action space into learnable branches.
H1 - CUDA fallback: gate GpuExperienceCollector when use_branching=true
(fused kernel hardcodes NUM_ACTIONS=5, incompatible with 45 factored)
H2 - Per-branch C51 distributional: each branch outputs [batch, n_d, atoms]
log-softmax, loss = avg of D cross-entropies vs projected Bellman target
M1 - NoisyNet: MaybeNoisyLinear enum in branch heads, reset_noise/disable_noise
wired through select_action, compute_loss, and set_eval_mode
M2 - Regime-conditional IS weights: Trending=1.2, Ranging=0.8, Volatile=0.6
applied to branching loss via ADX/CUSUM features at state[40:41]
M3 - State dim alignment: align_dim_for_tensor_cores() in from_dqn_params()
for H100 HMMA dispatch (8-byte alignment)
L1 - Fill simulator: splitmix64 replaces golden ratio hash (chi-squared tested)
L2 - Hyperopt 29D: branch_hidden_dim [64,256] added to PSO search space
Config plumbing: branch_hidden_dim, v_min/v_max/num_atoms, use_distributional,
use_noisy, noisy_sigma_init all flow from DQNConfig → BranchingConfig.
10 files, +3207/-125 lines, 33 branching tests + 387 ml-dqn + 284 ml-core pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Plan to reduce ml crate from 91K to ~12K LOC by extracting trainers
into model sub-crates, hyperopt adapters into ml-hyperopt, and
infrastructure into existing sub-crates. ml becomes an orchestration
layer owning inference, model factory, and training pipeline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split ml-rl into ml-dqn and ml-ppo for 3-way parallel compilation
and better sccache hit rate. Extract TradingAction to ml-core to
enable full DQN/PPO independence.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split the monolithic ml crate (260K lines, 55s compile) into 5 crates:
ml-core, ml-rl, ml-supervised, ml-infra, ml (facade).
15-task plan with full module inventory and import migration guide.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace MinIO binary distribution with GitLab Generic Package Registry.
Every code push to main auto-creates a CalVer tag (vYYYY.MM.N),
compiles, uploads binaries to GitLab packages, creates a Release
with auto-generated notes, and deploys via deployment patching.
- New CI templates: create-tag, upload-release
- Modified: compile-services/training upload to GitLab packages
- Modified: deploy-services patches FOXHUNT_RELEASE on deployments
- All 7 service initContainers fetch from GitLab (curl, deploy token)
- Training job-template binary fetch from GitLab (data stays MinIO)
- MinIO retains: sccache, training data, model checkpoints
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Walk-forward windows: replaced 3 non-overlapping with sliding (50% overlap, ~5 windows).
Aggregation changed from mean-0.5*std to median-0.5*IQR for outlier robustness.
2. Composite score: tanh normalization prevents Calmar ratio scale dominance
(0.02% drawdowns → values in thousands drowning out Sharpe/Sortino).
3. Q-value overestimation: new Prometheus gauge foxhunt_training_q_overestimation_ratio,
warning log when ratio>10 or q_mean>5, adaptive tau doubles when Q-mean growth>0.5/epoch
(capped at 0.01), decays back when stable.
2742 tests pass, 0 failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Addresses eval/training mismatch (B1-B3), reward architecture (C1-C3),
and early stopping (C4). See design doc for full analysis.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Unify both gateways into a single gRPC-only `services/api/` with tonic-web
for browser access. Drop REST+WebSocket, keep full 6-layer auth.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaces monolithic compile-services with 8 per-service compile jobs,
each with dependency-aware changes: filters. Deploy job only restarts
services whose binary actually changed. Also renames service crates
from snake_case to kebab-case to match k8s deployment names.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add live training metrics monitor CLI command (streaming & one-shot) using
the monitoring gRPC service. Update DQN tests to match post-fix defaults:
IQN disabled, CQL alpha=0.1, v_min/v_max widened, 26D search space.
- train.rs: `fxt train monitor [--once] [--model X] [--interval N]`
- Rewrite gradient collapse test for BF16 mixed precision awareness
- Update inference test config to match trainer defaults (IQN off, CQL on)
- Update production smoke test for 26D parameter space
- Add dqn_action_collapse_fix_test.rs verifying all 6 root cause fixes
- Add planning docs for monitoring service and epoch financial metrics
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Helm values (controller + server on platform node, MinIO artifact repo)
- WorkflowTemplate: parameterized 5-step DAG (fetch→hyperopt→train→eval→upload)
- Nginx proxy for argo.fxhnt.ai → Argo Server :2746
- DNS A record for argo.fxhnt.ai
- MinIO bucket foxhunt-training-results for Argo artifacts
- Kustomization for kubectl apply -k
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Full GitLab CI → Argo migration: Argo Events webhook trigger,
per-service compile WorkflowTemplates, selective deploy, test
workspace, training compile, and IaC templates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Eliminate foxhunt-binaries and training-binaries PVCs — services and
training jobs now fetch binaries from MinIO via rclone initContainers.
This removes L40S GPU autoscale-for-PVC-writes, removes RWO node
affinity constraints, and requires zero image rebuilds.
Changes:
- .gitlab-ci.yml: replace ~115 lines of binary-writer pod logic with
aws s3 cp to MinIO (~25 lines)
- 8 service YAMLs + 2 GPU overlays: add rclone initContainer + emptyDir
- Training job template: MinIO rclone fetch replaces PVC copy
- Delete foxhunt-binaries-pvc.yaml and training-binaries-pvc.yaml
- Add s3.fxhnt.ai DNS record (Terraform) and nginx proxy block
- Replace pod-writer-deploy skill with MinIO-based deploy skill
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
17-task plan for replacing hardcoded mock data in fxt watch
with real gRPC streaming data from the API Gateway.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Streaming-first architecture: 11 new poll-to-stream gateway adapters,
channel-based DataFetcher in the TUI, auto-reconnect with backoff.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fxt 2.0 overhaul: consolidated proto/, absorbed monitoring into API Gateway,
new 15-command CLI with gRPC, MCP server mode, TUI cockpit framework.
159 files changed, net -11,835 lines.
