e82049c7791b9b9bb1aeaea283f74df44f264d30
257 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
e41a732081 |
feat(ml-alpha): Phase 4-A — No-transaction-band foundation (ISV slot 799 gate)
Foundation of the no-transaction-band architectural turnover regulator
described in
docs/superpowers/specs/2026-06-03-no-transaction-band-architecture.md.
Davis-Norman (1990) / Imaki-Imajo-Ito (2021, arXiv:2103.01775) — when the
current position lies inside a learned band [b_l, b_u], the architectural
default is "do nothing", complementing Phase 3D's reward-side fixes which
the literature (Goodhart-Skalse 2024) bounds the effectiveness of.
Scope: ISV slots 799-808 + BandHead forward + ±|tanh|·N_max_eff
activation + rl_band_mask action override + rl_band_turnover_loss
(Option b, fixed-target) + 5 GPU-oracle invariants + diag emission.
Master gate `RL_BAND_ENABLED_INDEX` (slot 799) bootstraps to 0.0 (OFF)
so the foundation preserves bit-equality with Phase 3D `bd811a774` until
operator opt-in via `FOXHUNT_BAND_ENABLED=1`.
Adaptive controller, encoder backward chain, and cluster verification
are Phase 4-B / 4-C, not in scope here.
ISV slots (799-808, bumps RL_SLOTS_END to 809):
799 RL_BAND_ENABLED_INDEX (master gate, bootstrap 0.0)
800 RL_BAND_LOWER_INIT_INDEX (b_l init, -0.5 in tanh space)
801 RL_BAND_UPPER_INIT_INDEX (b_u init, +0.5 in tanh space)
802 RL_BAND_LOSS_WEIGHT_INDEX (λ_turnover, 0.01)
803 RL_BAND_TURNOVER_TARGET_INDEX (target frac unmasked, 0.05)
804 RL_BAND_GRAD_SHARPNESS_INDEX (sigmoid surrogate, 4.0)
805 RL_BAND_FLAT_RECENTER_RATE_INDEX (reserved for 4-B controller)
806 RL_BAND_WIDTH_MIN_INDEX (collapse detection, 0.1)
807 RL_BAND_WIDTH_MAX_INDEX (saturation detection, 1.8)
808 RL_BAND_DIAG_LAUNCH_EVERY_INDEX (diag cadence, 1.0)
CUDA kernels (3 new):
rl_band_head_forward.cu — 2-stage forward: linear projection
(tree-reduce over HIDDEN_DIM, matches
ppo_policy_logits_fwd shape) + asymmetric
±|tanh|·N_max_eff activation enforcing
b_l ≤ 0 ≤ b_u (Davis-Norman invariant).
rl_band_mask.cu — overrides actions[b]→Hold when
position_lots[b] ∈ [b_l, b_u]. Master-
gated at slot 799; no atomicAdd; runs
OUTSIDE graph capture per spec §9.5.
rl_band_turnover_loss.cu — Option (b) turnover regularizer with
sigmoid surrogate. Per-batch loss +
per-batch grad on (b_l, b_u). Phase 4-A
wires kernel + test; full encoder grad
fold is Phase 4-B.
Rust glue:
crates/ml-alpha/src/rl/band_head.rs — BandHead struct, forward,
launch_mask, launch_turnover_loss.
Xavier-0.01 init under
scoped_init_seed(seed+0xBA_5EED).
trainer/integrated.rs — BandHead field + construction
(bias init = atanh(0.5)) +
ISV bootstrap row + FOXHUNT_BAND_
ENABLED override + forward call
at both step_with_lobsim and
step_with_lobsim_gpu_body sites
+ mask launch BEFORE confidence
gate + per-step band aggregate
diag (lower/upper/width means,
frac_in_band, frac_masked,
collapse_warning).
Tests (5 GPU-oracle invariants, all PASS):
band_activation_clamps_correctly — b_l ≤ 0 ≤ b_u, |·| ≤ N_max_eff
band_mask_forces_hold_when_in_band — pos 0 ∈ [-4,+4] → action becomes Hold
band_mask_passes_through_when_out_of_band — pos 5 ∉ [-1,+1] → action unchanged
band_turnover_loss_correct — loss + grad match analytical form
band_disabled_means_no_mask — slot 799 = 0.0 → mask no-op
Verification:
cargo build --release --example alpha_rl_train: exit 0
cargo test band_invariants --release -- --ignored: 5/5 PASS
cargo test multi_head_policy_invariants -- --ignored: 18/18 PASS (regression)
determinism-check.sh --quick (band OFF): exit 0
FOXHUNT_BAND_ENABLED=1 determinism-check.sh --quick: exit 0
FOXHUNT_USE_MULTI_HEAD_POLICY=1 determinism-check.sh --quick: exit 0
FOXHUNT_USE_MULTI_HEAD_POLICY=1 FOXHUNT_BAND_ENABLED=1 …: exit 0
Phase 4-A local mid-smoke (b=128, 2000+500 steps, band enabled):
exit 0, no NaN observed
frac_masked = 1.0 throughout (band wide at [-4,+4] init)
total_trades = 0 (vs Phase 3D 11,767) — primary kill criterion PASS
band width drifts 8.02 → 7.58 over 2000 steps (slight narrowing from
encoder shared-h_t shift; band-head's own backward chain is Phase 4-B)
G_no_band_collapse FAILS (frac_masked saturates at 1.0) — expected at
Phase 4-A because turnover-loss gradient is not yet folded into the
encoder (per spec §7.1 / §9.1 Mitigation 1, which requires Phase 4-B
adaptive controller + backward chain).
Pearls:
pearl_bootstrap_must_respect_clamp_range (every bootstrap ∈ clamp)
pearl_scoped_init_seed_for_reproducibility (band-head init guard)
pearl_determinism_achieved (no PRNG / no atomicAdd in kernels)
pearl_foxhunt_pi_trained_by_q_distillation_not_ppo (band is structural,
not reward-side — sidesteps Goodhart-Skalse attenuation)
pearl_fleet_fraction_not_aggregate (frac_in_band / frac_masked emitted)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
0b3e401500 |
feat(ml-alpha): Phase 2A-A — MultiHeadPolicy foundation (inert)
K=3 policy mixture + regime-gated routing head, with gate reading
REGIME_DIM=6 features DIRECTLY (bypassing the VSN softmax bottleneck
per the regime-attenuation empirical finding). Components are
unconditional but not yet wired into the trainer — Phase 2A-B
adds backward + aux KL prior, 2A-C wires Q-distill grad routing
to the mixture.
* 4 new ISV slots 761-764 (K, gating entropy floor, head entropy
floor, aux prior β) + RL_SLOTS_END 761→765.
* multi_head_policy_forward.cu: per-batch K-head logits + mixture
combination. Grid=(B), Block=(N_ACTIONS=11). Persists pi_logits_k
+ pi_probs_k for backward.
* multi_head_policy_gate_forward.cu: per-batch K-thread softmax
over W·regime + b. Reads regime_h directly (parallel channel —
bypass VSN). Stores pre-softmax logits to gmem BEFORE the
in-place exp (caught during impl — plan pseudocode would have
corrupted gate_logits).
* MultiHeadPolicy struct with Option A asymmetry-break init:
Head 0 → ShortLarge bias (+0.5), Head 1 → LongSmall+LongLarge
bias (+0.5 each), Head 2 → Hold bias (+0.5). Indices verified
against rl/common.rs:56-70 N_ACTIONS=11 enum. htod via local
upload() helper using MappedF32Buffer + raw_memcpy_dtod_async
(mirrors rl/ppo.rs, rl/dueling_q.rs).
* 5 GPU-oracle invariant tests, all PASS:
- gate_probs_sum_to_one
- pi_probs_mixture_sums_to_one
- k1_reduces_to_single_head (bit-equal vs reference Linear)
- gate_responds_to_regime_change (TVD > 0.5, modal head flips)
- forward_is_deterministic_across_contexts (bit-equal across
two fresh CUDA contexts)
* Determinism preserved: ./scripts/determinism-check.sh --quick
exits 0 (pipeline unchanged, components inert).
* No memcpy_htod/memcpy_dtoh on regular slices, no atomicAdd, no
scoped-init-seed bypass.
Empirical motivation (3-seed mid-smoke at HEAD
|
||
|
jgrusewski
|
969caf26c3 |
docs(ml-alpha): spec + plan for Phase 1B trainer rollout-buffer + GAE refactor
Companion docs for commit |
||
|
jgrusewski
|
629ebd667c |
feat(ml-alpha): deterministic same-seed training + Tier 1.5 fast-dev-cycle
Two same-seed runs now produce bit-equal eval_summary.json, alpha_rl_train_summary.json,
and diag.jsonl (modulo wall-clock elapsed_s). The 5-phase falsification chain landed:
Phase 2 PER tree-rebuild: __threadfence is NOT a grid-wide barrier; multiple blocks
raced across sum-tree levels. Fix: Grid=(1) Block=(1024) + __syncthreads
in rl_per_tree_rebuild.cu.
Phase 2.3 cuBLAS GEMM_DFALT + TF32 default-math allowed split-K non-deterministic
accumulation at 3 sites. New crates/ml-alpha/src/cublas_determinism.rs
applies CUBLAS_PEDANTIC_MATH via FOXHUNT_DETERMINISTIC env toggle
(0=TF32 prod, 1=PEDANTIC dev default, 2=DEFAULT_MATH control).
Phase 2.6 Two bugs surfaced sequentially in the backward kernel chain:
(1) rl_iqn_tau_cos_features had a multi-block r/w race on prng_state[batch]
— all N_TAU=32 blocks read seed; only tau_idx==0 wrote back; no
inter-block barrier. Fix: split into READ-ONLY rl_iqn_tau_cos_features
+ new sibling rl_iqn_advance_prng_state launched on same stream
(kernel-launch ordering = grid-wide barrier).
(2) OutcomeHead::new called near_zero_xavier without scoped_init_seed,
falling back to time+thread-id RNG. Stayed dormant until first done
event activated non-sentinel labels and divergent weights flowed via
grad_h_t_outcome into encoder gradient. Fix: add seed param + install
scoped_init_seed(dqn_seed.wrapping_add(0x0CE0)) guard.
Validation (./scripts/determinism-check.sh --quick, RTX 3050, b=128, 200+50 steps):
- All 200 rows of checksums.* leaves match (rel-tol 1e-5, abs-tol 1e-7)
- eval_summary.json, alpha_rl_train_summary.json byte-equal between runs
- diag.jsonl byte-equal modulo elapsed_s
- Eval pnl identical run-A vs run-B at seed 42
Pre-fix baseline (Phase 2.5 measurement): same-seed eval pnl spread $450k
($187k vs -$261k). Post-fix: $0 spread.
Speed cost: ~1.5ms/step amortised; ~10-15% slower than TF32 production
(PEDANTIC tax — acceptable in dev, toggle to FOXHUNT_DETERMINISTIC=0 for prod).
Mapped-pinned discipline: all 11 NEW memcpy_dtoh sites in diagnostic dump methods
+ per-step checksum readback use a new pub(crate) helper
read_slice_d_into<T: Copy>(stream, src, dst) — MappedRecordBuffer + raw
memcpy_dtod_async + raw_stream_sync + volatile read. Generic over T (f32, f64,
i32, u32, u8). Satisfies feedback_no_htod_htoh_only_mapped_pinned + hook guard.
Bundled Tier 1.5 fast-dev-cycle infrastructure (spec
docs/superpowers/specs/2026-06-02-fast-dev-cycle.md):
- scripts/local-mid-smoke.sh b=128, 2000+500, ~10min on RTX 3050
- scripts/determinism-check.sh runs mid-smoke twice, diffs checksums
- scripts/tier1_5_verdict.py behavioral kill verdict
- AdamW checkpoint save/load (crates/ml-alpha/src/trainer/optim.rs)
- IntegratedTrainer checkpoint save/load (resume from checkpoint)
- 15 Phase 1 checksum leaves in build_diag_value
- Env-gated dump methods (FOXHUNT_DETERMINISM_DEBUG_PER/MAMBA2/RL/BACKWARD)
for future divergence-chasing — never run in production
Documentation:
- docs/superpowers/specs/2026-06-02-determinism-foundation.md
- docs/superpowers/specs/2026-06-02-fast-dev-cycle.md
- docs/superpowers/plans/2026-06-02-determinism-foundation-implementation.md
- docs/superpowers/notes/2026-06-02-determinism-phase{1,2,2.2,2.5,2.6}-*.md
- Adjacent specs/plans/notes from the analytical chain that surfaced determinism
as the load-bearing blocker (eval-summary, eval-boundary, regime-observer,
multi-head policy, regime-invariance, Phase 3 IQN-complement post-mortem)
Unlocks: every controller / architecture / reward-shaping A/B from this commit
onward attributes outcome differences to the change, not random-init kernel-race
drift cascading through training x eval LOB-sim trajectories. The eval-collapse
investigation (pearl_reward_signal_anti_aligned_with_pnl, multi-head spec,
regime-invariance spec) is now testable with trustworthy verdicts.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
fa347e4812 |
feat(rl): reward-policy alignment — pure-pnl mode default (spec 2026-06-01)
Empirical diagnosis (local analysis of two 20k+5k cluster runs on PVC, SHAs |
||
|
jgrusewski
|
34806b6b62 |
diag(rl): edge-decay detector Phase 1 — Page-Hinkley on per-trade EV
Pure observability. No behavior change. Implements the missing
"short-run trust adjustment" layer between Kelly long-run sizing and
CMDP tail kill, per pearl_edge_decay_detection_is_a_missing_abstraction_layer.
The 64 commits since
|
||
|
jgrusewski
|
f428be794b |
Revert "feat(rl): B-11-β Q-distill informativeness gate"
This reverts commit
|
||
|
jgrusewski
|
b93971726d |
feat(rl): B-11-β Q-distill informativeness gate
First behavior change since B-7 (preceding B-8/B-9/B-10 were
observability-only). Attenuates the distill gradient when softmax(Q/τ)
is near-uniform — when target_entropy → ln(N_ACTIONS), the distill term
is pure max-entropy regularization with no informational signal, so it
fights PPO instead of carrying Q's preferences into π.
Verified cluster cause @ alpha-rl-88f5c (B-10 smoke,
|
||
|
jgrusewski
|
8c7ce02da9 |
feat(rl): B-10 policy-quality cascade diagnostic
alpha-rl-8gtk2 (B-7+B-8+B-9 run at SHA
|
||
|
jgrusewski
|
29b5acad55 |
feat(rl): B-9 C51 Bellman-target saturation observability
Under B-7 (clamp default-disabled), per-transition rewards in bellman_target_projection / bellman_fused_select_project can exceed the adaptive atom support [V_MIN_eff, V_MAX_eff]; each t_z overshoot is silently clamped before being mapped onto the discrete support. B-9 publishes the per-step saturation rate + pre-clamp t_z extremes so we can decide if the atom span has become a bottleneck — without re-attempting the reverted Fix F atom-widening (pearl_c51_v_max_freeze _required_for_surfer). Changes: - 4 new ISV slots (726-729): top/bot saturation rate + max/min pre-proj. - bellman_target_projection.cu: both entry points (bellman_target_projection AND bellman_fused_select_project per feedback_no_partial_refactor) gain 4 new [B] f32 pointer params; thread-0 sequential reduction over Q_N_ATOMS=21 (odd count rules out symmetric tree-reduce — matches the kernel's existing softmax max/sum pattern at lines 157/171). - New cross-batch reducer cuda/rl_bellman_target_saturation_reduce.cu: single block, grid-stride gather + power-of-2 tree reduce, no atomicAdd per feedback_no_atomicadd. - dqn.rs: load reducer cubin, add saturation_reduce_fn handle, launch_saturation_reduce method, 4 scratch pointer params on both bellman methods. - integrated.rs: allocate 4 [B] f32 scratch buffers; pass through both fused_select_and_project_bellman call sites + launch reducer after each. 4 new bootstrap entries (213 → 217 fixed-size array). - build.rs: register new kernel. - 4 new diag leaves under risk_stack.atom_calibration.target_*. Comment distinguishes them from popart.max_abs_reward_ema (different signal: Bellman target = r + γ·atom_value, can exceed reward by γ·V_MAX_eff). - EXPECTED_LEAVES 653 → 657. - tests/c51_atom_saturation_diagnostic.rs: GPU-oracle test asserts 4 invariants over 249 rows — rates in [0,1], max≥min, rate>0 ⇒ overshoot exists, top+bot ≤ 1. Validation: - 200+50 b=16 fold-1 smoke clean. Locally V_MAX_eff adapts to ~19.3 (atom support is ISV-driven via rl_atom_support_update), so saturation is 0% in the smoke; both diag leaves emit + invariants hold. - popart_disaggregation_invariants: still passes (249 rows, identity). - eval_diag_emission: train = eval = 657 leaves. - Determinism preserved: kernel adds shared-mem reduction over per-thread t_z values that were already computed; target_dist output unchanged. Spec: docs/superpowers/specs/2026-06-01-b9-c51-atom-saturation-diagnostic.md Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
1739d9c173 |
feat(rl): B-8 popart σ_welford disaggregation + identity invariant
Under B-7 (clamp default-disabled), popart's Welford state now updates against unclamped magnitudes; σ_effective = max(σ_welford, env.max) can spike from either source but diag only emitted the combined value. This blocks attribution of any future σ shocks. Changes: - RL_POPART_SIGMA_WELFORD_INDEX (slot 725): Welford-only σ, BEFORE the F4 envelope floor at rl_popart_normalize.cu:156. Pure observability — no computation change. - rl_popart_normalize.cu: insert one ISV write between σ_welford computation (line 141) and envelope-floor application (line 156). Mirrors the existing #define-local-then-write pattern (POPART_SIGMA_INDEX). - build_diag_value: new leaf popart.sigma_welford in the canonical popart block. NOT duplicating max_abs_reward_ema (already emitted at risk_stack.regime.popart_envelope.max_abs_reward_ema per feedback_single_source_of_truth_no_duplicates). - EXPECTED_LEAVES 652 → 653. - Bootstrap array [(usize, f32); 212] → 213 with sentinel 0.0 (overwritten every step by popart kernel). - tests/popart_disaggregation_invariants.rs: GPU-oracle test asserts identity popart.sigma == max(σ_welford, env.max) across 249 rows (200 train + 50 eval), skipping step 0 bootstrap. - tests/eval_diag_emission.rs: migrate fold-idx 0/n_folds 2 → 1/3 (the n_folds=2 split picks the first 4 files which don't satisfy loader's 1033-snapshot minimum after test_data grew from 2 → 9 files). Validation: - popart_disaggregation_invariants passes locally (249 rows OK). - eval_diag_emission passes locally: train=653 eval=653 leaves. - B-9 spec at docs/superpowers/specs/2026-06-01-b9-c51-atom-saturation-diagnostic.md builds on slots 726-729 next (uncommitted, awaiting implementation). Spec: docs/superpowers/specs/2026-06-01-b8-popart-calibration-observability-and-floor.md Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
bc9eaac89d |
fix(rl): B-6 — ISV-driven adaptive asymmetric Wiener-α (Bayesian shrinkage)
B-5 (asymmetric α with static α_slow=0.001) revealed the static parameter problem: provably bounds cascades (avg_win peak $2k vs B-4's $40k) BUT over-conservative in train (dckcc step 800: avg_l > avg_w → Kelly says don't trade → model can't discover edges; wr_ema crashed to 0.145). The fundamental tension: static α_slow can't satisfy BOTH - Train convergence: asymmetry must FADE so model learns from real data - Boundary safety: asymmetry must ENGAGE at every fold to prevent cascade B-6 RESOLVES this via Bayesian shrinkage: trust(n) = min(1, cum_dones / n_full_threshold) [Phase 1] stability = exp(-CV × cv_gain) [Phase 2] trust_eff = trust(n) × stability α_slow_eff = α_slow_min + (α_fast − α_slow_min) × trust_eff Phase 1 (data-quantity): trust grows with cum_dones; reset_session_state zeroes cum_dones → asymmetry RESUMES at every boundary. Math: at n=0 α_slow_eff = α_slow_min = 0.001 (full skepticism). At n=n_full = 30k trades: α_slow_eff = α_fast = 0.05 (full standard Wiener). Phase 2 (data-quality): Welford CV of reward magnitude gates trust. Stable signal (CV→0): stability=1, trust opens normally. Volatile signal (CV high): stability→0, asymmetry persists. cv_gain=0 disables Phase 2. Per-EMA asymmetry direction encodes Kelly safety semantics: avg_win: slow-up (skeptical of wins), fast-down avg_loss: fast-up (admit losses), slow-down (slow forget) wr_ema: slow-up (skeptical of high WR), fast-down ISV slots (all signal-derived from cum_dones + Welford): 721 RL_EMA_ALPHA_SLOW_MIN_INDEX = 0.001 722 RL_EMA_TRUST_FULL_THRESHOLD_INDEX = 30000 723 RL_EMA_CV_GAIN_INDEX = 1.0 Threshold calibration (n_full=30k): - Train: ~1000 cluster steps for trust to fully open → asymmetry active during cold-start (first 30 steps, cascade prevention) then fades. - Eval: 30 dones/step × 500 eval steps = 15k dones → trust climbs to 0.5 by eval end → partial protection throughout eval. Composes: - B-3 Kelly fractional-trust (Kelly SIZING gated by cum_dones) - B-6 EMA asymmetric-α (Kelly INPUTS biased conservative by cum_dones) Both fade as data accumulates; both reset at boundary. Spec: docs/superpowers/specs/2026-06-01-ema-asymmetric-trust-with-cv-gain.md Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
16cf9f260c |
fix(rl): B-2 — pos_max_ema cold-start cascade eliminated, ISV-driven cap
The addendum's rate-cap (B-1) only protected the Wiener-α path; the first-
observation bootstrap branch let cold-start fat-tail events seed pos_max_ema
unbounded. At alpha-rl-4xmxm step 5, a single $947 scaled reward bootstrapped
pos_max_ema=879 directly, cascading through clamp_win → unclamped subsequent
rewards → env.max=11375 by step 37 (1500× the eventual steady-state σ).
B-2 fix per docs/superpowers/specs/2026-05-31-pos-max-ema-cold-start-redesign.md:
1. Bootstrap RL_POS/NEG_SCALED_REWARD_MAX_EMA_INDEX to MIN_WIN=1.0 (was 0)
in `with_controllers_bootstrapped`. Conservative neutral value → clamp_win
starts at MARGIN × 1.0 = 1.5 → rewards heavily clipped until adaptation.
2. Remove the `if (ema_prev == 0.0f) ema_new = pos_max;` branch from
`rl_reward_clamp_controller.cu`. Single uniform update rule (Wiener-α +
rate-cap) applies from cold-start onward. Mirror change for neg_max_ema.
3. Replace `#define POS_MAX_EMA_MAX_GROWTH_PER_STEP 1.5f` with ISV-driven
reads (per feedback_isv_for_adaptive_bounds). Three new slots:
717 RL_POS_MAX_EMA_COLD_START_INDEX = 1.0
718 RL_POS_MAX_EMA_GROWTH_CAP_BASE_INDEX = 1.225 (√1.5 for
twice-per-step inv)
719 RL_POS_MAX_EMA_GROWTH_CAP_CV_GAIN_INDEX = 0.0 (adaptive layer
disabled by default)
4. Adaptive growth_cap from Welford CV of reward magnitude (slot 615-617)
when cv_gain > 0: stable regime → tight cap, volatile regime → loose.
Disabled by default; opt-in via ISV tuning.
Local smoke validation (800+200 fold-1 b=16):
- step 1: pos_ema=1.0 (initialized, NOT bootstrapped from observation)
- step 5: pos_ema=1.0 (no observation yet, sparse-skip working)
- step 25: pos_ema=63.8 (vs 1314 without B-2 — 21× reduction)
- step 37: pos_ema=32.5 (vs 7074 without B-2 — 218× reduction)
- env.max @ step 37: 126 (vs 11375 without B-2 — 90× reduction)
Generalizes the pattern: NORMALIZATION EMAs that gate signal magnitudes
should bootstrap to CONSERVATIVE neutral values, NEVER to first observation.
Cross-references pearl_first_observation_bootstrap which needs revision.
Phase 4 audit (other controllers with same anti-pattern) deferred to
follow-up — see spec §4 Phase 4 for the grep target list.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
1aa92f57f0 |
fix(rl): eval-boundary addendum — reward_scale warmed_flag + pos_max_ema rate-cap
The parent eval-boundary fix (
|
||
|
jgrusewski
|
13d8ed76da | docs: checkpoints + eval-diag spec + plans (v1 superseded by v2) | ||
|
jgrusewski
|
6c4945fe16 |
docs(spec): v9 defensive eval-boundary calibration (completes adaptive principle)
Per the newly-saved memory pearl pearl_adaptive_carryover_discipline, diagnoses Fix D's "intentionally preserve train Kelly + inventory EMAs into eval" as the dominant cause of v8's eval-phase regression. Design — 3 layers at the train→eval boundary: Layer 1: Reset every adaptive EMA (Kelly wr/avg-win/avg-loss/ cumulative_dones, inventory β/variance, reward-clamp pos/neg/clip EMAs) to neutral sentinels. Lets the controllers re-bootstrap from eval-distribution observations rather than carrying poisoned train state. Layer 2: Defensive warmup window (500 steps) overrides risk-sizing controllers — Kelly safety_frac 0.5→0.25, IQN τ_min 0.10→0.30, entropy coef floor 0.01→0.05, PPO ε floor 0.05→0.10. Linear decay back to normal over additional 200 steps. Layer 3: 2× LR multiplier during warmup window. Network weights adapt fast to new-regime statistics; this is the slowest-adapting layer of the agent. Adds 8 new ISV slots (686-693), bumps RL_SLOTS_END 686→694. New small kernel rl_eval_warmup_decay applies overrides each step during warmup. Pure additive design — no kernel logic changes outside the new warmup-decay kernel. Validation requires running ALL 3 folds (vs v8's single fold-1) per pearl_single_window_oos_is_not_oos. Success criterion: mean eval pnl across 3 folds > v8 mean. No code change in this commit — design document only. Implementation deferred until decision is made on whether to ship v9 or first collect v8's fold-0 + fold-2 baseline for cleaner comparison. |
||
|
jgrusewski
|
82572ff3bd |
docs(spec): C51 atom span math validation + empirical proof
Formal proof of why the current `atom_max = EWMA(WIN_bound)` design trains successfully despite empirical violations of the "structural minimum atom_max ≥ WIN/(1-γ)" claim made in 2026-05-30-c51-atom- resolution-design-alternatives.md. Key findings (proven by local smoke + cluster v8 trajectory): 1. The fixed-point bound `WIN/(1-γ)` is overstated as a structural constraint. Empirical: smoke step 999 atom_max = 4.5× the bound, system trains healthy (qpa=+0.969). 2. The actually-binding constraint is the dynamic bound: `atom_max ≥ max V_target_observed ≤ WIN + γ × V_max_observed` Self-consistent and satisfied with margin 23 at smoke step 999. 3. Both speculative alternatives (Fix F = mean_abs_pnl anchor, Fix F-v2 = V_target_observed anchor) have closed-form instability at γ(1+ε) ≥ 1 — for γ=0.99 only ε ≤ 0.01 stable, no resolution benefit over current design. 4. Current design is *conservative* (atom_max ~5× larger than V_target 3σ requires) but *correct* — slow EWMA hysteresis absorbs WIN transients and keeps Q-V Bellman self-consistent. Includes empirical trajectory data: - Local smoke (b=128, HEAD |
||
|
jgrusewski
|
d57bee0542 |
docs(spec): atom resolution design alternatives (post Fix F failure)
Documents why Fix F crashed at cluster scale (qpa: +0.898 → -0.976
between step 371 and step 500 in alpha-rl-qrgjr v7), the structural
Bellman self-consistency constraint atom_max >= WIN/(1-γ) that makes
the resolution trade-off fundamental, and the design alternatives
considered (non-uniform atoms, adaptive γ, two-headed Q). Recommends
shipping A+B+C+D+E only — the pre-Fix-F design implements the
structurally-correct atom span via WIN-tracking EWMA.
No code change in this commit — design document only. Empirical
result from v8 (alpha-rl-vxbpq, SHA
|
||
|
jgrusewski
|
6d4a962e5c |
Revert "fix(rl): decouple C51 atom span from reward-clamp ceiling"
This reverts commit
|
||
|
jgrusewski
|
0fe825a8c5 |
fix(rl): decouple C51 atom span from reward-clamp ceiling
Spec: docs/superpowers/specs/2026-05-30-c51-atom-span-decouple-from-clamp.md
Diagnosed in cluster v5 (alpha-rl-rjsjq SHA
|
||
|
jgrusewski
|
285d42aa7b |
feat(rl): adaptive risk-management stack — 5 layers, all ISV-driven
Layered risk stack per spec docs/superpowers/specs/2026-05-30-adaptive-risk-management-design.md:
Layer 1 (CMDP) hard session-DD, cooldown, max-open, inventory limits
Layer 2 (IQN τ) risk-averse action selection adapts to session drawdown
Layer 3 (Inventory) Avellaneda-Stoikov penalty β scales with reward magnitude
and inventory variance
Layer 4 (Kelly) half-Kelly fraction sizing from observed win-rate +
R-multiple, warmup-gated until cumulative_dones >= 1000
Layer D (Trail) wire dead a7/a8 (TrailTighten/Loosen) via independent
ISV factors, replacing the symmetric reciprocal
Architecture: every threshold ISV-driven (22 new slots, RL_SLOTS_END 662→684).
Every adaptive bound follows the canonical Wiener-α blend with floor 0.4,
sentinel-zero bootstrap, and asymmetric Schulman where applicable.
Kernels:
rl_cmdp_constraints_check session pnl + cooldown + consec-loss tracking
rl_iqn_action_tau_controller τ = clamp(0.5 - 5·dd_frac, τ_min, 1.0)
rl_inventory_beta_controller β_target = 0.01·E|reward| / (2·σ_inventory)
rl_kelly_fraction_controller f = clamp(safety · (p·b - q)/b, 0, 1)
rl_win_rate_ema_update closed-trade win-rate EMA from rewards + dones
rl_avg_win_loss_ema_update separate avg-win and avg-loss EMAs
rl_inventory_variance_update Welford variance of net-position-per-batch
Integration:
- All 7 new cubins loaded in IntegratedTrainer + launched per step in spec order
(CMDP after reward pipeline, before actions_to_market_targets reads override
flags; Layer 2/3/4 controllers in rl_fused_controllers.cu).
- actions_to_market_targets.cu: Layer 1 hard overrides (DD-triggered → 0 lots;
cooldown → 0 lots; max-open → block opening actions; inventory cap → block
one-sided expansion) and Layer 4 Kelly fraction scaling on target lots.
- rl_fused_reward_pipeline.cu: Layer 3 inventory penalty term in reward shaping.
- rl_trail_mutate.cu: a7 multiplies trail by RL_TRAIL_TIGHTEN_FACTOR_INDEX,
a8 by RL_TRAIL_LOOSEN_FACTOR_INDEX (was symmetric reciprocal — pearls
pearl_dead_trail_stop_actions_a7_a8).
Validation:
- 12 GPU-oracle invariants pass (tests/risk_stack_invariants.rs):
G1-G4 CMDP, G5-G7 IQN τ, G8-G9 inventory β, G10-G12 Kelly.
- 20/20 trade_management_kernels.rs tests pass (a7/a8 migrated).
- 5/5 controller_adaptive_floors.rs tests still pass.
- integrated_trainer_smoke passes (end-to-end pipeline launch).
- Local 1k smoke b=128: completes 1000/1000 steps, no NaN, controllers steady.
- compute-sanitizer memcheck (5 steps b=128): ERROR SUMMARY: 0 errors.
Plan: docs/superpowers/plans/2026-05-30-adaptive-risk-management-plan.md
|
||
|
jgrusewski
|
083a88f7c3 |
feat(rl): adaptive controller floors — 12 controllers, all signal-driven
Replaces hardcoded thresholds AND clamp bounds across 12 RL controllers
with observed-signal-driven ISV-slot bounds. Eliminates the architectural
failure mode that surfaced in walk-forward fold 0 (alpha-rl-m9cx5) and
fold 1 (alpha-rl-jgdh6): under Phase 4.5 advantage normalization, eight
controllers (PPO clip, target_tau, rollout_steps, entropy_coef, per_α,
gamma, reward_scale, q_distill_lambda) saturated at extrema within ~50
steps and stayed pinned for the rest of training — same pattern across
two different data slices, confirming the bug is structural rather than
data-size-dependent.
Mechanism: each controller's noise floor was hardcoded as a small
fraction of its target (`_NOISE_FLOOR_FRAC = 0.01f`), calibrated against
a pre-Phase-4.5 signal regime. Phase 4.5 normalization reduces operating
KL by ~50× — observed signal stays below the 1%-of-target floor, the
Schulman widen path fires continuously (asymmetric in the wrong
direction), and ε hits MAX 0.50 within ~50 training steps. ksll2's full-
data run (n_folds=1) happened to escape via a single above-band KL
observation that triggered tighten; both walk-forward folds (3 and 6
files) did not.
Per `feedback_adaptive_not_tuned`, `feedback_isv_for_adaptive_bounds`,
`pearl_controller_anchors_isv_driven`: every threshold now derives from
observed signal statistics (Welford online variance) rather than
constants calibrated against a prior signal regime. User explicitly
expanded scope mid-implementation: "if all clamps ISV bound should be
added to this spec and tasks" — applying the principle consistently
means hardcoded MIN/MAX clamp bounds count too, not just the saturating
noise floors. 12 controllers, single atomic commit per
`feedback_no_partial_refactor`.
Spec: docs/superpowers/specs/2026-05-30-adaptive-controller-floor-design.md
Plan: docs/superpowers/plans/2026-05-30-adaptive-controller-floor-plan.md
# New kernel
- rl_signal_variance_update.cu (80 LOC) — Welford online variance.
Single-thread single-block. Sentinel-zero skip per pearl. Per-controller
Welford triple (count, mean, M²) drives every adaptive noise floor.
# Per-controller refactors (12 .cu files)
- ppo_clip + target_tau + rollout_steps + entropy_coef + per_α —
adaptive noise floor = max(target × 0.5, sqrt(observed_var) × 2);
asymmetric Schulman (tighten on single observation, widen requires 3
consecutive below-band).
- gamma (Special G) — hardcoded GAMMA_MIN = 0.995 → adaptive via
Welford MEAN of trade duration. Per spec Q6 resolution: the Welford
mean's natural N-smoothing lag breaks the gamma↔trade_duration
feedback loop without explicit step-period gating. 100-observation
warmup falls back to EMA before Welford has enough samples.
- reward_scale (Special R) — asymmetric DECREASE rate cap (5% per
step) on both bootstrap-replace and Wiener-blend paths; bootstrap-
fraction floor (10% of bootstrap) until 100 trades close.
- q_distill_lambda (Special Q) — hardcoded MIN_LAMBDA = 0.05 → adaptive
via Welford on q_distill_kl_ema: max(0.001, std × 0.05).
- v_blend_alpha (Phase 4.4) — 5 hardcoded constants → 5 ISV slots
(DEAD_SIGNAL_FLOOR, TARGET_TRACK_RATIO, SCHULMAN_STEP, EMA_ALPHA,
BOOTSTRAP_ALPHA) + adaptive dead-signal floor from V_scalar magnitude
variance.
- ppo_ratio_clamp — adaptive MIN/MAX from observed log-ratio variance.
Architectural 2.0 absolute floor preserved (don't degenerate to
vanilla policy gradient).
- reward_clamp — V_BOUND_FLOOR, V_BOUND_EWMA_ALPHA, MIN_WIN, MIN_RATIO,
MAX_RATIO, MIN/MAX_MARGIN, MARGIN_TOLERANCE/ADJUST_RATE,
CLIP_RATE_EMA_ALPHA → 10 ISV slots.
- gate_threshold — hardcoded alpha = 0.01 → ISV slot 638.
- Clamp-bound expansion: EPS_MIN/MAX, TAU_MIN, ROLLOUT_MIN/MAX,
COEF_MIN/MAX, PER_ALPHA_MIN/MAX, GAMMA_MAX, MAX_LAMBDA, KL_TOLERANCE,
LAMBDA_RAMP_RATE, LAMBDA_DECAY_RATE → all ISV.
- WIENER_ALPHA_FLOOR shared across 9 controllers → single ISV slot 659.
# Trainer integration (integrated.rs)
- rl_signal_variance_update kernel loaded + helper method
`launch_rl_signal_variance_update`.
- Per-step Welford launches for 9 controller inputs, placed between
EMA producers and rl_fused_controllers in the per-step pipeline.
- Input-slot lookup array for rl_fused_controllers updated: rollout_steps
now consumes RL_ADV_VAR_PRE_NORM_INDEX (emitted by
rl_advantage_normalize before in-place normalize) instead of the
post-norm advantage_var_ratio (definitionally ~0 under Phase 4.5).
- ~30 new ISV bootstrap entries in with_controllers_bootstrapped.
# ISV slot allocation (isv_slots.rs)
- 72 new slots, RL_SLOTS_END 588 → 660. +288 bytes mapped-pinned.
- 9 Welford variance triples + 5 asymmetric Schulman counters +
3 new input signals (var_pre_norm, gamma_min_adaptive, reward_mag) +
v_blend (5 + 3 Welford) + ppo_ratio_clamp (1 + 3 Welford) +
reward_clamp (5) + gate_threshold (1) + q_distill (1) + 20 clamp bounds +
shared wiener floor.
# Bootstrap-clamp consistency fix (caught by Task 16 testing)
The original draft bootstrapped RL_EPS_BOOTSTRAP at 0.01 (= KL target),
but EPS_MIN was 0.05 — bootstrap value below clamp range. First post-
bootstrap step always snapped ε to MIN regardless of signal direction
("snap to MIN" behavior the unit tests surfaced). Fixed by bootstrapping
ε at MIN (0.05) so asymmetric Schulman operates from a valid state.
# Validation
- cargo build --release: clean
- cargo build --tests: clean
- 3 GPU Welford kernel tests (G1 constant→0var, G2 sequence→known var,
G3 sentinel skip): all pass
- 5 GPU adaptive-floor invariant tests (G3 PPO clip holds at bootstrap
when signal below floor, G4 tightens on single above-band, G5 widens
only after 3 consecutive below-band, G6 post-warmup uses Welford mean,
G6 pre-warmup uses EMA): all pass
- integrated_trainer_smoke (full GPU pipeline): passes
- 1k local smoke at b=128: 1000/1000 steps, completed_clean, no NaN,
controllers genuinely adapting (γ 0.90→0.974, per_α 0.40→0.52,
q_distill_λ 0.05→0.21, ε held at 0.05 = MIN per Phase 4.5 small-KL
regime — was previously stuck at MAX 0.50 throughout fold 0/1)
- compute-sanitizer memcheck b=128 5 steps: 0 errors
Cluster validation (G7-G9) submitted as follow-up runs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
dac9cfef5c |
spec: bypass cudarc for hot path — raw CUDA driver API + mega-kernel fusion
cudarc adds 18.8ms/step overhead at b=256 (GPU kernels take 0.16ms). Phase 1: raw cuLaunchKernel wrapper. Phase 2: pre-extracted raw ptrs. Phase 3: fused mega-kernels (20 launches → 3). Target: 1ms/step → 500+ sps. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
e0ea71e90e |
spec+plan: DQN concepts adoption — PopArt, spectral norm, outcome head, enrichment E1-E8
7 tasks: PopArt normalization, spectral norm+decoupling, K=3 outcome aux head, Q-bias correction, per-branch LR, curriculum weights, adversarial regime injection. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
63416f7c12 |
spec: multi-stream pipeline + per-push rewrite — target 100 sps at b=256
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
6f973b3a3f |
spec: bidirectional HER — backward peak + forward continuation
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
c2825a7928 |
spec: GPU-native Hindsight Experience Replay for wave-exit timing
Backward-looking HER: on trade close, compute peak unrealized PnL during the trade's lifetime. If peak >> actual, inject synthetic transition with peak_pnl and boosted priority. Teaches the agent optimal wave-exit timing 10× faster than sparse reward alone. Two kernels: rl_hindsight_track (per-step mid accumulation) and rl_hindsight_inject (synthetic push on done with coordination). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
3f61d18735 |
spec: GPU-resident PER + async non-blocking diag streaming
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
4bed8f2dbf |
refactor(rl): pre-allocate 56 replay-step gradient buffers for Graph C
Move all step_synthetic/dqn_replay_step alloc_zeros to persistent trainer fields (ss_* prefix). Enables CUDA Graph capture of the replay training step — all device pointers are now stable across steps. Introduces reduce_axis0_free() to resolve borrow-checker E0502 when both source (per-batch scratch) and destination (reduced grad) are self fields passed to the same function. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
3be9996689 |
spec: update CUDA graph spec with all session findings and implementation plan
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
6308be794e |
spec: CUDA Graph capture for RL step pipeline (2-3× throughput)
Captures 50+ kernel launches as 3 CUDA Graphs (pre-fill, post-fill, replay-step) for single-launch replay. Eliminates ~300μs/step of CPU launch overhead at b=16. Key challenges: device-resident step counter (no scalar arg changes in graph), lobsim fill breaks the graph (split into pre/post), ISV write ordering. Estimated 1.5-3× throughput improvement. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
924448b55e |
spec: add mandatory constraints section — GPU-only, ISV-driven, TDD
Enumerates all project rules that apply to the Q-learning improvements: CPU read-only, no atomicAdd, mapped-pinned only, ISV-driven params, first-observation bootstrap, bilateral clamp, raw reward in replay, no stubs/TODO/feature flags, GPU oracle tests, local smoke before cluster, surfer philosophy constraints. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
6192bd4eb6 |
spec: Q-learning improvements — n-step + IQN ensemble + noisy nets
Three-phase plan to push Q convergence past profitability: 1. N-step returns (n=10): 10× more direct reward signal 2. IQN complementary head alongside C51: ensemble action selection 3. Noisy linear layers: state-dependent exploration C51 stays for the confidence gate's distributional LCB. IQN adds flexible quantile estimation. Ensemble combines both for action selection. All ISV-driven. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
40855bfd62 |
docs(sp20): trader-grade trade management spec + audit infrastructure
Adds SP20 — full production trader-management system in one
greenfield commit (3-4 weeks of implementation work to follow):
* Tier 0: multi-resolution time-scaled market features (3 horizons)
* Tier 1: trade-arc awareness (4 features per batch)
* Tier 2: per-unit trail-stop (entry + trail + stop per unit)
* Tier 3: pyramiding + partial profit-taking (HalfFlat actions,
N_ACTIONS=9→11)
* Tier 4: Forward-Return-Distribution head + confidence gate +
per-batch anti-martingale sizing + position heat cap +
vol-adjusted defaults
Spec went through critical-review pass (v1→v2→v3):
* v1: 3 tiers, side-channel features, single-gate acceptance
* v2: 5 tiers added partial-flat + anti-mart + multi-res + checklist
* v3: foundational fixes for 4 CRIT + 6 SIG + 6 MIN findings
(per-unit pyramid state, encoder-input injection vs side-channel,
FRD head replaces survivor-biased checklist, override stack
ordering, per-batch anti-mart, real-time multi-res scales,
P-1 ceiling falsification gate, multi-tier acceptance)
§0 Foundational Principles (NEW, non-negotiable):
* §0.1 every numerical constant ISV-resident (no hardcoded #defines
in new kernels; structural-dim exception only)
* §0.2 every kernel/slot/head/action fully wired in same commit
* §0.3 diagnostics baked in at birth (every observable in JSONL)
* §0.4 per-phase ship-gate: all three audits must pass
Audit infrastructure shipped with the spec:
* scripts/audit-isv.sh — greps new .cu for hardcoded #defines
* scripts/audit-wiring.sh — verifies kernels/slots/heads/actions
have producer + consumer in code
* scripts/audit-diag.sh — runs local 100-step smoke, validates
manifest-listed jq paths present in JSONL
* scripts/audit-manifest/ — per-phase append manifests (kernels,
slots, heads, actions, diag-fields)
Naming discipline: audit scripts and manifest are SP-agnostic (no
`sp20-` prefix) per new pearl `feedback_no_sp_or_version_prefixes_in_file_names`
— they'll serve future SPs too. SP numbers belong only in
docs/superpowers/{specs,plans}/ filenames.
Audit scripts dogfooded — already caught two real violations on
existing code that the formal review missed:
* audit-isv: KL_EMA_ALPHA=0.05f hardcoded in rl_q_pi_distill_grad.cu
* audit-wiring: TrailTighten action (a7) has no handler in any
kernel (per pearl_dead_trail_stop_actions_a7_a8)
These violations are SP20 P5/P10 fix targets.
User decision recorded in spec §3 P-1: ceiling-falsification phase
intentionally skipped — SP20 is the architectural launchpad for
the broader trader system regardless of whether current arch could
be pushed further at 1M steps. P-1 may be revisited as standalone
work after SP20 ships.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
ed34d356a8 |
docs(per-horizon-cfc): kickoff — spec + plan + historical record
Spec: docs/superpowers/specs/2026-05-21-per-horizon-cfc-inference-design.md Plan: docs/superpowers/plans/2026-05-21-per-horizon-cfc-inference-plan.md Spec went through 2 critical-review passes (32 total findings, all resolved). Bucketing source: CfC.tau (per-channel, trained, log-uniform init at HIDDEN_DIM=128). Atomic refactor reverts MTER scaffolding. 5 ISV-driven controllers. All-on-device transition (no bulk DtoH). Single fused per-branch kernel. Compact ragged heads_w_skip. Validated via 3 Argo smokes (training stability, CRT.diag inference differentiation, fxt-backtest end-to-end). Also committing historical record: superseded MTER spec/plan, intervention A plan, ISV λ controller spec/plan, GPU log ring spec/plan. Per spec §5.3 these stay as audit trail; not in build path. Plan has 18 tasks, full TDD with bite-sized steps, ready for subagent-driven-development execution. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
faa9a73c10 |
spec(crt-train): output smoothness retraining intervention (intervention A)
Per project_crt_diag_findings.md — CRT.diag + diag.2 empirically
falsified all three hypotheses about why the model's per-event output
doesn't track horizon-specific dynamics. Labels ARE differentiated per
horizon, AUC=0.66 IS per-event measured, but per-event predictions
behave identically across all 5 horizons (2.5-event mean run length
raw, 3.3-event after aggressive Wiener-α EMA smoothing). h6000
predictions should change every thousands of events, not every 3.3.
Diagnosis: training dynamics issue. The BCE loss provides no signal
pushing toward horizon-coherent predictions. Adjacent-event labels at
horizon K share K-1/K of the forward window so SHOULD produce similar
predictions, but the loss doesn't require this.
Intervention A (this spec, minimum-scope): output smoothness
regularizer
L_smooth[h] = λ[h] × mean_t (p[h](t) - p[h](t-1))²
With horizon-weighted λ (stronger for h6000 than h30): forces h6000
predictions to change slowly while leaving h30 responsive.
Implementation surface:
- multi_horizon_heads.cu backward: extend grad_probs with smoothness
gradient
- perception.rs trainer step: prev_probs_d buffer, per-horizon (p[t]
- p[t-1])² accumulation
- heads.rs: SMOOTHNESS_LAMBDA constant per horizon
- alpha_train.rs: --smoothness-base-lambda CLI flag
Validation gate (CRT.train Gate):
MUST: per-horizon AUC ≥ baseline - 0.02 (no signal destruction)
WIN: h6000 mean_run_len ≥ 100 events; h6000/h30 ratio ≥ 10× (horizons
actually differentiated post-training)
STRETCH: CRT.1 controller smoke shows mean PnL CORRELATED with
conviction (vs anti-correlated in lnfwd)
Future work if intervention A doesn't pass WIN:
B: horizon-conditional output structure (architecture change)
C: curriculum on horizon (multi-day training)
D: different label generation (majority-vote vs single-point)
Status: Design — awaiting user review before plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
9ad76c4dfe |
spec(crt): v3 architecture reset — collapse Phase A+B into CRT.1 unified controller
Empirical trigger: Phase A as designed in v2 failed Gate 1 catastrophically on both smoke vjmwc (commit |
||
|
jgrusewski
|
8ba8755b48 |
spec(crt): v2 amendments from critical review — 12 issues + greenfields
Self-critical review of v1 (commit
|
||
|
jgrusewski
|
0f34843253 |
spec: continuous-reasoning trader architecture (4-layer rebuild)
Integrated spec for the continuous-reasoning trader rework. Replaces
the rule-based discrete policy with a 4-layer signal-driven architecture:
Layer A: continuous control loop (every event, not every stride)
Layer B: signal-driven position management (multi-horizon fusion,
continuous sizing, conviction-degradation exit, open_trade_state
expanded 24→128 bytes)
Layer C: adaptive risk envelope (ISV-derived max_lots, threshold,
target_annual_vol)
Layer D: online weight adaptation (LoRA + EWC++, offline-batch
per-session, shadow-eval gated)
Phased gates: A unlocks B; B is where alpha lives; C amplifies B;
D amplifies whatever's working. Each gate has explicit pass criteria.
Conviction definition: ISV-weighted multi-horizon agreement.
weight_h = max(pnl_ema_win - pnl_ema_loss, 0) / (var + cost^2).
Net edge x SNR per horizon, scale-normalised, cold-start uniform fallback.
Pearl conformance: 13 existing pearls referenced and respected
(ISV-driven anchors, first-observation bootstrap, Wiener-alpha floor,
blend-with-floor, z-score normalisation, one-unbounded multiplicand,
trade-level vol bootstrap, deadline cadence, single-source-of-truth,
atomic refactor, adaptive-not-tuned).
Hardcoded boundary preserved for hardware/exchange realities (latency,
cost, annualisation, instrument bounds). Everything else adaptive.
Status: design — awaiting user review before implementation plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
bf619a2e7b |
feat(ml-backtesting): max-hold + exit_px defensive + spec criteria revision
Three follow-ups from cluster smoke gp74n (trade_vol pearl validation): 1. Max-hold force-close: max_hold_ns added as per-backtest config (default 0 = disabled). Fires force-flat (3, 0) when current_ts - entry_ts >= max_hold_ns, BEFORE SL/trail check. Tested via max_hold_forces_close. Sweep YAML sets 60s cap to bound the long tail observed in gp74n (263985s pathological hold). 2. exit_px defensive sanity check: 500/1024 gp74n trades reported exit_px = ±i32::MAX/100 (float→int saturation sentinel from likely NaN segment_realized). Defensive fix in pnl_track_step: if exit_px is non-finite or diverges from entry_px by >10%, fall back to entry_px with zero realised_pnl. Root cause to be traced separately. 3. Spec §9.2 criteria revision: mean_hold<30s replaced with p95<600s + max<=max_hold_ns. The 30s threshold reflected the pre-pearl sub-cost churning bug, not a real feature criterion. p95 catches long-tail pathology while letting alpha-driven exit timing breathe. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
9b29f9fd0a |
feat(ml-backtesting): trade-vol floor replaces 2×cost literal in stop_check_isv
Replaces the Task 12 `2.0f * cost` floor (hardcoded multiplier) with trade_vol = sqrt(realised_return_var) bootstrapped from cost². Per pearl_trade_level_vol_for_stop_distance.md: microstructure ATR is the wrong time scale for trade-level stop decisions; per-horizon realised_return_var is the right one, with cost² as a structural cold- start sentinel. cost now appears exactly once — inside the sqrt as a bootstrap sentinel, never as a distance multiplier. The 2.0f literal is eliminated; controller is fully ISV-driven. var_avg accumulates realised_return_var in the same single-pass horizon loop as ema_loss/ema_win. Cold-start (var_avg=0): trade_vol = cost. Post-bootstrap: sqrt(var_avg) dominates. Test retargeted: cost_floor_prevents_sub_cost_stops → trade_vol_floor_prevents_sub_cost_stops, with boundaries straddling trade_vol=cost=0.125 instead of the prior 2*cost=0.25 (no-fire Δ=0.08, fire Δ=0.20). Spec §5, §10, §11, §12 amended. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
da6e887174 |
feat(ml-backtesting): cost-floor in stop_check_isv prevents sub-cost churning
Cluster smoke w7b4p showed 44864 closed trades in 1M events at 200ms latency — physically impossible without sub-cost churning. Root cause: sl_distance = max(pnl_ema_loss, atr) ≈ 0.05 at cold-start, but round-trip cost = 2 × 0.125 = 0.25. Every stop-out guaranteed loss > 5× distance; EMA converges sub-cost. Amendment: triple-max sl_distance = max(pnl_ema_loss, atr, 2*cost); trail_distance = max(pnl_ema_win, atr, 2*cost). cost is an ISV-discipline strategy anchor (already per-backtest from P4 — no new state slot, no tuned constants). Added cost_per_lot_per_side_per_b param to both decision kernel signatures and threaded cost_per_lot_per_side_d through both launch sites in step_decision_with_latency. Test: cost_floor_prevents_sub_cost_stops validates unrealized in (ATR, 0.25) does NOT fire SL; unrealized > 0.25 DOES fire. Spec amended (§10, §12). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
4101796ad9 |
refactor(ml-backtesting): delete StopRules + use_cold_start_stopgap atomically
- StopRules struct + sl_tp_rules field + all literals deleted; the ISV stop controller (Tasks 2-9) replaces this dead data path. - use_cold_start_stopgap propagation deleted across harness, batched_config, fxt-backtest, sweep_smoke.yaml, and 5 test files. - Q1 stopgap branch in harness.rs deleted; replaced with the original simple strategy-upload loop. - decision_floor_coldstart test retargeted: cold_start_persistent_ bullish_with_default_stops_never_closes -> ..._now_closes, asserts trades > 0 AND |pos| <= max_lots (99 trades, pos=1 on local GPU). - CBSW spec + plan prefixed with SUPERSEDED headers. Per feedback_single_source_of_truth_no_duplicates + feedback_no_partial_refactor: contract change migrates every consumer in one commit. No legacy wrappers; no version suffixes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
1ecce0d826 |
spec(ml-backtesting): critical-review pass on ISV stop controller — 9 fixes
Issues caught in self-review and fixed:
1. Encoding ambiguity: stop-fire writing (2, 0) would have silently
collapsed weak-alpha no-op semantics with force-flat. Introduced
distinct side=3 = force-flat; preserved side=2 = no-op for entry
path. Updated §3 + §5 + §7 truth table.
2. trail_hwm reset on close was implied in §4 but missing from §8's
touched-files list. Added pnl_track.cu modification (§7.1) +
added to §8 Added list.
3. __device__ helper enforcement: §3 now mandates stop_check_isv()
as a shared __device__ function called from both decision kernels,
not duplicated code.
4. Kernel arg additions enumerated explicitly in new §4.1 table —
every kernel-launch site gets named, no "thread through every kernel"
hand-wave.
5. seed_inflight_limits_batched (resting_orders.cu:439–475) named
explicitly in §1 + §7 — the actual kernel that needs the position-
target-semantics fix.
6. In-flight order summation: naive delta = target_signed - pos was
strictly worse than the original additive bug under non-zero
latency (would produce pos = pre + K·delta after fills). Fixed
§7 algorithm to compute effective_position by scanning all
active∈{1, 2} slots and summing their signed sizes. Added
position_target_not_additive_with_latency test.
7. ATR α=0.4 honest justification in §10: not strictly derived from
pearl_wiener_alpha_floor_for_nonstationary (which is about co-
adapting control loops, not passive observation); chosen as a
pragmatic project-wide constant matching isv_kelly_update_on_close.
8. CUDA Graph host-branch-free affirmation added to §3 — explicit
compatibility with P5 graph capture.
9. Cold-start symmetry caveat (§5): at n_trades_seen=0 both EMAs are
0 so sl_distance == trail_distance == atr — asymmetry only
emerges post-bootstrap. Acknowledged honestly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
11d1279990 |
spec(ml-backtesting): ISV-driven stop controller — supersedes falsified CBSW
Replaces StopRules::default()-all-zeros (the actual cause of the cluster smoke's n_trades=0, falsifying the CBSW dilution diagnosis) with an ISV-driven SL + trail-stop controller folded into the existing decision kernel(s). Per-backtest ATR EMA on mid-price + per-horizon pnl_ema_* drive distances under max(real, floor) per pearl_blend_formulas_must_have_permanent_floor. No new kernel; single source of truth in step_decision*. Folds in the position-target semantics fix (200-event local repro showed position_lots=199 from additive-vs-target order semantics) — same commit. Removes: StopRules, sl_tp_rules, Q1 stopgap field/branch, use_cold_start_stopgap propagation. Marks CBSW spec + plan SUPERSEDED. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
566e8bcb0a |
spec(ml-backtesting): CBSW review pass — 11 fixes (perf + correctness)
Critical review surfaced 11 actionable issues in the v1 spec; 10 fixed inline (#11 — legacy-comparison test — dropped per user decision since empirical legacy behavior is already known: n_trades=0). Performance: 1. expf → piecewise-linear ramp (~5× cheaper on GPU, no transcendental in the hot path). Operationally equivalent: monotonic, saturating, midpoint at K. Side-benefit: pure max-confidence at cold-start (sq=0 instead of sigmoid's 11.9% leakage). 2. Single fused per-horizon pass replaces the two-loop sketch. 3. Tier 2 scope narrowed to decision_policy_default ONLY. The bytecode VM (decision_policy_program) stays unchanged — runs only for custom strategy experiments, never in production policy. Halves Tier 2's surface area + test cost. 4. __device__ helpers cbsw_signal_quality + cbsw_weight introduced for single source of truth. Correctness bugs (silently present in v1 sketch, would have shipped): 5. strong_h and sq_min_active initializers added (were referenced before init in the per-horizon loop). 6. Attribution mask at cold-start was setting all 5 bits because every w[h] >= floor > 1e-9; trades would pollute ALL horizons' recent_sharpe. Fix: binary split — at sq_min_active < 0.5 attribute only to strong_h; at mature, attribute per weights > floor + epsilon. 7. sq_min was "min over h", which permanently locked the aggregator into cold-start mode if any horizon never gets attributed (e.g., h6000-only-trading regime). Fix: sq_min_active = min over h with n_trades_seen > 0; cold horizons don't gate maturity. 8. Opposing-horizons case now correctly fires on the strongest single horizon at cold-start (piecewise-linear sq=0 → pure max-conf), with explicit design-choice note explaining why this conservatism trade- off favors firing. Spec hygiene: 9. Rate validation committed to a measurement gate (Q2 ev/s ≥ 95% of Q1 ev/s) instead of back-of-envelope estimate. 10. Kernel ABI explicitly stated as unchanged → feedback_no_partial_ refactor compliance is trivial at the kernel boundary. 12. Tier 1's use_cold_start_stopgap field is DROPPED in Q2 atomically (not orphaned), and DoD checklist includes a grep-zero gate for feedback_no_legacy_aliases compliance. Risks updated: removed the sigmoid-narrowing risk (irrelevant now); added register-pressure risk with the rate-gate mitigation; added explicit "cold-start may lose on noisy trades" risk with empirical escalation path (bump kelly_floor 0.20 → 0.40 if Q2 smoke shows large negative PnL). Math walk-throughs rewritten for piecewise-linear (different numbers at cold-start): cold = pure max-conf, mature = pure weighted-sharpe, clean separation at sq_min_active threshold. Awaiting review of the revised spec before writing-plans dispatch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
1271d03931 |
spec(ml-backtesting): CBSW cold-start aggregator design
The post-trunk-grows threshold-tuning smoke (
|
||
|
jgrusewski
|
d646c1eb3c |
spec(ml-backtesting): apply 9 critical-review fixes to parallelism spec
Critical review surfaced 9 issues; all fixed inline:
1. §1 — Reframed "140× amortization" as "amortization on forward; sim
runs parallel". True win is on the ~2ms forward shared across 140
cells, not on sim work which scales linearly with n_backtests.
2. §2 — Made the 1h target vs firm bound explicit (≤1h target, ≤2h
firm). Acknowledges Graph capture realistic speedup is 1.2-1.5×,
not 2×.
3. §5 — Dropped atomicAdd. Plain `+=` under the single-writer-per-block
convention (existing pattern across sim kernels). No race.
4. §4 — Documented max-over-horizons threshold rationale (vs per-
horizon or aggregate-conviction). Flagged per-horizon as a follow-up
tweak if dilution pathway matters in the verdict.
5. §7 P5 + §10 risk — Captured-vs-uncaptured tolerance is 1e-5 relative,
NOT strict bit-identity. CUDA Graph capture can reorder reductions
harmlessly by 1 ULP; strict bit-identity would be a false-positive.
6. §8 — Made explicit that parallel_sim_equivalence + independence
tests are BOTH required. Equivalence alone is necessary but not
sufficient (a shadow-backtest[0] bug still passes equivalence).
7. §3.3 — Specified output schema: `cell_W{n}/sim_<variant_name>/`
with summary.json carrying a resolved `sim_config` block (verdict
emitter reads that, not the directory name).
8. §7 P4 — Enumerated apply_fill_to_pos call sites + added grep-verify
step before commit, so no fill path silently loses cost.
9. §9 — Tightened rate-validation gates with hard targets (P2 ≤90s,
P5 ≤60s) instead of generous minute envelopes. Added §9.1 stride=8
fallback as explicit Plan-B if Graph capture under-delivers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
bd35bdb6a3 |
spec(ml-backtesting): deployability-sweep parallelism + rate design
Brainstormed live with claude-opus-4-7. Targets ≤1h wall-clock per
4-quarter sweep on 1-3 L40S pods (vs ~750 GPU-hours naive). Three
multiplicative levers: per-backtest sim parameter matrix (~140×),
CUDA Graph capture (~2×), pod-level parallelism (~3×). Verified by
code-read during scoping:
- forward_only has no graph capture (X11 plan said so, X11 commit
didn't ship it)
- two host-roundtrip loops in sim.rs need GPU-ization for batching
to pay off (dispatch_latent_market_orders + close-detection)
- cost system is a literal-zero placeholder in pnl_track.cu:92
7-commit atomic ladder (P1-P7), each gated by tests + a measured rate
target. bf16 explicitly deferred to follow-up.
Awaiting review before transitioning to writing-plans for the
implementation plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
da1dd92bf8 |
spec(ml-alpha): trunk-grows refactor + deployability validation (supersedes prior)
Supersedes the 2026-05-19 deployability spec (commit
|