gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(rl): B-6 — ISV-driven adaptive asymmetric Wiener-α (Bayesian shrinkage)

Browse Source 
B-5 (asymmetric α with static α_slow=0.001) revealed the static parameter
problem: provably bounds cascades (avg_win peak $2k vs B-4's $40k) BUT
over-conservative in train (dckcc step 800: avg_l > avg_w → Kelly says
don't trade → model can't discover edges; wr_ema crashed to 0.145).

The fundamental tension: static α_slow can't satisfy BOTH
  - Train convergence: asymmetry must FADE so model learns from real data
  - Boundary safety: asymmetry must ENGAGE at every fold to prevent cascade

B-6 RESOLVES this via Bayesian shrinkage:

  trust(n)   = min(1, cum_dones / n_full_threshold)           [Phase 1]
  stability  = exp(-CV × cv_gain)                              [Phase 2]
  trust_eff  = trust(n) × stability
  α_slow_eff = α_slow_min + (α_fast − α_slow_min) × trust_eff

Phase 1 (data-quantity): trust grows with cum_dones; reset_session_state
zeroes cum_dones → asymmetry RESUMES at every boundary. Math: at n=0
α_slow_eff = α_slow_min = 0.001 (full skepticism). At n=n_full = 30k
trades: α_slow_eff = α_fast = 0.05 (full standard Wiener).

Phase 2 (data-quality): Welford CV of reward magnitude gates trust. Stable
signal (CV→0): stability=1, trust opens normally. Volatile signal (CV high):
stability→0, asymmetry persists. cv_gain=0 disables Phase 2.

Per-EMA asymmetry direction encodes Kelly safety semantics:
  avg_win:  slow-up (skeptical of wins),    fast-down
  avg_loss: fast-up (admit losses),         slow-down (slow forget)
  wr_ema:   slow-up (skeptical of high WR), fast-down

ISV slots (all signal-derived from cum_dones + Welford):
  721 RL_EMA_ALPHA_SLOW_MIN_INDEX           = 0.001
  722 RL_EMA_TRUST_FULL_THRESHOLD_INDEX     = 30000
  723 RL_EMA_CV_GAIN_INDEX                  = 1.0

Threshold calibration (n_full=30k):
- Train: ~1000 cluster steps for trust to fully open → asymmetry active
  during cold-start (first 30 steps, cascade prevention) then fades.
- Eval: 30 dones/step × 500 eval steps = 15k dones → trust climbs to 0.5
  by eval end → partial protection throughout eval.

Composes:
- B-3 Kelly fractional-trust (Kelly SIZING gated by cum_dones)
- B-6 EMA asymmetric-α (Kelly INPUTS biased conservative by cum_dones)
Both fade as data accumulates; both reset at boundary.

Spec: docs/superpowers/specs/2026-06-01-ema-asymmetric-trust-with-cv-gain.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-06-01 01:13:27 +02:00
parent d725b77031
commit bc9eaac89d
6 changed files with 379 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
crates/ml-alpha/cuda/rl_avg_win_loss_ema_update.cu
View File

@@ -28,18 +28,20 @@
#define RL_AVG_WIN_USD_EMA_INDEX 678
#define RL_AVG_LOSS_USD_EMA_INDEX 679
#define EMA_ALPHA_FAST 0.05f
// B-5 asymmetric Wiener-α (2026-06-01). REPLACES B-4 multiplicative cap
// which did not bind over many steps (1.5^N → unbounded). Math: a per-
// step rate-cap cannot bound EMA convergence to E[X] given enough
// sustained observations. Asymmetric α gives provably biased estimates
// that are MATHEMATICALLY conservative for Kelly safety:
// avg_win: α_slow on upward moves (slow to trust large wins)
// α_fast on downward moves (normal correction)
// avg_loss: α_fast on upward moves (admit losses fast = safety)
// α_slow on downward moves (slow to forget losses)
// At α_slow=0.001: ema_N = X × (1 - 0.999^N) → after 100 steps reaches
// only 10% of true tail. zh96b avg_win peak $40k → bounded ~$4k.
#define RL_EMA_ALPHA_SLOW_INDEX 721
// B-6 ISV-driven adaptive asymmetric Wiener-α (2026-06-01 spec).
// α_slow_eff = α_slow_min + (α_fast − α_slow_min) × trust_eff
// trust_eff = trust(n) × stability(CV)
// trust(n) = min(1, cum_dones / n_full_threshold) [Phase 1]
// stability(CV) = exp(-CV × cv_gain) [Phase 2]
// Slot 721 α_slow_min, 722 n_full_threshold, 723 cv_gain.
// Welford triplet for reward_magnitude provides CV (Phase 2).
#define RL_EMA_ALPHA_SLOW_MIN_INDEX 721
#define RL_EMA_TRUST_FULL_THRESHOLD_INDEX 722
#define RL_EMA_CV_GAIN_INDEX 723
#define RL_CUMULATIVE_DONES_INDEX 660
#define RL_REWARD_MAG_VAR_COUNT_INDEX 615
#define RL_REWARD_MAG_VAR_MEAN_INDEX 616
#define RL_REWARD_MAG_VAR_M2_INDEX 617
extern "C" __global__ void rl_avg_win_loss_ema_update(
float* __restrict__ isv,
@@ -67,29 +69,45 @@ extern "C" __global__ void rl_avg_win_loss_ema_update(
}
}
// B-5 asymmetric Wiener-α (2026-06-01). Conservative for Kelly safety:
// - avg_win: slow up (skeptical of large wins), fast down (correct quickly)
// - avg_loss: fast up (admit losses promptly), slow down (slow to forget)
// Both bias Kelly's b̂ = avg_win/avg_loss DOWNWARD → smaller f_safe.
// Math proof: with α_slow=0.001, sustained observation X reaches
// ema = X × (1 - 0.999^N) → ema/X = 9.5% at N=100, 18% at N=200.
const float a_slow = isv[RL_EMA_ALPHA_SLOW_INDEX];
const float a_fast = EMA_ALPHA_FAST;
// B-6 ISV-driven adaptive asymmetric Wiener-α (Bayesian shrinkage).
// α_slow_eff blends α_slow_min → α_fast as data confidence grows.
const float a_slow_min = isv[RL_EMA_ALPHA_SLOW_MIN_INDEX];
const float a_fast = EMA_ALPHA_FAST;
const float n_trades = isv[RL_CUMULATIVE_DONES_INDEX];
const float n_full = isv[RL_EMA_TRUST_FULL_THRESHOLD_INDEX];
const float cv_gain = isv[RL_EMA_CV_GAIN_INDEX];
// Phase 1: data-quantity trust (resets at fold boundary via reset_session_state)
float trust = (n_full > 0.0f) ? fminf(1.0f, n_trades / n_full) : 1.0f;
// Phase 2: signal-volatility gain (set cv_gain=0 to disable)
if (cv_gain > 0.0f) {
const float wf_count = isv[RL_REWARD_MAG_VAR_COUNT_INDEX];
if (wf_count > 1.0f) {
const float wf_m2 = isv[RL_REWARD_MAG_VAR_M2_INDEX];
const float wf_mean = isv[RL_REWARD_MAG_VAR_MEAN_INDEX];
const float wf_var = wf_m2 / (wf_count - 1.0f);
const float cv = (wf_mean > 1e-6f) ? sqrtf(wf_var) / wf_mean : 0.0f;
const float stability = expf(-cv * cv_gain);
trust *= stability;
}
}
const float a_slow_eff = a_slow_min + (a_fast - a_slow_min) * trust;
if (n_win > 0) {
const float step_avg = sum_win / (float)n_win;
const float prev = isv[RL_AVG_WIN_USD_EMA_INDEX];
// Asymmetric: slow when admitting larger wins, fast when correcting down.
const float alpha = (step_avg > prev) ? a_slow : a_fast;
// avg_win: slow-up (skeptical of wins), fast-down (correct quickly)
const float alpha = (step_avg > prev) ? a_slow_eff : a_fast;
isv[RL_AVG_WIN_USD_EMA_INDEX] = (1.0f - alpha) * prev + alpha * step_avg;
}
if (n_loss > 0) {
const float step_avg = sum_loss / (float)n_loss;
const float prev = isv[RL_AVG_LOSS_USD_EMA_INDEX];
// Asymmetric: fast when admitting larger losses (safety),
// slow when "forgetting" (post-trauma persistence).
const float alpha = (step_avg > prev) ? a_fast : a_slow;
// avg_loss: fast-up (admit losses, safety), slow-down (slow forget)
const float alpha = (step_avg > prev) ? a_fast : a_slow_eff;
isv[RL_AVG_LOSS_USD_EMA_INDEX] = (1.0f - alpha) * prev + alpha * step_avg;
}
}

47
crates/ml-alpha/cuda/rl_win_rate_ema_update.cu
View File

@@ -29,14 +29,14 @@
#define RL_WIN_RATE_EMA_INDEX 677
#define EMA_ALPHA_FAST 0.05f
#define SENTINEL 0.5f
// B-5 asymmetric Wiener-α (2026-06-01). REPLACES B-4 additive cap which
// never engaged at α=0.05 (natural Wiener step ≤ 0.035 < cap=0.05).
// Asymmetric: α_slow on upward (slow to trust high WR optimism),
// α_fast on downward (admit pessimism quickly).
// Math: at α_slow=0.001, single 100% obs from prev=0.3 lifts ema by
// 0.001×0.7 = 0.0007. To reach 0.87 from 0.3: 0.999^N = 0.186 → N=694 steps.
// vs prior cascade reaching 0.87 in 140 steps.
#define RL_EMA_ALPHA_SLOW_INDEX 721
// B-6 ISV-driven adaptive asymmetric Wiener-α. See B-6 spec for math.
#define RL_EMA_ALPHA_SLOW_MIN_INDEX 721
#define RL_EMA_TRUST_FULL_THRESHOLD_INDEX 722
#define RL_EMA_CV_GAIN_INDEX 723
#define RL_CUMULATIVE_DONES_INDEX 660
#define RL_REWARD_MAG_VAR_COUNT_INDEX 615
#define RL_REWARD_MAG_VAR_MEAN_INDEX 616
#define RL_REWARD_MAG_VAR_M2_INDEX 617
extern "C" __global__ void rl_win_rate_ema_update(
float* __restrict__ isv,
@@ -62,14 +62,29 @@ extern "C" __global__ void rl_win_rate_ema_update(
const float step_wr = (float)wins / (float)closed;
const float prev = isv[RL_WIN_RATE_EMA_INDEX];
// B-5 asymmetric Wiener-α. No first-observation bootstrap branch
// (B-4 removed). Asymmetric admission rates: slow on upward moves
// (skeptical of high win-rate signals), fast on downward (admit
// pessimism). Conservatism for Kelly: under-estimated wr → smaller
// Kelly fraction → safer sizing.
const float a_slow = isv[RL_EMA_ALPHA_SLOW_INDEX];
const float a_fast = EMA_ALPHA_FAST;
const float alpha = (step_wr > prev) ? a_slow : a_fast;
// B-6 ISV-driven adaptive asymmetric Wiener-α.
// α_slow_eff = α_slow_min + (α_fast − α_slow_min) × trust_eff
const float a_slow_min = isv[RL_EMA_ALPHA_SLOW_MIN_INDEX];
const float a_fast = EMA_ALPHA_FAST;
const float n_trades = isv[RL_CUMULATIVE_DONES_INDEX];
const float n_full = isv[RL_EMA_TRUST_FULL_THRESHOLD_INDEX];
const float cv_gain = isv[RL_EMA_CV_GAIN_INDEX];
float trust = (n_full > 0.0f) ? fminf(1.0f, n_trades / n_full) : 1.0f;
if (cv_gain > 0.0f) {
const float wf_count = isv[RL_REWARD_MAG_VAR_COUNT_INDEX];
if (wf_count > 1.0f) {
const float wf_m2 = isv[RL_REWARD_MAG_VAR_M2_INDEX];
const float wf_mean = isv[RL_REWARD_MAG_VAR_MEAN_INDEX];
const float wf_var = wf_m2 / (wf_count - 1.0f);
const float cv = (wf_mean > 1e-6f) ? sqrtf(wf_var) / wf_mean : 0.0f;
trust *= expf(-cv * cv_gain);
}
}
const float a_slow_eff = a_slow_min + (a_fast - a_slow_min) * trust;
// wr_ema: slow-up (skeptical of high WR), fast-down (admit pessimism)
const float alpha = (step_wr > prev) ? a_slow_eff : a_fast;
float ema_new = (1.0f - alpha) * prev + alpha * step_wr;
// Hard-bound [0, 1] in case of any precision corner case.
if (ema_new < 0.0f) ema_new = 0.0f;

42
crates/ml-alpha/src/rl/isv_slots.rs
View File

@@ -1585,23 +1585,35 @@ pub const RL_POS_MAX_EMA_GROWTH_CAP_CV_GAIN_INDEX: usize = 719;
// positive position alive during warmup.
pub const RL_KELLY_BOOTSTRAP_FLOOR_INDEX: usize = 720;
// B-5 asymmetric Wiener-α for Kelly-input EMAs (2026-06-01).
// Math: single per-step rate-cap CANNOT bound EMA convergence to tail-
// dominated means — α=0.05 sustained observations admit ANY tail magnitude
// over O(50) steps. Asymmetric α gives provably biased estimates skewed
// toward safety (under-estimate b̂ in Kelly):
// avg_win: α_slow on up (slow trust optimism), α_fast on down
// avg_loss: α_fast on up (admit losses fast), α_slow on down
// wr_ema: α_slow on up (slow trust high WR), α_fast on down
// At α_slow=0.001: 100 steps of $40k tail → ema = $3800 (10% of true)
// rather than B-4's $40k peak (admit-anything via Wiener). Provably
// underestimates Kelly's b ratio → conservative sizing by construction.
// B-5 supersedes B-4's caps (which didn't engage / leaked over time).
pub const RL_EMA_ALPHA_SLOW_INDEX: usize = 721;
// B-5/B-6 asymmetric Wiener-α with adaptive trust schedule (2026-06-01).
//
// Math: Bayesian shrinkage. The "skeptical" admission rate α_slow blends
// toward the "normal" α_fast=0.05 as data confidence accumulates.
//
// trust_n = min(1, cum_dones / n_full_threshold) [Phase 1]
// stability = exp(-CV(reward_mag) × cv_gain) [Phase 2]
// trust_eff = trust_n × stability
// α_slow_eff = α_slow_min + (α_fast − α_slow_min) × trust_eff
//
// Phase 1 (slots 721, 722): data-quantity gating. cum_dones reset by
// reset_session_state → asymmetry resumes at every fold boundary.
// Phase 2 (slot 723): data-quality gating. Volatile signals (high CV)
// stay skeptical even with abundant data. cv_gain=0 disables Phase 2.
//
// Per-EMA asymmetry direction encodes Kelly safety semantics:
// avg_win: slow-up (skeptical of wins), fast-down
// avg_loss: fast-up (admit losses), slow-down (slow-forget)
// wr_ema: slow-up (skeptical of high WR), fast-down
//
// Spec: docs/superpowers/specs/2026-06-01-ema-asymmetric-trust-with-cv-gain.md
pub const RL_EMA_ALPHA_SLOW_MIN_INDEX: usize = 721;
pub const RL_EMA_TRUST_FULL_THRESHOLD_INDEX: usize = 722;
pub const RL_EMA_CV_GAIN_INDEX: usize = 723;
/// Last RL-allocated slot index (exclusive).
/// Pre-risk-stack: 662. Post-Fix-B: 685. Post-v9 (warmup boundary): 696.
/// Post-regime-observer (F1.1): 716. Post-warmed-flag addendum: 717.
/// Post-cold-start-redesign B-2: 720. Post-Kelly-trust B-3: 721.
/// Post-EMA-rate-cap-extension B-4 / B-5 asymmetric-α: 722.
pub const RL_SLOTS_END: usize = 722;
/// Post-EMA-rate-cap B-4 / B-5 asymmetric-α: 722.
/// Post-B-6 trust + CV gain: 724.
pub const RL_SLOTS_END: usize = 724;

21
crates/ml-alpha/src/trainer/integrated.rs
View File

@@ -3364,7 +3364,7 @@ impl IntegratedTrainer {
// (slot, value) pair — pure device write, no HtoD per
// `feedback_no_htod_htoh_only_mapped_pinned`.
{
let isv_constants: [(usize, f32); 209] = [
let isv_constants: [(usize, f32); 211] = [
// Static seeds for the adaptive reward-clamp controller —
// these are the initial values that
// `rl_reward_clamp_controller` will replace once it
@@ -3715,12 +3715,15 @@ impl IntegratedTrainer {
(crate::rl::isv_slots::RL_KELLY_BOOTSTRAP_FLOOR_INDEX, 0.05_f32),
(crate::rl::isv_slots::RL_AVG_WIN_USD_EMA_INDEX, 1.0_f32),
(crate::rl::isv_slots::RL_AVG_LOSS_USD_EMA_INDEX, 1.0_f32),
// B-5 asymmetric Wiener-α for Kelly inputs (replaces B-4 caps).
// α_slow=0.001 in the SAFETY direction (slow trust optimism for
// avg_win/wr; slow forget pessimism for avg_loss). Math: half-life
// 693 steps in slow direction → eval[100] reaches only 10% of true
// tail. Existing α=0.05 retained as fast direction (half-life 14).
(crate::rl::isv_slots::RL_EMA_ALPHA_SLOW_INDEX, 0.001_f32),
// B-6 ISV-driven adaptive asymmetric Wiener-α (Bayesian shrinkage).
// α_slow_min = floor admission rate (full skepticism at cold-start).
// n_full_threshold = cumulative dones for full data trust (asymmetry
// fades fully). Reset_session_state zeroes cum_dones → asymmetry
// resumes at every fold boundary. Phase 2: cv_gain modulates trust
// by signal volatility (Welford CV); 0 disables Phase 2.
(crate::rl::isv_slots::RL_EMA_ALPHA_SLOW_MIN_INDEX, 0.001_f32),
(crate::rl::isv_slots::RL_EMA_TRUST_FULL_THRESHOLD_INDEX, 30000.0_f32),
(crate::rl::isv_slots::RL_EMA_CV_GAIN_INDEX, 1.0_f32),
];
for (slot, value) in isv_constants.iter() {
let slot_i32 = *slot as i32;
@@ -9530,7 +9533,9 @@ impl IntegratedTrainer {
"pos_max_ema_growth_cap_base": isv[crate::rl::isv_slots::RL_POS_MAX_EMA_GROWTH_CAP_BASE_INDEX],
"pos_max_ema_growth_cap_cv_gain": isv[crate::rl::isv_slots::RL_POS_MAX_EMA_GROWTH_CAP_CV_GAIN_INDEX],
"kelly_bootstrap_floor": isv[crate::rl::isv_slots::RL_KELLY_BOOTSTRAP_FLOOR_INDEX],
"ema_alpha_slow": isv[crate::rl::isv_slots::RL_EMA_ALPHA_SLOW_INDEX],
"ema_alpha_slow_min": isv[crate::rl::isv_slots::RL_EMA_ALPHA_SLOW_MIN_INDEX],
"ema_trust_full_threshold": isv[crate::rl::isv_slots::RL_EMA_TRUST_FULL_THRESHOLD_INDEX],
"ema_cv_gain": isv[crate::rl::isv_slots::RL_EMA_CV_GAIN_INDEX],
},
"isv_lr": {
"bce": isv[RL_LR_BCE_INDEX],

8
crates/ml-alpha/tests/eval_diag_emission.rs
View File

@@ -212,8 +212,12 @@ fn eval_diag_jsonl_emitted_with_train_schema_parity() -> Result<()> {
// 2026-05-31 B-3: +1 leaf for kelly_bootstrap_floor (fractional-trust schedule).
// New total: 648.
// 2026-06-01 B-4: +1 leaf for wr_ema_max_delta (Hoeffding-bound additive cap).
// New total: 649.
const EXPECTED_LEAVES: usize = 649;
// New total: 649. B-5 reused slot 721 (renamed wr_ema_max_delta → ema_alpha_slow,
// diag leaf renamed; count unchanged).
// 2026-06-01 B-6: +2 leaves for ema_trust_full_threshold + ema_cv_gain
// (Bayesian shrinkage adaptive trust schedule + Welford-CV gain).
// New total: 651.
const EXPECTED_LEAVES: usize = 651;
anyhow::ensure!(
train_paths.len() == EXPECTED_LEAVES,
"train leaf count {} != expected {EXPECTED_LEAVES} (schema drifted; \

259
docs/superpowers/specs/2026-06-01-ema-asymmetric-trust-with-cv-gain.md Normal file
View File

@@ -0,0 +1,259 @@
# Asymmetric Wiener-α EMA with Trust Schedule and Welford-CV Gain (B-6)
**Date:** 2026-06-01
**Status:** Draft (post B-5 cluster diagnosis)
**Parent docs:**
- `2026-05-31-pos-max-ema-cold-start-redesign.md` (B-2)
- `2026-05-31-eval-boundary-addendum-reward-scale-and-train-spike.md` (A3)
**Pearls referenced:**
- `pearl_controller_anchors_isv_driven` — every controller anchor ISV-driven
- `feedback_adaptive_not_tuned` — signal-driven, not tuned constants
- `feedback_isv_for_adaptive_bounds` — bounds in ISV
- `pearl_first_observation_bootstrap` — the underlying anti-pattern
- `pearl_adaptive_carryover_discipline` — predictive vs normalization EMAs
- `pearl_multiplicative_controllers_need_bounded_step_and_noise_floor`
---
## 1. Motivation
The B-4 multiplicative/additive caps failed math gap (1.5^N unbounded; additive cap never engaged). B-5 introduced **asymmetric Wiener-α** with α_slow=0.001 in safety direction → provably biased estimator → Kelly under-sized by construction.
Cluster diagnosis (alpha-rl-dckcc step 800, B-5):
- avg_w max peak: **$2,037** (vs B-4's $40,911 — **20× reduction** ✓)
- wr_ema max: **0.50** (vs B-4's 0.88 — never exceeded ✓)
- BUT wr_ema crashed to 0.145 by step 800, avg_l > avg_w → Kelly says don't trade
- Model trains on near-zero positions → can't discover edges
**The fundamental tension:** static α_slow can't satisfy both:
- Train: asymmetry must FADE so model learns from real magnitudes
- Eval boundary: asymmetry must ENGAGE so no cascade
## 2. Goals
### G1 — ISV-driven adaptive α_slow
Replace static α_slow=0.001 with signal-derived value that adapts to data confidence. Per `pearl_controller_anchors_isv_driven`, every controller anchor must be ISV-driven (signal-derived, not just stored).
### G2 — Boundary safety via resettable counter
Asymmetry must resume at every eval boundary. Use `RL_CUMULATIVE_DONES_INDEX` (slot 660) which is reset by `reset_session_state` — same counter as B-3 Kelly trust schedule. Single source of truth for "data confidence".
### G3 — Train convergence
After sufficient data accumulation, asymmetry fades to standard Wiener-α=0.05. Model learns from real signal magnitudes; b̂ and wr̂ approach true values.
### G4 — Volatility-aware skepticism
Beyond cumulative count, the EFFECTIVE confidence depends on signal stability. A volatile signal (high CV) demands more skepticism even at high trade counts. Phase 2 adds Welford-CV gain modulating the trust schedule.
### G5 — Non-goals
- Don't change asymmetry DIRECTION (slow-up for wins/wr, fast-up for losses) — that encodes the loss-aversion safety semantic
- Don't replace Wiener-α with Kalman/median estimators — out of scope, bigger refactor
- Don't add additional EMAs beyond avg_win, avg_loss, wr_ema in this phase
## 3. Design
### 3.1 Phase 1 — Trust schedule on α_slow (data-confidence)
```
trust(n) = min(1, n_cumulative_dones / n_full_threshold)
α_slow_eff = α_slow_min + (α_fast − α_slow_min) × trust(n)
```
For each EMA, asymmetric direction blends α_fast and α_slow_eff:
```c
// avg_win: slow-up, fast-down (skeptical of wins)
alpha = (step_avg > prev) ? α_slow_eff : α_fast;
// avg_loss: fast-up, slow-down (admit losses, slow forget)
alpha = (step_avg > prev) ? α_fast : α_slow_eff;
// wr_ema: slow-up, fast-down (skeptical of high WR)
alpha = (step_wr > prev) ? α_slow_eff : α_fast;
```
#### Math properties
**Cold-start** (n=0): α_slow_eff = α_slow_min = 0.001 → full asymmetry → b̂ underestimated → conservative Kelly.
**Mid-training** (n=n_full/2): α_slow_eff = (α_slow_min + α_fast)/2 = 0.0255 → moderate asymmetry → b̂ slightly biased low.
**Post-warmup** (n ≥ n_full): α_slow_eff = α_fast = 0.05 → standard Wiener → unbiased.
**Eval boundary** (reset_session_state zeros cum_dones): α_slow_eff = α_slow_min → asymmetry RESUMES → conservative re-engagement.
#### Threshold calibration
`n_full_threshold = 30,000` derived from:
1. **Hoeffding for proportions**: `n = 1.844 / ε²` for confidence 95%. For wr precision ε=0.05: n=738. For ε=0.01 (very tight): n=18,440.
2. **For heavy-tailed magnitudes**: variance bounds don't apply. Need larger n for reliable mean estimation. Conservative choice: 30k.
3. **Eval phase coverage**: at b=1024 with ~30 dones/step, eval phase (500 steps) generates ~15k dones → trust climbs to 0.5 by eval end. Asymmetry remains partially active throughout eval. Good safety property.
4. **Train phase**: ~1000 training steps for trust to fully open (~30k dones / 30 dones/step). Sufficient for cascade prevention during cold-start (first 30 steps where peak occurs).
### 3.2 Phase 2 — Welford-CV gain on trust (volatility-aware)
The Phase 1 trust schedule treats all data as equally trustworthy after n_full. But noisy signals demand more skepticism. Phase 2 modulates trust by the signal's Coefficient of Variation:
```
CV(signal) = σ_welford / μ_welford (from existing Welford triplet)
stability = exp(-CV × γ_cv) // bounded [0, 1]
trust_eff = trust(n) × stability
α_slow_eff = α_slow_min + (α_fast − α_slow_min) × trust_eff
```
Where:
- `γ_cv` (ISV slot 723, default 1.0): sensitivity to CV. Higher = more skeptical of volatile signals.
- CV=0 (perfectly stable): stability=1, trust_eff = trust(n)
- CV=1 (σ=μ, classical "noisy"): stability=exp(-1)=0.37, trust gated to 37% of n-based value
- CV=2 (σ=2μ, highly volatile): stability=0.14, heavily skeptical
#### Math properties (Phase 2)
**Stable regime** (low CV): trust opens normally with data accumulation.
**Volatile regime** (high CV): trust capped low even with abundant data → α_slow_eff stays near α_slow_min → continued conservatism in volatile markets. Matches surfer-trading philosophy: be skeptical when the wave isn't clean.
**Adaptive across regimes**: as market changes volatility, asymmetry naturally tightens/loosens. No manual retuning needed.
### 3.3 Welford signals available
The signal_variance_update kernel maintains Welford triplets for:
- `RL_REWARD_MAGNITUDE_EMA_INDEX` (slot 614) — used by reward_scale controller
- `RL_PPO_RATIO_VAR_M2_INDEX` (slot ~422)
- `RL_ADV_VAR_M2_INDEX`, `RL_ENTROPY_OBS_VAR_M2_INDEX`, etc.
For our purposes we need Welford on:
- `RL_AVG_WIN_USD_EMA_INDEX` (slot 678) — per-step avg_win signal
- `RL_AVG_LOSS_USD_EMA_INDEX` (slot 679) — per-step avg_loss signal
- `RL_WIN_RATE_EMA_INDEX` (slot 677) — per-step wr signal
Two options:
- **A**: Extend signal_variance_update to also track avg_win/loss/wr triplets (new Welford slots)
- **B**: Approximate CV using existing reward_magnitude Welford (slot 614) — same underlying volatility
Option B is simpler and avoids slot proliferation. The reward magnitude EMA captures overall signal volatility, which is correlated with the per-EMA volatility we'd otherwise track separately.
**Decision**: Phase 2 uses Option B (existing Welford). If empirically inadequate, Phase 2.5 adds per-EMA Welford.
## 4. ISV slot allocation
| Slot | Name | Default | Description |
|---|---|---:|---|
| 721 | `RL_EMA_ALPHA_SLOW_MIN_INDEX` | 0.001 | Minimum α for asymmetric admission (= full skepticism) |
| 722 | `RL_EMA_TRUST_FULL_THRESHOLD_INDEX` | 30,000 | Cumulative dones for trust=1.0 (asymmetry fades) |
| 723 | `RL_EMA_CV_GAIN_INDEX` | 1.0 | Sensitivity to signal CV for Phase 2 gating. Set to 0.0 to disable Phase 2 (pure Phase 1) |
All ISV-CONFIGURABLE constants. The α_slow_eff itself is signal-DRIVEN (computed at runtime from cum_dones + Welford).
## 5. Implementation
### 5.1 Kernel updates
**`rl_avg_win_loss_ema_update.cu`** + **`rl_win_rate_ema_update.cu`**:
```c
const float a_slow_min = isv[RL_EMA_ALPHA_SLOW_MIN_INDEX];
const float a_fast = EMA_ALPHA_FAST; // 0.05
const float n_trades = isv[RL_CUMULATIVE_DONES_INDEX];
const float n_full = isv[RL_EMA_TRUST_FULL_THRESHOLD_INDEX];
const float cv_gain = isv[RL_EMA_CV_GAIN_INDEX];
// Phase 1: trust schedule
float trust = (n_full > 0.0f) ? fminf(1.0f, n_trades / n_full) : 1.0f;
// Phase 2: CV gain (only if cv_gain > 0)
if (cv_gain > 0.0f) {
const float wf_count = isv[RL_REWARD_MAG_VAR_COUNT_INDEX];
if (wf_count > 1.0f) {
const float wf_m2 = isv[RL_REWARD_MAG_VAR_M2_INDEX];
const float wf_mean = isv[RL_REWARD_MAG_VAR_MEAN_INDEX];
const float wf_var = wf_m2 / (wf_count - 1.0f);
const float cv = (wf_mean > 1e-6f) ? sqrtf(wf_var) / wf_mean : 0.0f;
const float stability = expf(-cv * cv_gain);
trust *= stability;
}
}
const float a_slow_eff = a_slow_min + (a_fast - a_slow_min) * trust;
// Asymmetric direction (per-EMA semantics)
// avg_win: slow-up, fast-down
// avg_loss: fast-up, slow-down (mirror)
// wr_ema: slow-up, fast-down
const float alpha = (X > prev) ? a_slow_eff : a_fast; // avg_win, wr_ema
// OR for avg_loss:
const float alpha = (X > prev) ? a_fast : a_slow_eff;
```
### 5.2 Bootstrap entries (`with_controllers_bootstrapped`)
```rust
(RL_EMA_ALPHA_SLOW_MIN_INDEX, 0.001_f32), // was just _SLOW_, renamed
(RL_EMA_TRUST_FULL_THRESHOLD_INDEX, 30000.0_f32),
(RL_EMA_CV_GAIN_INDEX, 1.0_f32), // Phase 2 enabled by default
```
### 5.3 Diag emission
Add to `build_diag_value`:
```rust
"ema_alpha_slow_min": isv[RL_EMA_ALPHA_SLOW_MIN_INDEX],
"ema_trust_full_threshold": isv[RL_EMA_TRUST_FULL_THRESHOLD_INDEX],
"ema_cv_gain": isv[RL_EMA_CV_GAIN_INDEX],
```
### 5.4 Test schema update
`tests/eval_diag_emission.rs`: EXPECTED_LEAVES 649 → 651 (+2 new leaves; one slot renamed).
## 6. Validation gates
### V1 — Math invariants
- At step 1 (cold-start, cum_dones=0): observed alpha = α_slow_min when admission direction is safety, α_fast otherwise.
- At step 1000 (cum_dones ≫ n_full): observed alpha → α_fast in both directions.
- At eval[1] (post-reset_session_state): observed alpha = α_slow_min again.
### V2 — Cascade bounds
- avg_win peak ≤ $5k (vs B-4 $40k, B-5 $2k baseline)
- wr_ema max ≤ 0.50 in first 100 steps; can rise above as trust ramps in train
- avg_loss > avg_win NOT permanent (should normalize as data accumulates)
### V3 — Train convergence
- By step 1000+: wr_ema ≈ true wr (0.30-0.40 range typical for this codebase)
- avg_win > avg_loss for profitable runs (model finds edges)
- Kelly active sizing (not constantly at f_floor)
### V4 — Cluster eval pnl
- vs B-3 (x56wn): -$118M
- vs B-4 (zh96b, killed): partial data showed similar peaks
- vs B-5 (dckcc): pending (likely terrible due to over-conservatism)
- Target: ≤ -$30M (10× improvement)
## 7. Risks
- **Risk α (low)**: n_full_threshold = 30,000 too high → asymmetry persists into train, hampers learning. Mitigation: ISV-tunable, can lower to 5,000 if needed.
- **Risk β (medium)**: Welford CV from reward_magnitude (Option B) inadequate proxy for per-EMA volatility. Mitigation: Phase 2.5 adds per-EMA Welford if validation fails.
- **Risk γ (low)**: CV gain γ_cv=1.0 too aggressive → trust permanently low in volatile markets. Mitigation: tunable, can disable Phase 2 by setting γ_cv=0.
- **Risk δ (low)**: Resetting cum_dones at boundary disrupts both Kelly trust AND EMA asymmetry → double safety effect at boundary might over-suppress. Mitigation: empirical — eval result reveals if too conservative; can split into separate counters if needed.
## 8. Generalization
The pattern is:
> **Every adaptive EMA whose value gates downstream behavior should use asymmetric Wiener-α with the SAFETY direction admitting at a rate that depends on (a) cumulative data confidence and (b) signal volatility. The asymmetry direction encodes the controller's risk preference; the magnitude adapts to data.**
This is the same statistical principle as **Bayesian shrinkage**: estimates are pulled toward a conservative prior in the absence of strong evidence, and toward empirical values as evidence accumulates. Welford CV provides the noise floor that gates the shrinkage rate.
Future controllers using this template:
- popart envelope (slot 714) — currently uses fast-up slow-decay; could be ISV-driven too
- inventory variance EMA — currently has dead-signal hold; could integrate this template
- regime_observer EMAs — currently varied semantics; unify under this framework
## 9. Done means
- 3 ISV slots allocated (or 2 if reusing 721)
- Both EMA kernels updated with trust-gated asymmetric α
- Diag emission for the new state
- Local smoke shows cold-start asymmetry → mid-train fading → eval boundary re-engagement
- Cluster validation V1-V4 pass
- Pearl update: refine `pearl_first_observation_bootstrap` to include trust-gated asymmetric α as the canonical fix pattern