From ff62c7968a23054ef6c0cf145abe6ca65085586f Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Sun, 19 Apr 2026 17:30:52 +0200 Subject: [PATCH] =?UTF-8?q?fix:=20gate=20conviction=20reward=20scaling=20b?= =?UTF-8?q?y=20readiness=20=E2=80=94=20was=20killing=20gradient=20signal?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The plan head's conviction output (sigmoid, [0,1]) scales rewards. At init, conviction ≈ 0.1 (Xavier random weights through sigmoid), which multiplied ALL rewards by 0.1 — killing the gradient signal. The model couldn't learn meaningful Q-values, defaulted to uniform random action selection → 1.7M trades on 4M bars → val_Sharpe -1000. Fix: only apply conviction scaling when readiness ≥ 0.5 (plan head mature). Before that, conviction defaults to 1.0 (identity). This matches the existing readiness gate on position sizing (line 1416). Co-Authored-By: Claude Opus 4.6 (1M context) --- .../ml/src/cuda_pipeline/experience_kernels.cu | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/crates/ml/src/cuda_pipeline/experience_kernels.cu b/crates/ml/src/cuda_pipeline/experience_kernels.cu index 5b1a323b3..a4d87dfcd 100644 --- a/crates/ml/src/cuda_pipeline/experience_kernels.cu +++ b/crates/ml/src/cuda_pipeline/experience_kernels.cu @@ -1922,14 +1922,16 @@ extern "C" __global__ void experience_env_step( reward -= opportunity_cost; } - /* Plan conviction as reward scaling (always active, ISV-driven). - * The plan head receives ISV-gated h_s2 (temporal attention → feature gate → plan MLP). - * Its conviction output scales the reward → Q-value → C51 loss gradient path. - * No hardcoded constants — conviction IS the learned scaling factor. - * ISV modulates conviction indirectly through the attention-gated trunk. */ + /* Plan conviction as reward scaling — ONLY when plan head is mature. + * Before readiness ≥ 0.5, conviction defaults to 1.0 (identity) so the + * reward signal flows undampened. Without this gate, random init conviction + * (~0.1) kills the gradient signal and prevents the model from learning. */ if (plan_params_ptr != NULL && fabsf(reward) > 1e-8f) { - float conviction = plan_params_ptr[i * 6 + 4]; /* [0, 1] */ - reward *= conviction; /* low conviction → dampened signal, high → amplified */ + float readiness_val = (readiness_ptr != NULL) ? readiness_ptr[0] : 0.0f; + float conviction = (readiness_val >= 0.5f) + ? plan_params_ptr[i * 6 + 4] /* [0, 1] from plan head */ + : 1.0f; /* identity until plan matures */ + reward *= conviction; } out_rewards[out_off] = reward;