fix: gate conviction reward scaling by readiness — was killing gradient signal

The plan head's conviction output (sigmoid, [0,1]) scales rewards.
At init, conviction ≈ 0.1 (Xavier random weights through sigmoid),
which multiplied ALL rewards by 0.1 — killing the gradient signal.
The model couldn't learn meaningful Q-values, defaulted to uniform
random action selection → 1.7M trades on 4M bars → val_Sharpe -1000.

Fix: only apply conviction scaling when readiness ≥ 0.5 (plan head
mature). Before that, conviction defaults to 1.0 (identity). This
matches the existing readiness gate on position sizing (line 1416).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-19 17:30:52 +02:00
parent ff7bbc7dd9
commit ff62c7968a

View File

@@ -1922,14 +1922,16 @@ extern "C" __global__ void experience_env_step(
reward -= opportunity_cost;
}
/* Plan conviction as reward scaling (always active, ISV-driven).
* The plan head receives ISV-gated h_s2 (temporal attention → feature gate → plan MLP).
* Its conviction output scales the reward → Q-value → C51 loss gradient path.
* No hardcoded constants — conviction IS the learned scaling factor.
* ISV modulates conviction indirectly through the attention-gated trunk. */
/* Plan conviction as reward scaling — ONLY when plan head is mature.
* Before readiness ≥ 0.5, conviction defaults to 1.0 (identity) so the
* reward signal flows undampened. Without this gate, random init conviction
* (~0.1) kills the gradient signal and prevents the model from learning. */
if (plan_params_ptr != NULL && fabsf(reward) > 1e-8f) {
float conviction = plan_params_ptr[i * 6 + 4]; /* [0, 1] */
reward *= conviction; /* low conviction → dampened signal, high → amplified */
float readiness_val = (readiness_ptr != NULL) ? readiness_ptr[0] : 0.0f;
float conviction = (readiness_val >= 0.5f)
? plan_params_ptr[i * 6 + 4] /* [0, 1] from plan head */
: 1.0f; /* identity until plan matures */
reward *= conviction;
}
out_rewards[out_off] = reward;