fix(crt-train): sqrt K-ratio in λ controller (tames h6000 bombardment)
Local trace at base_lambda=0.1 showed the linear ratio
{1, 0.3, 0.1, 0.03, 0.005} = HORIZONS[0]/HORIZONS[h] gave h6000 a
200x stronger target-undershoot signal than h30. The controller
slammed h6000 with smoothness gradient (peak λ[h6000]≈60) while h30
saw nearly none. h6000 val_auc collapsed to 0.514 (near-random)
while h100/h300 improved.
Replace with sqrt(HORIZONS[0]/HORIZONS[h]) = {1, 0.548, 0.316, 0.173,
0.0707}. Caps the differential at ~14x. Verified locally at base=0.1:
- h6000 val_auc recovered to 0.599 (vs 0.514 collapse, vs 0.621 no-smooth)
- jitter ratio h6000/h30 = 0.51 (meaningful differentiation, was 0.84 unforced)
- λ[h6000] equilibrium = 0.7-3 (was 4-60 with linear ratio at same base)
The design target (h6000 jitter = 7% of h30) is less aggressive than
linear (was 0.5%) but produces a tractable training equilibrium that
preserves h6000 predictive capacity. Both 500-step local AND the full
40k-step L40S retrain will tell us where it lands at scale.
This commit is contained in:
@@ -4,7 +4,7 @@
|
|||||||
// Reads `raw_per_h[5]` (emitted by output_smoothness_loss_and_grad),
|
// Reads `raw_per_h[5]` (emitted by output_smoothness_loss_and_grad),
|
||||||
// maintains a per-horizon Wiener-α-floor EMA of observed jitter,
|
// maintains a per-horizon Wiener-α-floor EMA of observed jitter,
|
||||||
// derives per-horizon target by anchoring on observed h30 jitter
|
// derives per-horizon target by anchoring on observed h30 jitter
|
||||||
// scaled by HORIZONS[0]/HORIZONS[h], and emits next-step λ[h] with a
|
// scaled by sqrt(HORIZONS[0]/HORIZONS[h]), and emits next-step λ[h] with a
|
||||||
// permanent floor.
|
// permanent floor.
|
||||||
//
|
//
|
||||||
// Per `pearl_controller_anchors_isv_driven`: target is signal-derived,
|
// Per `pearl_controller_anchors_isv_driven`: target is signal-derived,
|
||||||
@@ -33,14 +33,21 @@
|
|||||||
#define SLC_ALPHA_FLOOR 0.5f
|
#define SLC_ALPHA_FLOOR 0.5f
|
||||||
|
|
||||||
// HORIZONS = {30, 100, 300, 1000, 6000}.
|
// HORIZONS = {30, 100, 300, 1000, 6000}.
|
||||||
// Ratios HORIZONS[0]/HORIZONS[h] = {1, 0.3, 0.1, 0.03, 0.005}.
|
// Target ratio sqrt(HORIZONS[0]/HORIZONS[h]) = {1, 0.5477, 0.3162, 0.1732, 0.0707}.
|
||||||
// Constant array known at compile time.
|
//
|
||||||
|
// Rationale (2026-05-21 local-smoke trace finding): the linear ratio
|
||||||
|
// {1, 0.3, 0.1, 0.03, 0.005} gave h6000 a 200x stronger target-undershoot
|
||||||
|
// signal than h30, causing the controller to bombard h6000 with smoothness
|
||||||
|
// gradient while h30 saw little. At base_lambda=0.1 local smoke this
|
||||||
|
// collapsed h6000 val_auc to 0.514 (near-random) while middle horizons
|
||||||
|
// improved. Square-root scaling caps the differential at ~14x, preserving
|
||||||
|
// h6000 predictive capacity while still pushing toward slower change.
|
||||||
__device__ __constant__ float TARGET_K_RATIO[SLC_N_HORIZONS] = {
|
__device__ __constant__ float TARGET_K_RATIO[SLC_N_HORIZONS] = {
|
||||||
1.0f,
|
1.0f, // sqrt(30/30) = 1.0
|
||||||
30.0f / 100.0f,
|
0.5477226f, // sqrt(30/100)
|
||||||
30.0f / 300.0f,
|
0.3162278f, // sqrt(30/300)
|
||||||
30.0f / 1000.0f,
|
0.1732051f, // sqrt(30/1000)
|
||||||
30.0f / 6000.0f,
|
0.0707107f, // sqrt(30/6000)
|
||||||
};
|
};
|
||||||
|
|
||||||
extern "C" __global__ void smoothness_lambda_controller(
|
extern "C" __global__ void smoothness_lambda_controller(
|
||||||
|
|||||||
Reference in New Issue
Block a user