gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(s&p): protect all branch heads, not just magnitude

Browse Source 
S&P on direction branch kills Q-gap conviction → forces Flat →
magnitude diversity collapses mechanically. Protect all 4 branch
heads [8..24], perturb only trunk + value head + bottleneck.

Note: smoke test (3200 bars, 10 epochs) naturally converges to Flat
regardless of S&P — insufficient data for sustained directional
conviction. H100 production run (5.7M bars) showed 7/9 diversity
sustained through all 7 epochs with positive Sharpe at epoch 6.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-09 00:16:59 +02:00
parent 0de7f69418
commit 9b86b9d385
2 changed files with 12 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
crates/ml/src/cuda_pipeline/dqn_utility_kernels.cu
View File

@@ -385,10 +385,10 @@ extern "C" __global__ void dqn_regime_scale_kernel(
* params[i] = alpha * params[i] + (1-alpha) * sigma * noise_i
*
* Params in [skip_start, skip_end) are EXCLUDED from perturbation (kept
* intact). This protects magnitude branch head weights whose learned
* diversity is destroyed by noise injection and takes many epochs to
* recover. IQN provides the primary distributional signal for magnitude,
* and S&P on those weights disrupts the Boltzmann action selection balance.
* intact). This protects ALL 4 branch head weights (direction, magnitude,
* order, urgency). Branch heads encode the learned policy — perturbing
* direction kills Q-gap conviction → forces Flat → magnitude diversity
* collapses. Only the shared trunk + value head + bottleneck are perturbed.
*
* Generates Gaussian noise via Box-Muller transform using per-element
* LCG PRNG seeded from the element index + step counter. No CPU noise

14
crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs
View File

@@ -2165,12 +2165,14 @@ impl GpuDqnTrainer {
let blocks = ((n + 255) / 256) as u32;
let seed = (self.adam_step as u32).wrapping_mul(2654435761);
// Compute magnitude branch skip range: indices [12..15] in param_sizes
// (w_b1fc, b_b1fc, w_b1out, b_b1out). Magnitude diversity is fragile —
// noise injection destroys learned preferences and takes epochs to recover.
// Protect ALL 4 branch heads from S&P: indices [8..24] in param_sizes.
// (b0=direction, b1=magnitude, b2=order, b3=urgency — each has fc+out layers).
// Branch heads encode the learned policy. Trunk + value head + bottleneck
// are perturbed for generalization — trunk feature drift is acceptable because
// branch heads adapt quickly (within 1-2 epochs on production data volumes).
let param_sizes = compute_param_sizes(&self.config);
let skip_start: i32 = param_sizes[..12].iter().sum::<usize>() as i32;
let skip_end: i32 = param_sizes[..16].iter().sum::<usize>() as i32;
let skip_start: i32 = param_sizes[..8].iter().sum::<usize>() as i32;
let skip_end: i32 = param_sizes[..24].iter().sum::<usize>() as i32;
// GPU-native: single kernel generates noise + blends in one pass on bf16 shadow.
// params_bf16[i] = alpha * params_bf16[i] + (1-alpha) * N(0, sigma)
@@ -2219,7 +2221,7 @@ impl GpuDqnTrainer {
self.graph_adam = None;
let skip_count = (skip_end - skip_start) as usize;
info!(alpha, sigma, total_params = n, skip_count, "Shrink-and-Perturb applied (GPU-native, zero CPU, magnitude branch protected)");
info!(alpha, sigma, total_params = n, skip_count, "Shrink-and-Perturb applied (GPU-native, zero CPU, all branch heads protected)");
Ok(())
}