gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix: Expected SARSA tau floor scales with Q magnitude — was fixed 0.01

Browse Source 
Floor = max(|mean_Q| * 0.01, 1e-6). At Q-mean=0.005: floor=5e-5.
At Q-mean=1.0: floor=0.01. Fully adaptive, zero hardcoded constants.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-14 18:10:10 +02:00
parent 69b911a611
commit b1b5682e2b
1 changed files with 5 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
crates/ml/src/cuda_pipeline/c51_loss_kernel.cu
View File

@@ -497,7 +497,11 @@ extern "C" __global__ void c51_loss_batched(
min_eq = fminf(min_eq, eq_per_action[a]);
}
float q_gap_local = max_eq - min_eq;
float tau = fmaxf(q_gap_local, 0.01f);
/* Floor: proportional to mean Q magnitude so the Boltzmann is
* meaningful at any Q-value scale. 1% of |mean Q| or 1e-6. */
float mean_q = (max_eq + min_eq) * 0.5f;
float tau_floor = fmaxf(fabsf(mean_q) * 0.01f, 1e-6f);
float tau = fmaxf(q_gap_local, tau_floor);
float sum_exp = 0.0f;
for (int a = 0; a < n_d; a++) {
action_weights[a] = expf((eq_per_action[a] - max_eq) / tau);