jgrusewski 9395075a19 feat(rl): confidence gate on ALL non-Hold + post-gate log_pi + surfer fixes ...

- Confidence gate now gates ALL non-Hold actions (was opening-only)
- Gate warmup reduced 10000 → 100 steps, threshold raised 0.01 → 0.3
- log_pi_at_action moved to AFTER gates in all paths so PPO trains
  on the post-gate action (gated-to-Hold gets log π(Hold))
- Hold prior advantage (ISV-driven, 0.1) for non-done steps
- Entropy gradient in PPO backward (fires all batch elements)
- Entropy controller: COEF_MAX=0.5, COEF_FLOOR=0.01, emergency bypass

Hold still collapses to 0% — investigating gate effectiveness.
gated_count shows low fire rate despite threshold=0.3 and near-uniform
Q distributions. The conf computation or gated action writing may
have a bug.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-27 01:30:01 +02:00

..

abs_copy.cu

feat(rl): R7a — lift remaining host work in step_with_lobsim to GPU

2026-05-23 11:30:14 +02:00

actions_to_market_targets.cu

feat(rl): per-batch outcome EMA, vol-adjusted trail, ISV-driven P_MIN

2026-05-24 21:59:34 +02:00

adamw_step.cu

perf(rl): fuse adamw_increment into adamw_step + tau into iqn_forward

2026-05-26 16:46:10 +02:00

apply_reward_scale.cu

feat(rl): adaptive RATIO + EWMA V_MIN/V_MAX + λ_distill bump

2026-05-24 14:20:05 +02:00

argmax_expected_q.cu

feat(rl): SP20 P4 — N_ACTIONS 9→11 with HalfFlat actions

2026-05-24 17:17:33 +02:00

attention_pool.cu

refactor(ml-alpha): revert axes B/C/D/E — keep only Kendall σ (axis A)

2026-05-18 18:11:05 +02:00

aux_heads.cu

feat(aux-heads): rewrite for A+B paired (BCE + conditional-Huber) (CB3+CB4)

2026-05-22 11:16:29 +02:00

aux_loss.cu

feat(aux-heads): rewrite for A+B paired (BCE + conditional-Huber) (CB3+CB4)

2026-05-22 11:16:29 +02:00

aux_trunk.cu

feat(aux-supervision): wire AuxTrunk + AuxHeads + Huber loss into PerceptionTrainer (B5)

2026-05-22 09:45:29 +02:00

aux_vec_add.cu

feat(aux-supervision): wire AuxTrunk + AuxHeads + Huber loss into PerceptionTrainer (B5)

2026-05-22 09:45:29 +02:00

bce_loss_multi_horizon.cu

refactor(per-horizon): N_HORIZONS 5→3 — horizon_lambda + bce_loss + smoothness kernels

2026-05-22 01:14:09 +02:00

bellman_target_projection.cu

perf(rl): cuBLAS SGEMM for DQN/IQN + Bellman/outcome kernel fusion

2026-05-26 17:17:56 +02:00

bucket_transition_kernels.cu

refactor(per-horizon): N_HORIZONS 5→3 — bucket-coupled CUDA kernels

2026-05-22 01:26:37 +02:00

cfc_step_per_branch.cu

refactor(per-horizon): N_HORIZONS 5→3 — bucket-coupled CUDA kernels

2026-05-22 01:26:37 +02:00

cfc_step.cu

perf(ml-alpha): NVIDIA-grade rewrite of CfC K-loop hot kernels — 2.15× faster

2026-05-18 19:07:29 +02:00

compute_advantage_return.cu

wip(rl): Hold prior + entropy controller fixes — investigating gates

2026-05-27 01:15:16 +02:00

dqn_distributional_q.cu

feat(rl): /B normalize loss reporting — batch-invariant JSONL metrics

2026-05-26 23:41:03 +02:00

dqn_target_soft_update.cu

feat(rl): R5 — wire 7 controllers per-step + target-net soft update

2026-05-23 10:19:13 +02:00

ema_update_on_done.cu

feat(rl): R3 — GPU-resident EMA + advantage/return kernels

2026-05-23 09:56:28 +02:00

ema_update_per_step.cu

feat(rl): R3 — GPU-resident EMA + advantage/return kernels

2026-05-23 09:56:28 +02:00

extract_realized_pnl_delta.cu

feat(rl): R6 — replace LobEnv with kernel-driven GPU-pure env step

2026-05-23 10:39:25 +02:00

gpu_log_helpers.cuh

feat(gpu-log): wire smoothness_lambda_controller as first ring producer

2026-05-21 01:36:18 +02:00

gpu_log_ids.h

feat(gpu-log): add gpu_log_ids.h — single source of truth for ring IDs

2026-05-21 01:04:50 +02:00

gpu_log_ring.cu

feat(gpu-log): wire smoothness_lambda_controller as first ring producer

2026-05-21 01:36:18 +02:00

gpu_sample_and_gather.cu

fix(rl): wire BCE + aux losses into GPU loader path + event-based sync

2026-05-26 22:18:30 +02:00

grad_h_accumulate.cu

feat(rl): real Q/π/V forward+backward + per-head Adam (Phase E.2)

2026-05-22 23:25:42 +02:00

grad_norm.cu

…

heads_block_diagonal_fwd.cu

refactor(per-horizon): N_HORIZONS 5→3 — bucket-coupled CUDA kernels

2026-05-22 01:26:37 +02:00

horizon_lambda.cu

refactor(per-horizon): N_HORIZONS 5→3 — horizon_lambda + bce_loss + smoothness kernels

2026-05-22 01:14:09 +02:00

layer_norm.cu

…

log_pi_at_action.cu

feat(rl): SP20 P4 — N_ACTIONS 9→11 with HalfFlat actions

2026-05-24 17:17:33 +02:00

mamba2_alpha_kernel.cu

arch(crt-a): forward_step incremental SSM state — enables event-rate trunk forward

2026-05-20 18:05:44 +02:00

multi_horizon_heads.cu

refactor(per-horizon): N_HORIZONS 5→3 — bucket-coupled CUDA kernels

2026-05-22 01:26:37 +02:00

output_smoothness.cu

refactor(per-horizon): N_HORIZONS 5→3 — bucket-coupled CUDA kernels

2026-05-22 01:26:37 +02:00

ppo_clipped_surrogate.cu

wip(rl): entropy gradient + controller fix — still collapsing

2026-05-27 01:05:42 +02:00

ppo_log_ratio_abs_max_b.cu

feat(rl): ISV-driven PPO importance-ratio clamp + log-ratio diagnostic

2026-05-23 20:55:00 +02:00

projection.cu

…

reduce_axis0.cu

perf(ml-alpha): NVIDIA-grade rewrite of CfC K-loop hot kernels — 2.15× faster

2026-05-18 19:07:29 +02:00

rl_action_kernel.cu

feat(rl): SP20 P4 — N_ACTIONS 9→11 with HalfFlat actions

2026-05-24 17:17:33 +02:00

rl_adversarial_boost.cu

feat(cuda): curriculum weights E8 + adversarial regime boost kernels

2026-05-26 12:00:21 +02:00

rl_asymmetric_trail_decay.cu

fix(rl): asymmetric trail win threshold 100% → 25% of initial_r

2026-05-25 20:59:44 +02:00

rl_atom_support_update.cu

feat(rl): adaptive C51 atom span ratchet to lift Q learning ceiling

2026-05-24 13:22:31 +02:00

rl_confidence_gate.cu

feat(rl): confidence gate on ALL non-Hold + post-gate log_pi + surfer fixes

2026-05-27 01:30:01 +02:00

rl_curriculum_weights.cu

feat(cuda): curriculum weights E8 + adversarial regime boost kernels

2026-05-26 12:00:21 +02:00

rl_encoder_context_broadcast.cu

feat(rl): encoder input expansion 40 → 56 dims (trade-context + multires)

2026-05-24 22:35:26 +02:00

rl_ensemble_action_value.cu

feat(rl): IQN loss + backward + ensemble action selection

2026-05-25 21:03:04 +02:00

rl_entropy_coef_controller.cu

wip(rl): entropy gradient + controller fix — still collapsing

2026-05-27 01:05:42 +02:00

rl_frd_fwd.cu

feat(rl): FRD head forward pass + GPU-oracle tests (F.2)

2026-05-24 18:13:35 +02:00

rl_frd_gate.cu

perf(rl): device-resident step counter + fused controllers (-9 launches)

2026-05-25 21:35:50 +02:00

rl_frd_layer1_bwd.cu

feat(rl): FRD layer-1 backward (dW1, db1, dh_t with ReLU mask) — F.3c

2026-05-24 18:40:30 +02:00

rl_frd_layer2_bwd.cu

feat(rl): FRD layer-2 backward (dW2, db2, dhidden) — F.3b

2026-05-24 18:36:29 +02:00

rl_frd_softmax_ce_grad.cu

feat(rl): FRD softmax + CE + dL/dlogits backward stage 1 (F.3a)

2026-05-24 18:31:03 +02:00

rl_fused_controllers.cu

wip(rl): entropy gradient + controller fix — still collapsing

2026-05-27 01:05:42 +02:00

rl_fused_reward_pipeline.cu

fix(rl): trail distance units mismatch — was 400× too small

2026-05-26 00:24:48 +02:00

rl_gamma_controller.cu

feat(rl): tier 2 wave-scale — γ=0.995, PER 32k, min-hold 100

2026-05-25 15:26:58 +02:00

rl_gate_threshold_controller.cu

perf(rl): device-resident step counter + fused controllers (-9 launches)

2026-05-25 21:35:50 +02:00

rl_hindsight_forward.cu

feat(cuda): bidirectional HER kernels — track, inject, forward

2026-05-26 08:41:05 +02:00

rl_hindsight_inject.cu

feat(cuda): bidirectional HER kernels — track, inject, forward

2026-05-26 08:41:05 +02:00

rl_hindsight_track.cu

feat(cuda): bidirectional HER kernels — track, inject, forward

2026-05-26 08:41:05 +02:00

rl_increment_step.cu

perf(rl): device-resident step counter + fused controllers (-9 launches)

2026-05-25 21:35:50 +02:00

rl_iqn_backward.cu

perf(rl): cuBLAS SGEMM for DQN/IQN + Bellman/outcome kernel fusion

2026-05-26 17:17:56 +02:00

rl_iqn_forward.cu

perf(rl): cuBLAS SGEMM for DQN/IQN + Bellman/outcome kernel fusion

2026-05-26 17:17:56 +02:00

rl_iqn_loss.cu

feat(rl): IQN complementary Q-head — quantile embedding + Huber loss

2026-05-25 20:35:06 +02:00

rl_isv_write.cu

fix(rl): ISV-ify ALL remaining tunable design constants (10 new slots)

2026-05-24 01:10:53 +02:00

rl_kl_approx_b.cu

feat(rl): wire remaining 3 EMA inputs (kl_pi, q_divergence, trade_duration)

2026-05-23 16:37:34 +02:00

rl_kurtosis_streaming.cu

audit: ISV-ify 10 more design constants — Schulman + bootstraps + streaming α

2026-05-24 01:47:18 +02:00

rl_l2_diff_norm.cu

feat(rl): wire remaining 3 EMA inputs (kl_pi, q_divergence, trade_duration)

2026-05-23 16:37:34 +02:00

rl_l2_norm.cu

feat(rl): signal-driven LR controller from per-head grad-norm EMAs

2026-05-23 18:11:58 +02:00

rl_lr_controller.cu

fix(rl): ISV-ify ALL remaining tunable design constants (10 new slots)

2026-05-24 01:10:53 +02:00

rl_min_hold_check.cu

fix(rl): min-hold exempts positions at heat cap level

2026-05-25 13:40:53 +02:00

rl_multires_features_update.cu

refactor(rl): pre-allocate 56 replay-step gradient buffers for Graph C

2026-05-25 23:00:40 +02:00

rl_noisy_linear_backward.cu

feat(rl): Phase 3 noisy linear layers — state-dependent exploration

2026-05-25 21:09:41 +02:00

rl_noisy_linear_forward.cu

feat(rl): Phase 3 noisy linear layers — state-dependent exploration

2026-05-25 21:09:41 +02:00

rl_outcome_bwd.cu

feat(rl): outcome head backward + diag JSONL enrichment

2026-05-26 11:47:19 +02:00

rl_outcome_ce.cu

feat(rl): P1+P2 kernels — PopArt, spectral norm, outcome head, Q-bias, per-branch LR

2026-05-26 09:36:21 +02:00

rl_outcome_fused.cu

perf(rl): cuBLAS SGEMM for DQN/IQN + Bellman/outcome kernel fusion

2026-05-26 17:17:56 +02:00

rl_outcome_fwd.cu

feat(rl): P1+P2 kernels — PopArt, spectral norm, outcome head, Q-bias, per-branch LR

2026-05-26 09:36:21 +02:00

rl_outcome_label.cu

feat(rl): P1+P2 kernels — PopArt, spectral norm, outcome head, Q-bias, per-branch LR

2026-05-26 09:36:21 +02:00

rl_per_alpha_controller.cu

audit: ISV-ify 10 more design constants — Schulman + bootstraps + streaming α

2026-05-24 01:47:18 +02:00

rl_per_branch_lr.cu

feat(rl): wire P1+P2 — PopArt, spectral norm/decouple, outcome head, Q-bias, per-branch LR

2026-05-26 09:49:57 +02:00

rl_per_push_flush.cu

feat(cuda): coalesced PER push — ring + flush kernels (Grid=B, Block=128)

2026-05-26 01:41:15 +02:00

rl_per_push_ring.cu

feat(cuda): coalesced PER push — ring + flush kernels (Grid=B, Block=128)

2026-05-26 01:41:15 +02:00

rl_per_sample.cu

feat(cuda): GPU-resident PER kernels — push, sample, update, tree_rebuild

2026-05-26 00:25:31 +02:00

rl_per_tree_rebuild.cu

feat(cuda): GPU-resident PER kernels — push, sample, update, tree_rebuild

2026-05-26 00:25:31 +02:00

rl_per_update_priority.cu

feat(cuda): GPU-resident PER kernels — push, sample, update, tree_rebuild

2026-05-26 00:25:31 +02:00

rl_pi_action_kernel.cu

feat(rl): per-batch outcome EMA, vol-adjusted trail, ISV-driven P_MIN

2026-05-24 21:59:34 +02:00

rl_popart_normalize.cu

feat(rl): P1+P2 kernels — PopArt, spectral norm, outcome head, Q-bias, per-branch LR

2026-05-26 09:36:21 +02:00

rl_popart_v_correct.cu

feat(rl): P1+P2 kernels — PopArt, spectral norm, outcome head, Q-bias, per-branch LR

2026-05-26 09:36:21 +02:00

rl_position_heat_check.cu

fix(rl): heat cap uses >= not > (pos=8 at cap=8 must trigger)

2026-05-25 11:00:56 +02:00

rl_ppo_clip_controller.cu

audit: ISV-ify 10 more design constants — Schulman + bootstraps + streaming α

2026-05-24 01:47:18 +02:00

rl_ppo_ratio_clamp_controller.cu

audit: ISV-ify 10 more design constants — Schulman + bootstraps + streaming α

2026-05-24 01:47:18 +02:00

rl_q_bias_correction.cu

feat(rl): P1+P2 kernels — PopArt, spectral norm, outcome head, Q-bias, per-branch LR

2026-05-26 09:36:21 +02:00

rl_q_distill_lambda_controller.cu

fix(rl): asymmetric Q→π distillation λ controller — ramp fast, decay slow

2026-05-25 11:54:17 +02:00

rl_q_pi_agree_b.cu

fix(rl): q_pi_agree metric → cosine similarity instead of argmax match

2026-05-25 12:04:26 +02:00

rl_q_pi_distill_grad.cu

feat(rl): done-gated π update — fixes qpa crash with sparse rewards

2026-05-27 00:28:49 +02:00

rl_recent_outcome_update.cu

feat(rl): per-batch outcome EMA, vol-adjusted trail, ISV-driven P_MIN

2026-05-24 21:59:34 +02:00

rl_reward_clamp_controller.cu

fix(rl): WIN/LOSS + C51 atom span are STRUCTURAL, not adaptive (G.2)

2026-05-24 19:56:16 +02:00

rl_reward_scale_controller.cu

fix(rl): per-step ±2% clamp on reward_scale movement

2026-05-25 13:50:25 +02:00

rl_reward_shaping.cu

feat(rl): longer horizon γ=0.99, min-hold 50, ride bonus for winners

2026-05-25 15:00:21 +02:00

rl_rollout_steps_controller.cu

audit: ISV-ify 10 more design constants — Schulman + bootstraps + streaming α

2026-05-24 01:47:18 +02:00

rl_sample_noise.cu

test(rl): GPU oracle tests for IQN + NoisyNet + PRNG advance fix

2026-05-25 22:37:40 +02:00

rl_sample_tau.cu

test(rl): GPU oracle tests for IQN + NoisyNet + PRNG advance fix

2026-05-25 22:37:40 +02:00

rl_session_risk_check.cu

feat(rl): asymmetric trail + session risk — structural P&L edge

2026-05-25 19:53:54 +02:00

rl_spectral_decouple.cu

feat(rl): P1+P2 kernels — PopArt, spectral norm, outcome head, Q-bias, per-branch LR

2026-05-26 09:36:21 +02:00

rl_spectral_norm.cu

feat(rl): P1+P2 kernels — PopArt, spectral norm, outcome head, Q-bias, per-branch LR

2026-05-26 09:36:21 +02:00

rl_step_counter_update.cu

feat(rl): wire remaining 3 EMA inputs (kl_pi, q_divergence, trade_duration)

2026-05-23 16:37:34 +02:00

rl_streaming_clamp_init.cu

fix(rl): ISV-driven K-loop divisor + max ceiling (slot 450, 451)

2026-05-24 00:23:38 +02:00

rl_target_tau_controller.cu

feat(rl): revert to V-advantage + normalization, ISV-driven TAU_MAX

2026-05-27 00:10:57 +02:00

rl_trade_context_update.cu

perf(rl): device-resident step counter + fused controllers (-9 launches)

2026-05-25 21:35:50 +02:00

rl_trail_mutate.cu

fix(rl): wire TrailTighten/TrailLoosen + SP20 P1+P5 foundation

2026-05-24 16:47:53 +02:00

rl_trail_stop_check.cu

feat(rl): pyramiding semantics — add/partial-flat/anti-martingale sizing

2026-05-24 21:15:04 +02:00

rl_unit_state_update.cu

fix(rl): trail distance units mismatch — was 400× too small

2026-05-26 00:24:48 +02:00

rl_var_over_abs_mean_streaming.cu

audit: ISV-ify 10 more design constants — Schulman + bootstraps + streaming α

2026-05-24 01:47:18 +02:00

rl_write_u64.cu

refactor(rl): pre-allocate 56 replay-step gradient buffers for Graph C

2026-05-25 23:00:40 +02:00

smoothness_lambda_controller.cu

refactor(per-horizon): N_HORIZONS 5→3 — horizon_lambda + bce_loss + smoothness kernels

2026-05-22 01:14:09 +02:00

snap_feature_assemble.cu

feat(rl): encoder input expansion 40 → 56 dims (trade-context + multires)

2026-05-24 22:35:26 +02:00

snapshot_aos_to_soa.cu

perf(rl): GPU AoS→SoA scatter replaces host-side snapshot staging

2026-05-26 19:18:26 +02:00

v_head_fwd_bwd.cu

feat(rl): V_pred structurally clamped to C51 atom span (G.1)

2026-05-24 19:47:25 +02:00

variable_selection.cu

…