WIP: training quality fixes + VarStore elimination (INCOMPLETE — needs xavier_init_params_buf)
Phase 1 — Training quality (14 structural bugs):
- Reward: sqrt micro-reward scaling, remove sparse normalization, Kelly penalty,
additive OFI, magnitude-scaled capital floor penalty
- Gradient: CQL 25→10%, IQN 60→75%, MSE magnitude 4×, 3-phase counterfactual,
tau_anneal 100K→500K, epsilon floor 2%, PopArt fold reset, dd_threshold 0.5%
- Monitoring: per-branch Q-value instrumentation
- Hold: min_hold_bars 5→1
Phase 2 — VarStore elimination:
- Deleted copy_weights_from from all network types
- Deleted to_varstore from DuelingQNetwork + DistributionalDuelingQNetwork
- Deleted branching_to_varstore, 7 extract/sync VarStore functions
- Deleted update_target_networks, legacy CPU training paths
- Deleted iqn_target_network, target_network fields
- Deleted gpu_kernel_parity_test.rs (705 lines)
- Refactored GpuExperienceCollector to single flat buffer
- Added weight_sets_from_branching() zero-copy replacement
- EMA tau=1.0 init for target_params_buf
KNOWN ISSUE: params_buf is zero-initialized but from_flat_buffer() creates
weight set pointers into it BEFORE flatten_online_weights runs. The online
forward reads zeros → NaN. Needs xavier_init_params_buf() at construction.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
31c610b69a