Files
foxhunt/crates/ml/build.rs
jgrusewski 3cb083f182 feat(dqn-v2): B.3 + C.5 GPU-only replay seed warm-start + CQL α ramp
Plan 3 Tasks 8 + 9. Single commit because Task 9 directly consumes Task 8's
seed-fraction signal; no useful intermediate state.

ISV tail-append:
- [82] SEED_STEPS_TARGET_INDEX — config replay_seed_steps (CPU constructor write)
- [83] SEED_STEPS_DONE_INDEX — GPU-incremented per collect_experiences_gpu
- [84] SEED_FRAC_EMA_INDEX — adaptive EMA of (1 - done/target)
- Fingerprint shifted [80,81] → [85,86]; ISV_TOTAL_DIM 82 → 87

GPU-only design (per user direction "fully gpu driven no cpu involvement"):
- 4 scripted policies as ONE CUDA kernel (scripted_policy_kernel.cu)
- Per-sample policy mix (40% uniform LCG / 20% momentum / 20% mean-rev /
  20% vwap-deviation) deterministic by `i % 5`
- Action source switched at launch boundary (CPU per-epoch read of ISV slot
  decides which kernel to dispatch; the action computation itself is 100% GPU)
- No CPU physics mirror — existing GPU `experience_env_step` runs unchanged

seed_step_counter_update_kernel.cu:
- Single-thread cold-path; increments DONE, computes FRAC = max(0, 1-done/target)
- Adaptive α matches Task 3/4 convention (α_base × (1 + 0.5×|clamp(sharpe,±2)|))

cql_alpha_seed_update_kernel.cu (Task 9):
- target = config.cql_alpha × max(0, 1 - seed_frac)
- During seed phase (frac=1) → target=0 → CQL α decays to 0 (no pessimism on
  exploration data); as frac → 0 → CQL α ramps to config value
- Updates ISV[CQL_ALPHA_INDEX=48]; CQL gradient kernel reads slot 48 via
  pinned device-mapped ISV (Plan 1 Task 12 consumer pattern unchanged)

Registry: SEED_STEPS_DONE + SEED_FRAC_EMA both FoldReset; CQL_ALPHA flipped
SchemaContract → FoldReset; SEED_STEPS_TARGET stays SchemaContract.

Read-only monitors (mirror PlanThresholdMonitor / StateKlMonitor pattern):
- monitors/seed_monitor.rs — surfaces ISV[82..85) for HEALTH_DIAG +
  controller_activity smoke fire-rate
- monitors/cql_alpha_monitor.rs — surfaces ISV[48] + ISV[84] dependency

Smoke (RTX 3050 Ti, 3 folds × 5 epochs, dqn-smoketest profile with
replay_seed_steps=1000 override so seed phase completes mid-fold):
- All 3 folds saved best-checkpoint
- Fold 2 best Sharpe = 92.4938 at epoch 1 (target range 80-120) ✓
- Per-fold val_metric: f0=3.80 / f1=9.73 / f2=20.24 (loss-based)
- HEALTH_DIAG[3..4] cql_alpha=0.0500 with health=0.49 → base ≈ 0.10 from ISV[48],
  consistent with kernel ramping toward final×(1-frac); regime gate
  (1-regime)×health applies on top
- 11 cargo check warnings (matches pre-task baseline; no new warnings)
- 6/6 monitor unit tests pass (read/diagnose/observe×fire_rate)

Smoke override rationale: smoke runs ~200 samples per collect (4 episodes ×
50 timesteps) × 5 epochs × 3 folds ≈ 3000 total. Default 100k target would
keep entire smoke in seed phase. Override to 1000 lets the seed→network
transition complete mid-fold so the CQL α ramp is observable.

Per pearl_one_unbounded_signal_per_reward.md: cql_alpha is bounded (clamped
to config_final × (1 - seed_frac) ∈ [0, config_final]), composes safely with
downstream CQL loss.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 09:13:39 +02:00

228 lines
8.4 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
use std::path::{Path, PathBuf};
use std::process::Command;
fn main() {
println!("cargo:rerun-if-changed=build.rs");
// Only compile CUDA kernels when the cuda feature is enabled
if std::env::var("CARGO_FEATURE_CUDA").is_err() {
return;
}
let out_dir = PathBuf::from(std::env::var("OUT_DIR").unwrap());
let kernel_dir = Path::new("src/cuda_pipeline");
let common_header = kernel_dir.join("common_device_functions.cuh");
let trade_physics_header = kernel_dir.join("trade_physics.cuh");
// Rebuild cubins when shared headers change
println!("cargo:rerun-if-changed={}", trade_physics_header.display());
// Detect GPU architecture from env or default to sm_80
let cuda_compute_cap = std::env::var("CUDA_COMPUTE_CAP").unwrap_or_else(|_| "80".to_string());
let arch = format!("sm_{cuda_compute_cap}");
// Check if nvcc is available -- gracefully skip if not (non-CUDA builds)
let nvcc_path = find_nvcc();
let nvcc = match nvcc_path {
Some(p) => p,
None => {
eprintln!(" warning: nvcc not found, skipping CUDA kernel precompilation");
eprintln!(" Install CUDA toolkit or set CUDA_HOME for GPU builds");
return;
}
};
// Read common header once
let common_src = std::fs::read_to_string(&common_header)
.unwrap_or_else(|e| panic!("Failed to read {}: {e}", common_header.display()));
println!("cargo:rerun-if-changed={}", common_header.display());
// All kernels to precompile.
// Kernels marked "standalone" have their own helpers and don't need common header.
// All others get common_device_functions.cuh prepended.
let kernels_with_common = [
// Original 24 kernels
"epsilon_greedy_kernel.cu",
"backtest_env_kernel.cu",
"backtest_forward_ppo_kernel.cu",
"backtest_forward_supervised_kernel.cu",
"backtest_metrics_kernel.cu",
"dt_kernels.cu",
"ensemble_kernels.cu",
"her_episode_kernel.cu",
"her_relabel_kernel.cu",
"signal_adapter_kernel.cu",
"statistics_kernel.cu",
"training_guard_kernel.cu",
"c51_loss_kernel.cu",
"mse_loss_kernel.cu",
"curiosity_training_kernel.cu",
"curiosity_inference_kernel.cu",
"dqn_utility_kernels.cu",
"attention_kernel.cu",
"attention_backward_kernel.cu",
"iql_value_kernel.cu",
"iqn_dual_head_kernel.cu",
"monitoring_kernel.cu",
"nstep_kernel.cu",
"reward_shaping_kernel.cu",
"ppo_experience_kernel.cu",
"bias_kernels.cu",
// Formerly standalone — now need common header for BF16 types
"experience_kernels.cu",
"ema_kernel.cu",
"relu_mask_kernel.cu",
"c51_grad_kernel.cu",
"mse_grad_kernel.cu",
"q_stats_kernel.cu",
"cql_grad_kernel.cu",
"trade_stats_kernel.cu",
"backward_kernels.cu",
"iqn_cvar_kernel.cu",
"mamba2_temporal_kernel.cu",
"graph_utility_kernels.cu",
"grad_decomp_kernel.cu",
"branch_grad_balance_kernel.cu",
"backtest_plan_kernel.cu",
"tau_update_kernel.cu",
"epsilon_update_kernel.cu",
"per_branch_gamma_update_kernel.cu",
"kelly_cap_update_kernel.cu",
"atoms_update_kernel.cu",
"q_quantile_kernel.cu",
// D.8 Plan 2 Task 6C: TLOB attention kernels (SDP + state scatter/read)
"tlob_kernel.cu",
// C.2 Plan 3 Task 1: per-component |reward| EMA into ISV[63..69)
"reward_component_ema_kernel.cu",
// B.2 Plan 3 Task 3: Flat→Positioned transition rate EMA into ISV[71]
"trade_rate_ema_kernel.cu",
// B.4 Plan 3 Task 4: per-batch readiness EMA + derived plan_threshold
// (producer for ISV[PLAN_THRESHOLD_INDEX=49] + ISV[READINESS_EMA_INDEX=75])
"plan_threshold_update_kernel.cu",
// C.3 Plan 3 Task 7: train-vs-val state-distribution KL EMA + Flat-trap
// escape amplifier (producer for ISV[78]+ISV[79]). Adaptive amp
// multiplies B.1 opp_cost and B.2 bonus consumers.
"state_kl_divergence_kernel.cu",
// B.3 Plan 3 Task 8: GPU-only seeded warm-start scripted policies.
// Replaces network-Q action source during the seed phase. 4 policies
// (uniform/momentum/mean-rev/vwap-deviation) deterministically mixed
// 40/20/20/20 by `i % 5`. Driven by ISV[SEED_STEPS_DONE/TARGET]
// dispatch at the launcher boundary.
"scripted_policy_kernel.cu",
// B.3 Plan 3 Task 8: per-collect_experiences seed-phase progress
// counter. Increments ISV[SEED_STEPS_DONE], EMAs the derived
// seed-fraction into ISV[SEED_FRAC_EMA] (consumed by Task 9).
"seed_step_counter_update_kernel.cu",
// C.5 Plan 3 Task 9: CQL α ramp coupled to ISV[SEED_FRAC_EMA].
// EMAs ISV[CQL_ALPHA_INDEX=48] toward `final × (1 - seed_frac)`.
// Producer-only upgrade for slot 48; CQL gradient kernel consumer
// (already reading ISV[48] per Plan 1 Task 12) unchanged.
"cql_alpha_seed_update_kernel.cu",
];
// ALL kernels get common header (BF16 types + wrappers)
let mut failed: Vec<&str> = Vec::new();
for kernel_name in &kernels_with_common {
if !try_compile_kernel(
&nvcc, kernel_dir, kernel_name, &arch, &out_dir, Some(&common_src),
) {
failed.push(kernel_name);
}
}
let passed = kernels_with_common.len() - failed.len();
eprintln!(" Precompiled {passed}/{} CUDA kernels ({arch}) — f32/TF32",
kernels_with_common.len());
if !failed.is_empty() {
eprintln!(" FAILED: {}", failed.join(", "));
panic!("nvcc failed to compile {} kernel(s): {}", failed.len(), failed.join(", "));
}
}
/// Compile a single .cu kernel file to a .cubin via nvcc.
///
/// If `common_header` is Some, it is prepended to the kernel source.
fn try_compile_kernel(
nvcc: &Path,
kernel_dir: &Path,
kernel_name: &str,
arch: &str,
out_dir: &Path,
common_header: Option<&str>,
) -> bool {
let kernel_path = kernel_dir.join(kernel_name);
let cubin_name = kernel_name.replace(".cu", ".cubin");
let cubin_path = out_dir.join(&cubin_name);
println!("cargo:rerun-if-changed={}", kernel_path.display());
let kernel_src = std::fs::read_to_string(&kernel_path)
.unwrap_or_else(|e| panic!("Failed to read {}: {e}", kernel_path.display()));
// Compose source: optional common header + kernel
let full_source = match common_header {
Some(header) => format!("{header}\n{kernel_src}"),
None => kernel_src,
};
// Write composed source to temp file
let tmp_src = out_dir.join(format!("_{kernel_name}"));
std::fs::write(&tmp_src, &full_source).unwrap();
// Compile with nvcc (-I kernel_dir for shared .cuh includes)
let include_flag = format!("-I{}", kernel_dir.display());
let status = Command::new(nvcc)
.args([
"-cubin",
&format!("-arch={arch}"),
"-O3",
"--ftz=true",
"--fmad=true",
"--prec-div=true",
"--prec-sqrt=true",
&include_flag,
"-o", cubin_path.to_str().unwrap(),
tmp_src.to_str().unwrap(),
])
.status();
match status {
Ok(s) if s.success() => {
eprintln!(" Compiled {kernel_name} -> {cubin_name} ({arch})");
true
}
Ok(s) => {
eprintln!(" FAILED: {kernel_name} (exit={})", s.code().unwrap_or(-1));
false
}
Err(e) => {
eprintln!(" FAILED: {kernel_name} (nvcc error: {e})");
false
}
}
}
/// Find nvcc: prefer $CUDA_HOME/bin/nvcc, then check PATH
fn find_nvcc() -> Option<PathBuf> {
// Try $CUDA_HOME/bin/nvcc first
if let Ok(home) = std::env::var("CUDA_HOME") {
let nvcc = PathBuf::from(home).join("bin/nvcc");
if nvcc.exists() {
return Some(nvcc);
}
}
// Try common CUDA paths
for path in &["/usr/local/cuda/bin/nvcc", "/usr/bin/nvcc"] {
let p = PathBuf::from(path);
if p.exists() {
return Some(p);
}
}
// Check if nvcc is in PATH
match Command::new("nvcc").arg("--version").output() {
Ok(output) if output.status.success() => Some(PathBuf::from("nvcc")),
_ => None,
}
}