gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

refactor(rl): FRD horizons + range_σ are ISV-driven, not literals

Browse Source 
Closes the literal/const drift gap that F.5 introduced. Per
feedback_isv_for_adaptive_bounds + feedback_single_source_of_truth_no_duplicates:
adaptive bounds belong in ISV (or in a single canonical const that
ISV references), never duplicated as literals across modules.

Single canonical source: `crate::rl::common::FRD_HORIZON_TICKS` +
`FRD_BUCKET_RANGE_SIGMA` (already declared in F.5).

Producer-side fixes:
  * Trainer ISV bootstrap (integrated.rs): the seed values for slots
    500-503 now dereference the canonical consts instead of hardcoded
    60.0/300.0/1800.0/3.0 literals. Future tuning of the consts
    automatically propagates to both ISV seeds and loader-side
    labels — no manual sync required, no drift possible.
  * compute_frd_labels (loader.rs): takes `horizon_ticks` and
    `range_sigma` as parameters instead of reading consts directly.
    Caller (the file-load closure) sources them from the new
    MultiHorizonLoaderConfig fields.

Consumer-side fixes — 8 MultiHorizonLoaderConfig literal sites now
provide the two new fields, all defaulting to the canonical consts:
  * crates/ml-alpha/src/data/loader.rs (2 internal test-fixture sites)
  * crates/ml-alpha/tests/multi_horizon_loader.rs (2 sites)
  * crates/ml-alpha/examples/alpha_train.rs (2 sites)
  * crates/ml-alpha/examples/alpha_rl_train.rs (2 sites)
  * crates/ml-backtesting/src/harness.rs (1 site)
  * crates/ml-backtesting/tests/{trainer_parity,ring3_replay}.rs (2 sites)

The "optimal by default" property is preserved: every caller that
doesn't explicitly override gets the spec-recommended 60/300/1800
ticks + ±3σ. Callers that need to retune set the config fields, and
the trainer's ISV slots provide a runtime knob for the same numerics.

Verification (RTX 3050 Ti):
  * cargo check -p ml-alpha -p ml-backtesting --examples --tests → clean
  * cargo test --lib (6/6 unit tests for FRD label gen + loss_balance) → pass
  * frd_head 10/10 + integrated_trainer_smoke 1/1 + trade_mgmt 5/5 → pass
  * audit-rust-consts → 0 flags

The two new MultiHorizonLoaderConfig fields are required (no Default
impl) — callers MUST opt in to the FRD label-generation contract by
naming the fields. This is the same discipline applied across other
config consumers; making them Option<...> would silently default to
"no FRD labels" and break F.4's expected supervised signal.
This commit is contained in:
jgrusewski
2026-05-24 19:23:39 +02:00
parent 125c667a34
commit 7df7c81d37
8 changed files with 82 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
crates/ml-alpha/examples/alpha_rl_train.rs
View File

@@ -389,6 +389,8 @@ fn main() -> Result<()> {
inference_only: false,
outcome_label_cost: cli.outcome_label_cost,
instrument_filter: cli.instrument_mode,
frd_horizon_ticks: ml_alpha::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: ml_alpha::rl::common::FRD_BUCKET_RANGE_SIGMA,
};
let mut loader = MultiHorizonLoader::new(&loader_cfg).context("MultiHorizonLoader::new")?;
@@ -1002,6 +1004,8 @@ fn main() -> Result<()> {
inference_only: false,
outcome_label_cost: cli.outcome_label_cost,
instrument_filter: cli.instrument_mode,
frd_horizon_ticks: ml_alpha::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: ml_alpha::rl::common::FRD_BUCKET_RANGE_SIGMA,
};
let mut eval_loader = MultiHorizonLoader::new(&eval_loader_cfg)
.context("MultiHorizonLoader::new (eval)")?;

4
crates/ml-alpha/examples/alpha_train.rs
View File

@@ -506,6 +506,8 @@ fn main() -> Result<()> {
inference_only: false,
outcome_label_cost: cli.outcome_label_cost,
instrument_filter: cli.instrument_mode,
frd_horizon_ticks: ml_alpha::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: ml_alpha::rl::common::FRD_BUCKET_RANGE_SIGMA,
})
.context("train loader")?;
let mut val_loader = MultiHorizonLoader::new(&MultiHorizonLoaderConfig {
@@ -518,6 +520,8 @@ fn main() -> Result<()> {
inference_only: false,
outcome_label_cost: cli.outcome_label_cost,
instrument_filter: cli.instrument_mode,
frd_horizon_ticks: ml_alpha::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: ml_alpha::rl::common::FRD_BUCKET_RANGE_SIGMA,
})
.context("val loader")?;
tracing::info!(

62
crates/ml-alpha/src/data/loader.rs
View File

@@ -258,6 +258,28 @@ pub struct MultiHorizonLoaderConfig {
/// `FrontMonth` auto-detects the dominant id per file (handles the
/// quarterly roll); single-instrument datasets can use `All`.
pub instrument_filter: InstrumentFilter,
/// SP20 P3 F.5 — FRD head forward-return horizons in tick units.
/// Determines `compute_frd_labels`'s lookahead distances. Default
/// is `crate::rl::common::FRD_HORIZON_TICKS = [60, 300, 1800]` —
/// the spec-recommended microstructure / minute / hour timescales.
/// The trainer-side ISV slots (`RL_FRD_HORIZON_{1,2,3}_TICKS_INDEX`,
/// slots 500/501/502) seed the SAME defaults so loader + trainer
/// agree out of the box. Override here to retune both the
/// supervised labels AND the ISV seeds (caller is responsible for
/// the latter via `RL_FRD_HORIZON_*` writes if they want runtime
/// alignment).
pub frd_horizon_ticks: [usize; crate::rl::common::FRD_N_HORIZONS],
/// SP20 P3 F.5 — FRD bucket-range in σ units. The atom span is
/// `[-range_σ, +range_σ]` mapped onto `FRD_N_ATOMS=21` atoms.
/// Default = `crate::rl::common::FRD_BUCKET_RANGE_SIGMA = 3.0`
/// (matches the ISV seed at `RL_FRD_BUCKET_RANGE_SIGMA_INDEX`
/// slot 503). Per `pearl_glm_fitter_link_must_match_inference`:
/// the label generator AND the trainer-side ISV-driven runtime
/// MUST stay aligned — caller is responsible for matching ISV
/// slot 503 if overriding here.
pub frd_bucket_range_sigma: f32,
}
/// Default round-trip cost (price units) baked into D-style outcome labels
@@ -575,7 +597,11 @@ impl MultiHorizonLoader {
regime_full,
)
};
let frd_labels_full = compute_frd_labels(&snapshots);
let frd_labels_full = compute_frd_labels(
&snapshots,
cfg.frd_horizon_ticks,
cfg.frd_bucket_range_sigma,
);
Ok(Some(LoadedFile {
snapshots,
labels_full,
@@ -927,10 +953,20 @@ fn mid_price_f32(s: &Mbp10Snapshot) -> f32 {
/// Per `pearl_glm_fitter_link_must_match_inference`: the bucket
/// edges here MUST match the trainer-side FRD head's softmax + CE
/// interpretation (atom `a` = bucket center `(a / (N-1) × 2 - 1) ×
/// range_σ`). Both reference `FRD_BUCKET_RANGE_SIGMA` const so the
/// label and forward interpretations stay aligned.
fn compute_frd_labels(snapshots: &[Mbp10Snapshot]) -> [Vec<i32>; crate::rl::common::FRD_N_HORIZONS] {
use crate::rl::common::{FRD_BUCKET_RANGE_SIGMA, FRD_HORIZON_TICKS, FRD_N_ATOMS, FRD_N_HORIZONS};
/// range_σ`). The `range_sigma` argument and trainer-side ISV slot
/// 503 MUST stay aligned — caller is responsible for keeping them
/// in sync (default path: both come from `FRD_BUCKET_RANGE_SIGMA`).
///
/// Per `feedback_isv_for_adaptive_bounds`: horizons + range come in
/// as parameters (sourced from `MultiHorizonLoaderConfig`, defaults
/// from the canonical consts) — NOT hardcoded literals in the
/// function body. Single source of truth: `rl::common`.
fn compute_frd_labels(
snapshots: &[Mbp10Snapshot],
horizon_ticks: [usize; crate::rl::common::FRD_N_HORIZONS],
range_sigma: f32,
) -> [Vec<i32>; crate::rl::common::FRD_N_HORIZONS] {
use crate::rl::common::{FRD_N_ATOMS, FRD_N_HORIZONS};
let n = snapshots.len();
let mids: Vec<f32> = snapshots.iter().map(mid_price_f32).collect();
@@ -964,9 +1000,9 @@ fn compute_frd_labels(snapshots: &[Mbp10Snapshot]) -> [Vec<i32>; crate::rl::comm
let mut out: [Vec<i32>; FRD_N_HORIZONS] = Default::default();
let n_atoms_f = FRD_N_ATOMS as f32;
let half = (FRD_N_ATOMS - 1) as f32 / 2.0;
let scale = half / FRD_BUCKET_RANGE_SIGMA;
let scale = half / range_sigma.max(1e-6);
for (h_idx, &h_ticks) in FRD_HORIZON_TICKS.iter().enumerate() {
for (h_idx, &h_ticks) in horizon_ticks.iter().enumerate() {
// sqrt(h_ticks) Brownian scaling — variance of an h-step
// sum of iid increments is h × per-step variance, so std
// scales with sqrt(h).
@@ -1147,7 +1183,7 @@ mod trade_flow_tests {
#[cfg(test)]
mod frd_label_tests {
use super::*;
use crate::rl::common::{FRD_HORIZON_TICKS, FRD_N_ATOMS, FRD_N_HORIZONS};
use crate::rl::common::{FRD_BUCKET_RANGE_SIGMA, FRD_HORIZON_TICKS, FRD_N_ATOMS, FRD_N_HORIZONS};
/// Construct a synthetic snapshot stream with a deterministic
/// mid-price function. Only L1 bid/ask matter — `compute_frd_labels`
@@ -1172,7 +1208,7 @@ mod frd_label_tests {
fn frd_labels_flat_price_maps_to_mid_bucket() {
// Constant mid → all forward returns 0 → bucket = round(half) = 10.
let snaps: Vec<Mbp10Snapshot> = (0..2500).map(|_| snap_with_mid(100.0)).collect();
let labels = compute_frd_labels(&snaps);
let labels = compute_frd_labels(&snaps, FRD_HORIZON_TICKS, FRD_BUCKET_RANGE_SIGMA);
let mid_bucket = (FRD_N_ATOMS as i32 - 1) / 2;
for h in 0..FRD_N_HORIZONS {
let h_ticks = FRD_HORIZON_TICKS[h];
@@ -1205,7 +1241,7 @@ mod frd_label_tests {
// 7.75 is way > range_σ=3, so bucket clamps to FRD_N_ATOMS-1 = 20.
let snaps: Vec<Mbp10Snapshot> =
(0..2500).map(|i| snap_with_mid(100.0 + 0.01 * i as f64)).collect();
let labels = compute_frd_labels(&snaps);
let labels = compute_frd_labels(&snaps, FRD_HORIZON_TICKS, FRD_BUCKET_RANGE_SIGMA);
let max_bucket = (FRD_N_ATOMS - 1) as i32;
for h in 0..FRD_N_HORIZONS {
let h_ticks = FRD_HORIZON_TICKS[h];
@@ -1224,7 +1260,7 @@ mod frd_label_tests {
// n=10 < h_ticks[0]=60 → every label sentinel for h0; same for
// h1 (=300) and h2 (=1800).
let snaps: Vec<Mbp10Snapshot> = (0..10).map(|_| snap_with_mid(100.0)).collect();
let labels = compute_frd_labels(&snaps);
let labels = compute_frd_labels(&snaps, FRD_HORIZON_TICKS, FRD_BUCKET_RANGE_SIGMA);
for h in 0..FRD_N_HORIZONS {
for (i, v) in labels[h].iter().enumerate() {
assert_eq!(*v, -1, "sub-h_ticks input must be all -1; got {} at h={} i={}", v, h, i);
@@ -1252,6 +1288,8 @@ mod inference_mode_tests {
inference_only,
outcome_label_cost: DEFAULT_OUTCOME_LABEL_COST_ES,
instrument_filter: InstrumentFilter::All,
frd_horizon_ticks: crate::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: crate::rl::common::FRD_BUCKET_RANGE_SIGMA,
})
}
@@ -1334,6 +1372,8 @@ mod inference_mode_tests {
inference_only: false,
outcome_label_cost: DEFAULT_OUTCOME_LABEL_COST_ES,
instrument_filter: InstrumentFilter::All,
frd_horizon_ticks: crate::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: crate::rl::common::FRD_BUCKET_RANGE_SIGMA,
};
if let Ok(mut loader) = MultiHorizonLoader::new(&cfg) {
let err = loader.next_inference_input();

17
crates/ml-alpha/src/trainer/integrated.rs
View File

@@ -1467,12 +1467,21 @@ impl IntegratedTrainer {
(crate::rl::isv_slots::RL_TRAIL_K_INIT_INDEX, 2.0),
(crate::rl::isv_slots::RL_TRAIL_ADJUST_RATE_INDEX, 0.9),
// SP20 P3 FRD head seeds — λ, LR, h1-3 tick offsets, ±σ atom range.
// Per feedback_isv_for_adaptive_bounds + single-source-of-truth:
// horizon ticks + bucket-range σ derive from the canonical
// `rl::common` consts (same source the loader's
// `compute_frd_labels` reads via `MultiHorizonLoaderConfig`),
// so the two sides can't drift across literal-vs-const edits.
(crate::rl::isv_slots::RL_FRD_LAMBDA_INDEX, 0.5),
(crate::rl::isv_slots::RL_FRD_LR_INDEX, 1e-3),
(crate::rl::isv_slots::RL_FRD_HORIZON_1_TICKS_INDEX, 60.0),
(crate::rl::isv_slots::RL_FRD_HORIZON_2_TICKS_INDEX, 300.0),
(crate::rl::isv_slots::RL_FRD_HORIZON_3_TICKS_INDEX, 1800.0),
(crate::rl::isv_slots::RL_FRD_BUCKET_RANGE_SIGMA_INDEX, 3.0),
(crate::rl::isv_slots::RL_FRD_HORIZON_1_TICKS_INDEX,
crate::rl::common::FRD_HORIZON_TICKS[0] as f32),
(crate::rl::isv_slots::RL_FRD_HORIZON_2_TICKS_INDEX,
crate::rl::common::FRD_HORIZON_TICKS[1] as f32),
(crate::rl::isv_slots::RL_FRD_HORIZON_3_TICKS_INDEX,
crate::rl::common::FRD_HORIZON_TICKS[2] as f32),
(crate::rl::isv_slots::RL_FRD_BUCKET_RANGE_SIGMA_INDEX,
crate::rl::common::FRD_BUCKET_RANGE_SIGMA),
(crate::rl::isv_slots::RL_KL_TARGET_INDEX, 0.01),
(crate::rl::isv_slots::RL_IMPROVEMENT_THRESHOLD_INDEX, 0.99),
(crate::rl::isv_slots::RL_PLATEAU_PATIENCE_INDEX, 1000.0),

4
crates/ml-alpha/tests/multi_horizon_loader.rs
View File

@@ -28,6 +28,8 @@ fn cfg_from_env() -> Option<MultiHorizonLoaderConfig> {
inference_only: false,
outcome_label_cost: DEFAULT_OUTCOME_LABEL_COST_ES,
instrument_filter: InstrumentFilter::All,
frd_horizon_ticks: ml_alpha::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: ml_alpha::rl::common::FRD_BUCKET_RANGE_SIGMA,
})
}
@@ -72,6 +74,8 @@ fn loader_errors_on_empty_files() {
inference_only: false,
outcome_label_cost: DEFAULT_OUTCOME_LABEL_COST_ES,
instrument_filter: InstrumentFilter::All,
frd_horizon_ticks: ml_alpha::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: ml_alpha::rl::common::FRD_BUCKET_RANGE_SIGMA,
};
let res = MultiHorizonLoader::new(&cfg);
assert!(res.is_err(), "expected error for empty file list");

2
crates/ml-backtesting/src/harness.rs
View File

@@ -261,6 +261,8 @@ impl BacktestHarness {
// are still the legacy all-records files. A follow-up that plumbs
// a CLI flag through `BacktestHarnessConfig` can lift this.
instrument_filter: InstrumentFilter::All,
frd_horizon_ticks: ml_alpha::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: ml_alpha::rl::common::FRD_BUCKET_RANGE_SIGMA,
};
let loader = MultiHorizonLoader::new(&loader_cfg)?;

2
crates/ml-backtesting/tests/ring3_replay.rs
View File

@@ -50,6 +50,8 @@ fn try_loader() -> Option<MultiHorizonLoader> {
inference_only: true,
outcome_label_cost: ml_alpha::data::loader::DEFAULT_OUTCOME_LABEL_COST_ES,
instrument_filter: InstrumentFilter::All,
frd_horizon_ticks: ml_alpha::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: ml_alpha::rl::common::FRD_BUCKET_RANGE_SIGMA,
};
match MultiHorizonLoader::new(&cfg) {
Ok(l) => Some(l),

2
crates/ml-backtesting/tests/trainer_parity.rs
View File

@@ -37,6 +37,8 @@ fn try_loader(inference_only: bool) -> Option<MultiHorizonLoader> {
inference_only,
outcome_label_cost: ml_alpha::data::loader::DEFAULT_OUTCOME_LABEL_COST_ES,
instrument_filter: InstrumentFilter::All,
frd_horizon_ticks: ml_alpha::rl::common::FRD_HORIZON_TICKS,
frd_bucket_range_sigma: ml_alpha::rl::common::FRD_BUCKET_RANGE_SIGMA,
};
match MultiHorizonLoader::new(&cfg) {
Ok(l) => Some(l),