merge(sp15): bring phase2a (LobBar + behavioral test scaffold + 17 tests) into phase1
Brings in Phase 2A.1 (LobBar canonical ABI + 4 synthetic market generators), Phase 2A.2 (oracle + harness + pre-commit hook), and Phase 2B (17 #[ignore] behavioral test contracts) so the phase1 honest-numbers branch has access to the LobBar (price, half_spread, ofi) ABI needed for Phase 1.2.b cost-net sharpe consumer wiring. Path 2 of the BLOCKED 1.2.b investigation: the cost-net kernel needs GPU-resident streams (half_spread, ofi, rt_ind, side_ind, position) that do not exist on phase1; Phase 2A.1's LobBar provides the canonical ABI. Conflict resolution: docs/dqn-wire-up-audit.md — both branches prepended entries; merged by keeping all three (Phase 2A.1 from phase2a, Phase 1.6, Phase 1.7 from phase1). Phase 2A.1 entry placed above Phase 1.6 / 1.7 to keep this region's audit ordering consistent (newer-first locally). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
21
.claude/helpers/pre_commit_behavioral_suite.sh
Executable file
21
.claude/helpers/pre_commit_behavioral_suite.sh
Executable file
@@ -0,0 +1,21 @@
|
||||
#!/bin/bash
|
||||
# .claude/helpers/pre_commit_behavioral_suite.sh
|
||||
#
|
||||
# SP15 Phase 2 — behavioral suite gate. Blocks argo-train.sh if any
|
||||
# behavioral test fails. Suite runs <10 min on dev RTX 3050 Ti.
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(git rev-parse --show-toplevel)"
|
||||
|
||||
echo "Running SP15 behavioral suite (gate for argo-train.sh)..."
|
||||
|
||||
if ! SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 \
|
||||
cargo test -p ml --test behavioral_suite --features cuda \
|
||||
-- --include-ignored --test-threads=1 2>&1 \
|
||||
| tee /tmp/sp15_behavioral_suite.log; then
|
||||
echo "Behavioral suite FAILED. argo-train.sh blocked."
|
||||
echo " See /tmp/sp15_behavioral_suite.log for details."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Behavioral suite PASS — argo-train.sh unblocked."
|
||||
@@ -1,5 +1,17 @@
|
||||
{
|
||||
"hooks": {
|
||||
"PreToolUse": [
|
||||
{
|
||||
"matcher": "Bash",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "if echo \"${CLAUDE_TOOL_INPUT:-}\" | grep -qE 'argo-train\\.sh'; then \"$CLAUDE_PROJECT_DIR/.claude/helpers/pre_commit_behavioral_suite.sh\"; fi",
|
||||
"timeout": 600
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"SessionStart": [
|
||||
{
|
||||
"hooks": [
|
||||
|
||||
@@ -272,6 +272,10 @@ path = "examples/hyperopt_baseline_rl.rs"
|
||||
name = "hyperopt_baseline_supervised"
|
||||
path = "examples/hyperopt_baseline_supervised.rs"
|
||||
|
||||
[[test]]
|
||||
name = "behavioral_suite"
|
||||
path = "tests/behavioral/main.rs"
|
||||
|
||||
[[bench]]
|
||||
name = "microstructure_bench"
|
||||
harness = false
|
||||
|
||||
33
crates/ml/src/cuda_pipeline/lob_bar.rs
Normal file
33
crates/ml/src/cuda_pipeline/lob_bar.rs
Normal file
@@ -0,0 +1,33 @@
|
||||
//! Canonical ABI for synthetic and real LOB data.
|
||||
//! Phase 2A defines this; Phase 1.2 cost kernel and main eval consume it.
|
||||
//! Per spec §4.4.
|
||||
|
||||
#[repr(C)]
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct LobBar {
|
||||
pub price: f32, // mid-price, MES futures dollar units
|
||||
pub spread: f32, // bid-ask spread, same units as price (e.g., 0.25 = quarter-tick)
|
||||
pub ofi: f32, // signed OFI normalized to ~[-3, 3]
|
||||
}
|
||||
|
||||
impl LobBar {
|
||||
pub const fn new(price: f32, spread: f32, ofi: f32) -> Self {
|
||||
Self { price, spread, ofi }
|
||||
}
|
||||
}
|
||||
|
||||
/// Convert AoS Vec<LobBar> → SoA (prices, spreads, ofis).
|
||||
/// Cost-net kernel and synthetic markets both flatten on the GPU side;
|
||||
/// this helper centralises the conversion.
|
||||
pub fn into_soa(bars: &[LobBar]) -> (Vec<f32>, Vec<f32>, Vec<f32>) {
|
||||
let n = bars.len();
|
||||
let mut prices = Vec::with_capacity(n);
|
||||
let mut spreads = Vec::with_capacity(n);
|
||||
let mut ofis = Vec::with_capacity(n);
|
||||
for b in bars {
|
||||
prices.push(b.price);
|
||||
spreads.push(b.spread);
|
||||
ofis.push(b.ofi);
|
||||
}
|
||||
(prices, spreads, ofis)
|
||||
}
|
||||
@@ -64,6 +64,7 @@ pub mod sp11_isv_slots;
|
||||
pub mod sp13_isv_slots;
|
||||
pub mod sp14_isv_slots;
|
||||
pub mod sp15_isv_slots;
|
||||
pub mod lob_bar;
|
||||
pub use sp4_isv_slots::{
|
||||
TARGET_Q_BOUND_INDEX, ATOM_POS_BOUND_BASE, WEIGHT_BOUND_BASE,
|
||||
ADAM_M_BOUND_BASE, ADAM_V_BOUND_BASE, WD_RATE_BASE,
|
||||
|
||||
53
crates/ml/tests/behavioral/harness.rs
Normal file
53
crates/ml/tests/behavioral/harness.rs
Normal file
@@ -0,0 +1,53 @@
|
||||
//! SP15 Phase 2A — behavioral evaluator harness.
|
||||
//!
|
||||
//! Runs a policy on a synthetic market, captures action distribution +
|
||||
//! sharpe_net + max_dd. Trainer-side methods (eval_actions_on_features etc)
|
||||
//! are not yet implemented — see plan IMPORTANT 7. Each Phase 2B test
|
||||
//! adds the specific trainer methods it needs.
|
||||
|
||||
use ml::cuda_pipeline::lob_bar::LobBar;
|
||||
use std::collections::HashMap;
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
pub struct BehavioralResult {
|
||||
/// Action distribution keys: "hold", "flat", "long_quarter",
|
||||
/// "long_half", "long_full", "short_quarter", "short_half", "short_full".
|
||||
pub action_dist: HashMap<&'static str, f32>,
|
||||
pub sharpe_net: f32,
|
||||
pub max_dd: f32,
|
||||
pub trade_count: usize,
|
||||
}
|
||||
|
||||
/// Runs the policy on `bars`, captures action distribution + sharpe_net
|
||||
/// + max_dd. `n_warm` warmup bars precede action collection.
|
||||
///
|
||||
/// Phase 2A contract (locked here): trainer-side `eval_actions_on_features`
|
||||
/// is added per-test in Phase 2B as each behavioral test needs it (plan
|
||||
/// execution-time gap #7). This function returns a default BehavioralResult;
|
||||
/// each Phase 2B test rewrites the body to call the trainer methods it
|
||||
/// requires. The signature here is the load-bearing piece — keep it stable.
|
||||
#[allow(dead_code, unused_variables)]
|
||||
pub fn evaluate_policy_on_market(
|
||||
// Phase 2B swaps `&mut ()` for `&mut GpuDqnTrainer` once the per-test
|
||||
// trainer methods land. Until then the unit param keeps the signature
|
||||
// callable in tests that don't yet exercise the trainer path.
|
||||
_trainer: &mut (),
|
||||
_bars: &[LobBar],
|
||||
_n_warm: usize,
|
||||
) -> BehavioralResult {
|
||||
// Phase 2B wires this to trainer.eval_actions_on_features +
|
||||
// launch_sp15_cost_net_sharpe + ISV DD-slot reads.
|
||||
BehavioralResult::default()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn behavioral_result_default_is_empty() {
|
||||
let r = BehavioralResult::default();
|
||||
assert_eq!(r.trade_count, 0);
|
||||
assert!(r.action_dist.is_empty());
|
||||
}
|
||||
}
|
||||
34
crates/ml/tests/behavioral/main.rs
Normal file
34
crates/ml/tests/behavioral/main.rs
Normal file
@@ -0,0 +1,34 @@
|
||||
//! SP15 Phase 2 behavioral test suite — runs locally on dev RTX 3050 Ti.
|
||||
//! Pre-commit hook gates argo-train.sh: failing test blocks L40S deploy.
|
||||
|
||||
mod synthetic_markets;
|
||||
mod oracle;
|
||||
mod harness;
|
||||
|
||||
// Group 1: trading behavior (7 tests)
|
||||
mod test_2_1_flat_market_holds;
|
||||
mod test_2_2_trend_market_directional;
|
||||
mod test_2_3_mean_revert_reverses;
|
||||
mod test_2_4_cost_sensitivity;
|
||||
mod test_2_6_regime_silences;
|
||||
mod test_2_7_stop_discipline;
|
||||
mod test_2_18_action_latency;
|
||||
|
||||
// Group 2: state/arch grounding (10 tests)
|
||||
mod test_2_8_hold_vs_flat_semantics;
|
||||
mod test_2_9_eval_fold_isolation;
|
||||
mod test_2_13_eval_train_consistency;
|
||||
mod test_2_14_magnitude_uses_all_buckets;
|
||||
mod test_2_15_fold_transition_stability;
|
||||
mod test_2_16_q_value_bounded;
|
||||
mod test_2_17_gradient_flow_to_all_heads;
|
||||
mod test_2_19_kelly_calibration;
|
||||
mod test_2_20_state_input_grounding;
|
||||
mod test_2_21_egf_gate_opens;
|
||||
|
||||
// Group 3 (Phase 2C) lands paired with Phase 3.5 commits:
|
||||
// mod test_2_5_drawdown_de_risks;
|
||||
// mod test_2_10_recovery_after_streak;
|
||||
// mod test_2_11_dd_state_aware_sizing;
|
||||
// mod test_2_12_cooldown_engagement;
|
||||
// mod test_2_22_plasticity_cooldown_interlock;
|
||||
80
crates/ml/tests/behavioral/oracle.rs
Normal file
80
crates/ml/tests/behavioral/oracle.rs
Normal file
@@ -0,0 +1,80 @@
|
||||
//! SP15 Phase 2A — oracle policies for behavioral test harness.
|
||||
//! Each oracle is a known-correct action sequence for a given synthetic
|
||||
//! market type; tests assert the trained policy approaches the oracle.
|
||||
|
||||
use ml::cuda_pipeline::lob_bar::LobBar;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub enum MagBucket { Quarter, Half, Full }
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub enum OracleAction {
|
||||
Hold,
|
||||
Long(MagBucket),
|
||||
Short(MagBucket),
|
||||
Flat,
|
||||
}
|
||||
|
||||
/// Oracle for flat markets: always Hold.
|
||||
pub fn flat_market_oracle(bars: &[LobBar]) -> Vec<OracleAction> {
|
||||
vec![OracleAction::Hold; bars.len()]
|
||||
}
|
||||
|
||||
/// Oracle for drift markets: take direction-of-trend, mag = Half.
|
||||
pub fn drift_market_oracle(bars: &[LobBar]) -> Vec<OracleAction> {
|
||||
let mut actions = Vec::with_capacity(bars.len());
|
||||
actions.push(OracleAction::Hold); // first bar: no prior
|
||||
for w in bars.windows(2) {
|
||||
let action = if w[1].price > w[0].price + 0.05 {
|
||||
OracleAction::Long(MagBucket::Half)
|
||||
} else if w[1].price < w[0].price - 0.05 {
|
||||
OracleAction::Short(MagBucket::Half)
|
||||
} else {
|
||||
OracleAction::Hold
|
||||
};
|
||||
actions.push(action);
|
||||
}
|
||||
actions
|
||||
}
|
||||
|
||||
/// Oracle for OU markets: reverse at ±2σ extremes.
|
||||
pub fn ou_market_oracle(bars: &[LobBar], mu: f32, sigma: f32) -> Vec<OracleAction> {
|
||||
bars.iter().map(|b| {
|
||||
let z = (b.price - mu) / sigma.max(1e-6);
|
||||
if z > 2.0 { OracleAction::Short(MagBucket::Half) }
|
||||
else if z < -2.0 { OracleAction::Long(MagBucket::Half) }
|
||||
else { OracleAction::Hold }
|
||||
}).collect()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::synthetic_markets::*;
|
||||
|
||||
#[test]
|
||||
fn flat_oracle_all_hold() {
|
||||
let bars = flat_market(100, 0.1, 0.25, 42);
|
||||
let actions = flat_market_oracle(&bars);
|
||||
assert_eq!(actions.len(), 100);
|
||||
assert!(actions.iter().all(|a| *a == OracleAction::Hold));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn drift_oracle_takes_uptrend() {
|
||||
let bars = drift_market(1000, 0.5, 0.1, 0.25, 42);
|
||||
let actions = drift_market_oracle(&bars);
|
||||
let long_count = actions.iter().filter(|a| matches!(a, OracleAction::Long(_))).count();
|
||||
// Strong uptrend → most actions should be Long
|
||||
assert!(long_count > 700, "drift_market_oracle Long count = {}, expected > 700", long_count);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ou_oracle_reverses_at_extremes() {
|
||||
let bars = ou_market(1000, 0.05, 1.0, 4500.0, 0.25, 42);
|
||||
let actions = ou_market_oracle(&bars, 4500.0, 1.0);
|
||||
// OU oracle should produce SOME reversion actions (not all Hold)
|
||||
let action_count = actions.iter().filter(|a| !matches!(a, OracleAction::Hold)).count();
|
||||
assert!(action_count > 0, "OU oracle produced zero non-Hold actions");
|
||||
}
|
||||
}
|
||||
123
crates/ml/tests/behavioral/synthetic_markets.rs
Normal file
123
crates/ml/tests/behavioral/synthetic_markets.rs
Normal file
@@ -0,0 +1,123 @@
|
||||
//! SP15 Phase 2A — synthetic market generators producing LobBar streams.
|
||||
//! Used by Phase 2B behavioral tests + Phase 1.2 cost kernel verification.
|
||||
|
||||
use ml::cuda_pipeline::lob_bar::LobBar;
|
||||
use rand::{Rng, SeedableRng};
|
||||
use rand::rngs::StdRng;
|
||||
|
||||
/// Flat market: constant base price + Gaussian noise.
|
||||
pub fn flat_market(n: usize, noise_sigma: f32, mean_spread: f32, seed: u64) -> Vec<LobBar> {
|
||||
let mut rng = StdRng::seed_from_u64(seed);
|
||||
(0..n).map(|_| {
|
||||
let noise: f32 = rng.gen_range(-1.0..1.0) * noise_sigma;
|
||||
let price = 4500.0 + noise;
|
||||
let spread = (mean_spread + rng.gen_range(-0.05f32..0.05)).max(0.05);
|
||||
let ofi = rng.gen_range(-1.0..1.0);
|
||||
LobBar::new(price, spread, ofi)
|
||||
}).collect()
|
||||
}
|
||||
|
||||
/// Drift market: drift μ + noise σ.
|
||||
pub fn drift_market(n: usize, mu: f32, sigma: f32, mean_spread: f32, seed: u64) -> Vec<LobBar> {
|
||||
let mut rng = StdRng::seed_from_u64(seed);
|
||||
let mut price: f32 = 4500.0;
|
||||
(0..n).map(|_| {
|
||||
let dz: f32 = rng.gen_range(-1.0f32..1.0);
|
||||
price += mu + sigma * dz;
|
||||
let spread = (mean_spread + rng.gen_range(-0.05f32..0.05)).max(0.05);
|
||||
let ofi = (mu / sigma.max(1e-6)).tanh() + rng.gen_range(-0.5f32..0.5);
|
||||
LobBar::new(price, spread, ofi)
|
||||
}).collect()
|
||||
}
|
||||
|
||||
/// OU process: mean-reverting around equilibrium.
|
||||
pub fn ou_market(n: usize, theta: f32, sigma_eq: f32, mu: f32,
|
||||
mean_spread: f32, seed: u64) -> Vec<LobBar> {
|
||||
let mut rng = StdRng::seed_from_u64(seed);
|
||||
let mut price: f32 = mu;
|
||||
(0..n).map(|_| {
|
||||
let dz: f32 = rng.gen_range(-1.0f32..1.0);
|
||||
price += theta * (mu - price) + sigma_eq * dz;
|
||||
let spread = (mean_spread + rng.gen_range(-0.05f32..0.05)).max(0.05);
|
||||
let ofi = ((mu - price) / sigma_eq.max(1e-6)).tanh() + rng.gen_range(-0.3f32..0.3);
|
||||
LobBar::new(price, spread, ofi)
|
||||
}).collect()
|
||||
}
|
||||
|
||||
/// Regime-switch: Markov chain over `n_regimes`, each with (mu, sigma).
|
||||
pub fn regime_switch_market(
|
||||
n: usize,
|
||||
regime_params: &[(f32, f32)],
|
||||
transition_matrix: &[Vec<f32>],
|
||||
mean_spread: f32,
|
||||
seed: u64,
|
||||
) -> Vec<LobBar> {
|
||||
assert_eq!(regime_params.len(), transition_matrix.len());
|
||||
let mut rng = StdRng::seed_from_u64(seed);
|
||||
let mut current_regime = 0usize;
|
||||
let mut price: f32 = 4500.0;
|
||||
(0..n).map(|i| {
|
||||
if i > 0 {
|
||||
let r: f32 = rng.gen_range(0.0..1.0);
|
||||
let mut cum = 0.0f32;
|
||||
for (next, p) in transition_matrix[current_regime].iter().enumerate() {
|
||||
cum += p;
|
||||
if r < cum { current_regime = next; break; }
|
||||
}
|
||||
}
|
||||
let (mu, sigma) = regime_params[current_regime];
|
||||
let dz: f32 = rng.gen_range(-1.0f32..1.0);
|
||||
price += mu + sigma * dz;
|
||||
let spread = (mean_spread + rng.gen_range(-0.05f32..0.05)).max(0.05);
|
||||
let ofi = (mu / sigma.max(1e-6)).tanh() + rng.gen_range(-0.5f32..0.5);
|
||||
LobBar::new(price, spread, ofi)
|
||||
}).collect()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn flat_market_returns_n_bars_with_mean_near_base() {
|
||||
let bars = flat_market(1000, 0.1, 0.25, 42);
|
||||
assert_eq!(bars.len(), 1000);
|
||||
let mean: f32 = bars.iter().map(|b| b.price).sum::<f32>() / bars.len() as f32;
|
||||
assert!((mean - 4500.0).abs() < 0.5,
|
||||
"flat market mean = {}, expected ~4500", mean);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn drift_market_drifts_in_mu_direction() {
|
||||
let bars = drift_market(1000, 0.5, 0.1, 0.25, 42);
|
||||
let first = bars[0].price;
|
||||
let last = bars[999].price;
|
||||
assert!(last > first + 100.0,
|
||||
"drift_market with mu=0.5 should drift up by ~500 over 1000 bars; got {} → {}",
|
||||
first, last);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ou_market_reverts_to_mu() {
|
||||
let bars = ou_market(1000, 0.05, 1.0, 4500.0, 0.25, 42);
|
||||
let mean: f32 = bars.iter().map(|b| b.price).sum::<f32>() / bars.len() as f32;
|
||||
assert!((mean - 4500.0).abs() < 5.0,
|
||||
"OU mean = {}, should revert near mu=4500", mean);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regime_switch_all_regimes_visited() {
|
||||
// Sticky regimes: 0.99/0.01 produces ~100-step regime runs at mu=±0.5,
|
||||
// so a single bull run drifts +50 and a single bear run drifts -50,
|
||||
// giving a wide price range. Per spec §4.4, regime markets are
|
||||
// characterised by regime persistence — 50/50 transition is a
|
||||
// random walk, not a regime switch.
|
||||
let regimes = vec![(0.5f32, 0.1f32), (-0.5, 0.1)];
|
||||
let trans = vec![vec![0.99f32, 0.01], vec![0.01, 0.99]];
|
||||
let bars = regime_switch_market(1000, ®imes, &trans, 0.25, 42);
|
||||
let max_p = bars.iter().map(|b| b.price).fold(f32::NEG_INFINITY, f32::max);
|
||||
let min_p = bars.iter().map(|b| b.price).fold(f32::INFINITY, f32::min);
|
||||
assert!(max_p > min_p + 50.0,
|
||||
"regime switch should produce wide price range; got [{}, {}]", min_p, max_p);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
//! KL(train-mode, eval-mode action dists) < 0.5.
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — requires train/eval mode toggle harness"]
|
||||
fn test_2_13_eval_train_consistency() {
|
||||
// Phase 2B.b: trainer.set_eval_mode(bool) + paired evaluation.
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
//! Each magnitude bucket > 5% of trade events.
|
||||
use crate::synthetic_markets::drift_market;
|
||||
use crate::harness::{evaluate_policy_on_market, BehavioralResult};
|
||||
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green when Phase 3.5 entropy-aware Hold + magnitude lands"]
|
||||
fn test_2_14_magnitude_uses_all_buckets() {
|
||||
let bars = drift_market(20_000, 0.5, 0.5, 0.25, 42);
|
||||
let mut trainer = ();
|
||||
let result: BehavioralResult = evaluate_policy_on_market(&mut trainer, &bars[5_000..], 0);
|
||||
for key in ["long_quarter", "long_half", "long_full"] {
|
||||
let pct = result.action_dist.get(key).copied().unwrap_or(0.0);
|
||||
assert!(pct >= 0.05, "test 2.14 FAIL: mag bucket {} only {:.2}%", key, pct * 100.0);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
//! Sharpe collapse at fold boundary < 50%.
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green via existing StateResetRegistry"]
|
||||
fn test_2_15_fold_transition_stability() {
|
||||
// Train on synthetic fold 1; capture sharpe at end. Reset state. Train fold 2;
|
||||
// capture sharpe at start. assert!((end - start).abs() / end.max(0.01) < 0.50);
|
||||
//
|
||||
// Phase 2B.b: requires fold-boundary scaffolding the harness doesn't yet expose.
|
||||
}
|
||||
6
crates/ml/tests/behavioral/test_2_16_q_value_bounded.rs
Normal file
6
crates/ml/tests/behavioral/test_2_16_q_value_bounded.rs
Normal file
@@ -0,0 +1,6 @@
|
||||
//! Q-values stay in [V_MIN, V_MAX] under K=1000 replay updates.
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green via existing C51 V-range; requires Q snapshot harness"]
|
||||
fn test_2_16_q_value_bounded() {
|
||||
// Phase 2B.b: trainer.snapshot_q_values() + replay-update loop.
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
//! Non-zero grad reaches every head (dir/mag/ord/urg/aux/CQL/IQN/C51/ens).
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green via architecture; requires per-head grad-norm harness"]
|
||||
fn test_2_17_gradient_flow_to_all_heads() {
|
||||
// Phase 2B.b: trainer.snapshot_per_head_grad_norms() after one backward pass.
|
||||
}
|
||||
27
crates/ml/tests/behavioral/test_2_18_action_latency.rs
Normal file
27
crates/ml/tests/behavioral/test_2_18_action_latency.rs
Normal file
@@ -0,0 +1,27 @@
|
||||
//! SP15 Phase 2B Group 1 — action latency: Hold→Long within 5 bars of +5σ signal.
|
||||
//! Anchor for: Phase 3 (responsiveness via teaching).
|
||||
|
||||
use crate::synthetic_markets::flat_market;
|
||||
use crate::harness::BehavioralResult;
|
||||
use ml::cuda_pipeline::lob_bar::LobBar;
|
||||
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — requires per-bar action access from harness (Phase 2B.b)"]
|
||||
fn test_2_18_action_latency() {
|
||||
// Flat for 5000 bars, then step-function +drift starts at bar 5000
|
||||
let mut bars = flat_market(5_000, 0.1, 0.25, 42);
|
||||
let drift_start_price = bars.last().unwrap().price;
|
||||
for i in 0..5_000 {
|
||||
let p = drift_start_price + 0.5 * (i as f32);
|
||||
bars.push(LobBar::new(p, 0.25, 1.0));
|
||||
}
|
||||
|
||||
// Once harness exposes per-bar actions:
|
||||
// let switch_bar = result.actions.iter().skip(5_000)
|
||||
// .position(|a| matches!(a, ActionEnum::Long(_)))
|
||||
// .map(|p| p + 5_000);
|
||||
// assert!(switch_bar.unwrap_or(usize::MAX) < 5_005, ...);
|
||||
|
||||
let _ = bars; // contract document
|
||||
let _ = BehavioralResult::default();
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//! kelly_f monotonically decreasing in Q-variance.
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green via existing Kelly; requires Q-variance sweep harness"]
|
||||
fn test_2_19_kelly_calibration() {
|
||||
// Phase 2B.b: trainer.set_q_variance_for_test(v) + read kelly_f from ISV.
|
||||
// Sweep v ∈ {0.01, 0.1, 1.0}, assert kelly_f[0] > kelly_f[1] > kelly_f[2].
|
||||
}
|
||||
18
crates/ml/tests/behavioral/test_2_1_flat_market_holds.rs
Normal file
18
crates/ml/tests/behavioral/test_2_1_flat_market_holds.rs
Normal file
@@ -0,0 +1,18 @@
|
||||
//! SP15 Phase 2B Group 1 — flat market should produce ≥80% Hold.
|
||||
//! Anchor for: Phase 3.5 confidence-aware Hold floor (sigmoid).
|
||||
|
||||
use crate::synthetic_markets::flat_market;
|
||||
use crate::harness::{evaluate_policy_on_market, BehavioralResult};
|
||||
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green when Phase 3.5 Hold floor lands"]
|
||||
fn test_2_1_flat_market_holds() {
|
||||
let bars = flat_market(10_000, 0.1, 0.25, 42);
|
||||
// Phase 2B.b will replace this with minimal_trainer_for_tests() once that helper exists.
|
||||
let mut trainer = ();
|
||||
let result: BehavioralResult = evaluate_policy_on_market(&mut trainer, &bars[5_000..], 0);
|
||||
|
||||
let hold_pct = result.action_dist.get("hold").copied().unwrap_or(0.0);
|
||||
assert!(hold_pct >= 0.80,
|
||||
"test 2.1 FAIL: flat market Hold rate = {:.2}, expected >= 0.80", hold_pct);
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//! KL(uptrend-context, downtrend-context action dists) > 0.5.
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green via Phase 1.5 dd_pct trunk concat (already landed) + Phase 1.5.b state_dim bump"]
|
||||
fn test_2_20_state_input_grounding() {
|
||||
// Run policy on uptrend then downtrend (same dd_pct=0); compute KL.
|
||||
// Phase 2B.b: action-dist KL helper.
|
||||
}
|
||||
15
crates/ml/tests/behavioral/test_2_21_egf_gate_opens.rs
Normal file
15
crates/ml/tests/behavioral/test_2_21_egf_gate_opens.rs
Normal file
@@ -0,0 +1,15 @@
|
||||
//! Phase 0 anchor: aux/Q deliberate disagreement → gate1 opens within 100 steps.
|
||||
//! ALREADY IMPLEMENTED in crates/ml/tests/sp14_oracle_tests.rs (Task 0.C will land it
|
||||
//! once Phase 0.B retune completes). This file is a thin re-export hook so the
|
||||
//! behavioral_suite test target picks it up.
|
||||
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green via Phase 0.B (test lives in sp14_oracle_tests.rs)"]
|
||||
fn test_2_21_egf_gate_opens() {
|
||||
// The actual implementation lives in:
|
||||
// crates/ml/tests/sp14_oracle_tests.rs::gpu::egf_gate_opens_under_disagreement
|
||||
//
|
||||
// This re-export hook documents that test 2.21 EXISTS in the canonical
|
||||
// behavioral_suite contract. The implementation is reachable via the GPU
|
||||
// oracle test path; this stub is the catalog entry.
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
//! SP15 Phase 2B Group 1 — trend market should produce ≥70% direction-of-trend.
|
||||
//! Anchor for: Phase 1+3 (cost + trader teachings).
|
||||
|
||||
use crate::synthetic_markets::drift_market;
|
||||
use crate::harness::{evaluate_policy_on_market, BehavioralResult};
|
||||
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green when Phase 1+3 land"]
|
||||
fn test_2_2_trend_market_directional() {
|
||||
let bars = drift_market(10_000, 0.5, 0.1, 0.25, 42);
|
||||
let mut trainer = ();
|
||||
let result: BehavioralResult = evaluate_policy_on_market(&mut trainer, &bars[5_000..], 0);
|
||||
|
||||
let long_pct: f32 = result.action_dist.iter()
|
||||
.filter(|(k, _)| k.starts_with("long_"))
|
||||
.map(|(_, v)| *v)
|
||||
.sum();
|
||||
assert!(long_pct >= 0.70,
|
||||
"test 2.2 FAIL: trend market Long rate = {:.2}, expected >= 0.70 on +drift", long_pct);
|
||||
}
|
||||
32
crates/ml/tests/behavioral/test_2_3_mean_revert_reverses.rs
Normal file
32
crates/ml/tests/behavioral/test_2_3_mean_revert_reverses.rs
Normal file
@@ -0,0 +1,32 @@
|
||||
//! SP15 Phase 2B Group 1 — mean-revert market should reverse direction at ±2σ within 5 bars.
|
||||
//! Anchor for: Phase 1+3.
|
||||
|
||||
use crate::synthetic_markets::ou_market;
|
||||
use crate::harness::{evaluate_policy_on_market, BehavioralResult};
|
||||
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green when Phase 1+3 land + harness exposes per-bar actions"]
|
||||
fn test_2_3_mean_revert_reverses() {
|
||||
let mu: f32 = 4500.0;
|
||||
let sigma: f32 = 1.0;
|
||||
let bars = ou_market(10_000, 0.05, sigma, mu, 0.25, 42);
|
||||
let mut trainer = ();
|
||||
let _result: BehavioralResult = evaluate_policy_on_market(&mut trainer, &bars[5_000..], 0);
|
||||
|
||||
// The full assertion requires per-bar action access from BehavioralResult,
|
||||
// which the Phase 2A harness stub does not yet expose. Phase 2B.b will
|
||||
// extend BehavioralResult with `actions: Vec<ActionEnum>`. Until then,
|
||||
// this test serves as the contract specification.
|
||||
//
|
||||
// Once per-bar actions are exposed:
|
||||
// for (i, bar) in bars.iter().enumerate() {
|
||||
// let z = (bar.price - mu) / sigma;
|
||||
// if z.abs() > 2.0 {
|
||||
// let next_5 = &result.actions[i+1..(i+6).min(result.actions.len())];
|
||||
// let expected_dir = if z > 0.0 { /* Short */ } else { /* Long */ };
|
||||
// assert!(next_5.iter().any(|a| matches_direction(a, expected_dir)),
|
||||
// "no reverse at z={:.2}", z);
|
||||
// }
|
||||
// }
|
||||
// For now the test compiles + runs (no-op past the smoke).
|
||||
}
|
||||
24
crates/ml/tests/behavioral/test_2_4_cost_sensitivity.rs
Normal file
24
crates/ml/tests/behavioral/test_2_4_cost_sensitivity.rs
Normal file
@@ -0,0 +1,24 @@
|
||||
//! SP15 Phase 2B Group 1 — cost sensitivity: trade freq drops ≥40% under high spread.
|
||||
//! Anchor for: Phase 3.2 explicit cost in r_quality.
|
||||
|
||||
use crate::synthetic_markets::drift_market;
|
||||
use crate::harness::{evaluate_policy_on_market, BehavioralResult};
|
||||
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green when Phase 3.2 lands"]
|
||||
fn test_2_4_cost_sensitivity() {
|
||||
// Same +drift series at different mean spreads
|
||||
let bars_low = drift_market(10_000, 0.5, 0.1, /*mean_spread*/ 0.1, 42);
|
||||
let bars_high = drift_market(10_000, 0.5, 0.1, /*mean_spread*/ 2.0, 42);
|
||||
|
||||
let mut trainer = ();
|
||||
let result_low: BehavioralResult = evaluate_policy_on_market(&mut trainer, &bars_low[5_000..], 0);
|
||||
let result_high: BehavioralResult = evaluate_policy_on_market(&mut trainer, &bars_high[5_000..], 0);
|
||||
|
||||
let low = result_low.trade_count as f32;
|
||||
let high = result_high.trade_count as f32;
|
||||
assert!(low > 0.0, "low-spread trade_count = 0; cannot compute sensitivity");
|
||||
let drop = (low - high) / low;
|
||||
assert!(drop >= 0.40,
|
||||
"test 2.4 FAIL: trade-freq drop = {:.2}, expected >= 0.40 (high-spread)", drop);
|
||||
}
|
||||
18
crates/ml/tests/behavioral/test_2_6_regime_silences.rs
Normal file
18
crates/ml/tests/behavioral/test_2_6_regime_silences.rs
Normal file
@@ -0,0 +1,18 @@
|
||||
//! SP15 Phase 2B Group 1 — high-noise no-edge regime: trade freq ≤20%.
|
||||
//! Anchor for: Phase 3.4 regret + Phase 3.5 confidence-aware Hold floor.
|
||||
|
||||
use crate::synthetic_markets::flat_market;
|
||||
use crate::harness::{evaluate_policy_on_market, BehavioralResult};
|
||||
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green when Phase 3.4 + 3.5 land"]
|
||||
fn test_2_6_regime_silences() {
|
||||
// High-noise flat market: no edge to extract
|
||||
let bars = flat_market(10_000, /*noise_sigma*/ 5.0, 0.25, 42);
|
||||
let mut trainer = ();
|
||||
let result: BehavioralResult = evaluate_policy_on_market(&mut trainer, &bars[5_000..], 0);
|
||||
|
||||
let trade_freq = result.trade_count as f32 / 5_000.0;
|
||||
assert!(trade_freq <= 0.20,
|
||||
"test 2.6 FAIL: trade freq = {:.2}, expected <= 0.20 in no-edge regime", trade_freq);
|
||||
}
|
||||
22
crates/ml/tests/behavioral/test_2_7_stop_discipline.rs
Normal file
22
crates/ml/tests/behavioral/test_2_7_stop_discipline.rs
Normal file
@@ -0,0 +1,22 @@
|
||||
//! SP15 Phase 2B Group 1 — adverse excursion past trail dist → close within 1 bar.
|
||||
//! Anchor for: existing trail-stop logic.
|
||||
|
||||
use crate::harness::BehavioralResult;
|
||||
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — requires forced-action harness extension (Phase 2B.b)"]
|
||||
fn test_2_7_stop_discipline() {
|
||||
// This test requires driving a forced Long entry, then injecting an
|
||||
// adverse-excursion price move past trail distance, then verifying the
|
||||
// policy emits Flat within 1 bar of the breach.
|
||||
//
|
||||
// Harness extensions needed (Phase 2B.b):
|
||||
// - trainer.step_with_forced_action(action: ActionEnum) -> Result<BarMetrics, MLError>
|
||||
// - bar-by-bar action observation
|
||||
//
|
||||
// Contract: after a trail breach detected at bar T, action[T+1] == Flat.
|
||||
|
||||
// Until the harness exposes forced-action stepping, this test is a
|
||||
// contract document. Phase 2B.b will populate the assertion body.
|
||||
let _ = BehavioralResult::default();
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
//! Hold preserves pre_pos; Flat zeros it. Anchor for: existing semantics.
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — requires forced-action + position-state inspection harness"]
|
||||
fn test_2_8_hold_vs_flat_semantics() {
|
||||
// Drive entry → Hold → assert pre_pos != 0
|
||||
// Drive entry → Flat → assert pre_pos == 0
|
||||
// Phase 2B.b harness extension required.
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
//! KL(shuffled-future, unshuffled-future) < 0.01 — proves no temporal leak.
|
||||
#[test]
|
||||
#[ignore = "Phase 2B contract — green via StateResetRegistry coverage; requires KL helper"]
|
||||
fn test_2_9_eval_fold_isolation() {
|
||||
// Phase 2B.b: harness exposes action_distribution_kl(actions_a, actions_b).
|
||||
}
|
||||
@@ -6798,6 +6798,18 @@ User asked at the start of this work: **does the model learn the directional sig
|
||||
The path from -24 sharpe to +16 sharpe in 4 epochs validates the underlying training mechanics. **30-epoch full validation should be deferred** until L1 + L2 are fixed, so the EGF pearl actually does work and the val numbers reflect genuine architectural improvement.
|
||||
|
||||
|
||||
## SP15 Phase 2A.1 — LobBar canonical ABI + synthetic market generators (2026-05-06)
|
||||
|
||||
**Files added:**
|
||||
- `crates/ml/src/cuda_pipeline/lob_bar.rs` — defines `#[repr(C)] struct LobBar { price, spread, ofi: f32 }` plus `into_soa()` AoS→SoA helper. Per spec §4.4 ABI contract, this is the canonical bar format consumed by both Phase 1.2 cost kernel and Phase 2 behavioral tests; dev synthetic generators and prod fxcache both produce LobBar (dev/prod parity per Q3).
|
||||
- `crates/ml/tests/behavioral/synthetic_markets.rs` — 4 seeded RNG generators: `flat_market`, `drift_market`, `ou_market`, `regime_switch_market`. Each consumes scalar params + seed, returns `Vec<LobBar>`.
|
||||
- `crates/ml/tests/behavioral/main.rs` — empty harness for the new `behavioral_suite` test target; will gain Phase 2A.2 oracle/harness modules and Phase 2B test_2_*.rs files in subsequent commits.
|
||||
- `crates/ml/Cargo.toml` — adds `[[test]] name = "behavioral_suite"` target so the suite runs via `cargo test -p ml --test behavioral_suite`.
|
||||
|
||||
**Wire-up status:** Phase 2A.1 lands FIRST per spec — Phase 1.2 cost kernel will conform to the LobBar ABI when it lands. No production code consumes `lob_bar.rs` yet (only the test crate); this is intentional scaffolding. The 4 unit tests inside `synthetic_markets::tests` validate generator behavior (mean centering for flat, drift direction for drift, mean reversion for OU, price-range spread for sticky regime-switch). All 4 pass on RTX 3050 Ti.
|
||||
|
||||
**Spec deviation noted:** spec gave the regime-switch test a 50/50 transition matrix; that produces a fast-flipping random walk, not a regime market. Test uses sticky 0.99/0.01 instead, with code comment explaining — this is what "regime switch" actually means and is what Phase 1.2 cost kernel will be tested against.
|
||||
|
||||
## SP15 Phase 1.6 — `--holdout-quarters` + `--dev-quarters` CLI flags + sealed Q1-Q7/Q8/Q9 split
|
||||
|
||||
**What landed (this commit):**
|
||||
|
||||
Reference in New Issue
Block a user