feat(dqn-v2): C.6 AdaptiveController trait + test harness
Unified protocol for every adaptive controller in the DQN trainer. FireRateStats for controller_activity smoke. DiagSnapshot for standardised HEALTH_DIAG emission. IsvBus abstraction over pinned device-mapped memory (&mut [f32] wrapper). No concrete controller migration yet — Tasks 9-17 migrate existing controllers (atoms → gamma → Kelly → cql_alpha → tau → epsilon → conviction_floor → plan_threshold → balancer) one per sub-commit. Old scaffolding for each controller is removed in the SAME commit that migrates its last consumer to the new protocol, per feedback_no_partial_refactor.md. Tests: 2 unit tests on FireRateStats and DiagSnapshot. Plan 1 Task 8. Spec §4.C.6. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
122
crates/ml/src/trainers/dqn/adaptive_controller.rs
Normal file
122
crates/ml/src/trainers/dqn/adaptive_controller.rs
Normal file
@@ -0,0 +1,122 @@
|
||||
//! Unified protocol for adaptive controllers in the DQN trainer.
|
||||
//! Spec §4.C.6.
|
||||
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// Per-controller fire-rate statistics. A "fire" is any observation where
|
||||
/// the controller emitted a control value different from its previous emission.
|
||||
/// Used by the `controller_activity` smoke test to verify load-bearing
|
||||
/// controllers don't fire in > 50% of epochs.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct FireRateStats {
|
||||
pub fire_count: u64,
|
||||
pub observation_count: u64,
|
||||
}
|
||||
|
||||
impl FireRateStats {
|
||||
pub fn record_fire(&mut self, fired: bool) {
|
||||
self.observation_count += 1;
|
||||
if fired {
|
||||
self.fire_count += 1;
|
||||
}
|
||||
}
|
||||
|
||||
pub fn fire_rate(&self) -> f64 {
|
||||
if self.observation_count == 0 {
|
||||
0.0
|
||||
} else {
|
||||
self.fire_count as f64 / self.observation_count as f64
|
||||
}
|
||||
}
|
||||
|
||||
pub fn reset(&mut self) {
|
||||
*self = Self::default();
|
||||
}
|
||||
}
|
||||
|
||||
/// Standardised diagnostic emission from an adaptive controller.
|
||||
/// Emitted at epoch boundary into HEALTH_DIAG.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct DiagSnapshot {
|
||||
/// Named fields → value. Example: `{"gamma": 0.95, "fire_rate": 0.12}`.
|
||||
pub fields: HashMap<String, f64>,
|
||||
}
|
||||
|
||||
impl DiagSnapshot {
|
||||
pub fn with(mut self, name: impl Into<String>, value: f64) -> Self {
|
||||
self.fields.insert(name.into(), value);
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
/// Unified protocol for adaptive controllers in the DQN trainer.
|
||||
///
|
||||
/// Each controller reads signals from the ISV bus, computes a control value
|
||||
/// (gamma, atom positions, Kelly floor, CQL alpha, etc.), writes its output
|
||||
/// back to ISV, and emits diagnostics.
|
||||
pub trait AdaptiveController {
|
||||
/// Opaque signal vector the controller reads.
|
||||
type Signal;
|
||||
/// Opaque control value the controller emits.
|
||||
type Control;
|
||||
|
||||
/// Read inputs from the ISV bus.
|
||||
fn read_signals(&self, isv: &IsvBus<'_>) -> Self::Signal;
|
||||
|
||||
/// Compute the new control value. May update internal state.
|
||||
fn update(&mut self, signals: Self::Signal) -> Self::Control;
|
||||
|
||||
/// Write the control value back to ISV / consumer buffers.
|
||||
fn write_output(&self, ctrl: &Self::Control, isv: &mut IsvBus<'_>);
|
||||
|
||||
/// Per-controller fire-rate statistics.
|
||||
fn fire_rate(&self) -> &FireRateStats;
|
||||
|
||||
/// Emit a diagnostic snapshot at epoch boundary.
|
||||
fn diagnose(&self) -> DiagSnapshot;
|
||||
|
||||
/// Controller name, used in HEALTH_DIAG field labels.
|
||||
fn name(&self) -> &'static str;
|
||||
}
|
||||
|
||||
/// Abstraction over the ISV bus for controllers.
|
||||
/// In production, this wraps `isv_signals_pinned: *mut f32`.
|
||||
/// In tests, a plain `Vec<f32>` is used for isolation.
|
||||
#[derive(Debug)]
|
||||
pub struct IsvBus<'a> {
|
||||
pub(crate) slots: &'a mut [f32],
|
||||
}
|
||||
|
||||
impl<'a> IsvBus<'a> {
|
||||
pub fn new(slots: &'a mut [f32]) -> Self {
|
||||
Self { slots }
|
||||
}
|
||||
pub fn read(&self, idx: usize) -> f32 {
|
||||
self.slots[idx]
|
||||
}
|
||||
pub fn write(&mut self, idx: usize, v: f32) {
|
||||
self.slots[idx] = v;
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn controller_fire_rate_stats_compute() {
|
||||
let mut stats = FireRateStats::default();
|
||||
stats.record_fire(true);
|
||||
stats.record_fire(false);
|
||||
stats.record_fire(true);
|
||||
assert_eq!(stats.fire_count, 2);
|
||||
assert_eq!(stats.observation_count, 3);
|
||||
assert!((stats.fire_rate() - 2.0 / 3.0).abs() < 1e-9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn diag_snapshot_empty_by_default() {
|
||||
let snap = DiagSnapshot::default();
|
||||
assert!(snap.fields.is_empty());
|
||||
}
|
||||
}
|
||||
@@ -18,6 +18,7 @@
|
||||
//! - `statistics` - Feature normalization and Q-value monitoring
|
||||
//! - `trainer` - Main DQNTrainer implementation (core training loop)
|
||||
|
||||
pub mod adaptive_controller;
|
||||
pub(crate) mod adversarial_self_play;
|
||||
mod config;
|
||||
mod data_loading;
|
||||
@@ -36,6 +37,7 @@ pub mod state_reset_registry;
|
||||
pub use state_reset_registry::{ResetCategory, RegistryEntry, StateResetRegistry};
|
||||
|
||||
// Re-export all public items for backward compatibility
|
||||
pub use adaptive_controller::{AdaptiveController, FireRateStats, DiagSnapshot, IsvBus};
|
||||
pub use config::{DQNAgentType, DQNHyperparameters};
|
||||
pub use early_stopping::EarlyStopping;
|
||||
pub use lr_scheduler::{LRDecayType, LRScheduler};
|
||||
|
||||
@@ -42,6 +42,7 @@
|
||||
| `trainers/dqn/trainer/metrics.rs` | `training_loop.rs` | Wired | Epoch-boundary metric compilation + backtest eval | — |
|
||||
| `trainers/dqn/trainer/state.rs` | `trainer/mod.rs` | Wired | Mutable training state | — |
|
||||
| `trainers/dqn/trainer/training_loop.rs` | called from `DQNTrainer::train()` | Wired | Main epoch loop | — |
|
||||
| `trainers/dqn/adaptive_controller.rs` | Trait + harness (`FireRateStats`, `DiagSnapshot`, `IsvBus<'a>`); consumers added in Plan 1 Tasks 9-17 (atoms, gamma, Kelly, cql_alpha, tau, epsilon, conviction_floor, plan_threshold, grad_balancer migrations) | Wired (consumers added in same plan) | C.6 trait foundation | — |
|
||||
|
||||
## CUDA Pipeline — Rust Wrappers
|
||||
|
||||
@@ -216,16 +217,16 @@
|
||||
|
||||
## Summary
|
||||
|
||||
Updated after Task 6 cleanup (2026-04-24): 5 confirmed-orphan files deleted, 3 Orphan rows reclassified Partial, 1 Orphan reclassified with crate-level follow-up action.
|
||||
Updated after Task 6 cleanup (2026-04-24): 5 confirmed-orphan files deleted, 3 Orphan rows reclassified Partial, 1 Orphan reclassified with crate-level follow-up action. Plan 1 Task 8 (2026-04-24): added `adaptive_controller.rs` (C.6 trait foundation).
|
||||
|
||||
| Classification | Count |
|
||||
|---|---|
|
||||
| Wired | 74 |
|
||||
| Wired | 75 |
|
||||
| Partial | 10 |
|
||||
| Orphan (held for follow-up) | 3 |
|
||||
| Ghost | 0 |
|
||||
| OUT-of-DQN-scope | 17 |
|
||||
| **Total** | **104** |
|
||||
| **Total** | **105** |
|
||||
|
||||
The 3 remaining Orphan rows are:
|
||||
- `cuda_pipeline/gpu_statistics.rs` + `statistics_kernel.cu` — held for Plan 2 D.2 wire-or-delete decision.
|
||||
|
||||
Reference in New Issue
Block a user