feat(dqn-v2): A.2 ISV layout fingerprint at ISV[37..39) (tail placement)

Implements spec §4.A.2 structural layout fingerprint with tail placement
rather than head placement (spec alternative: §4.A.2 Step 5.3 alt).

Head placement (ISV[0..2)) was rejected because isv_signals[0] and [1]
are actively written by the isv_signal_update kernel (Q-drift EMA and
gradient-norm EMA). Shifting those would require updating every literal
reference in experience_kernels.cu — a larger change than warranted for
pure contract enforcement. Tail placement leaves all existing indices
intact, touches zero kernel .cu files, and fulfils the same design contract.

Key changes:
- ISV_LAYOUT_FINGERPRINT_LO_INDEX = 37, HI_INDEX = 38 (u64 across 2×f32).
- LAYOUT_FINGERPRINT_CURRENT: u64 = FNV-1a of slot-list seed bytes.
  Value: 0x85d4d76b578a7c17. Any slot change updates seed bytes,
  which updates the hash automatically.
- Constructor writes fingerprint after zero-init; calls
  check_layout_fingerprint() to self-verify before returning.
- check_layout_fingerprint(): reads pinned slots [37..39), recomposes u64,
  fails-fast on mismatch with "retrain required" message.
- Error message does NOT mention migration as an option.
- Pre-commit hook rejects `fn migrate_isv|upgrade_isv` names — makes the
  no-migration rule structurally enforced (check_no_isv_migrations).
- ISV_TOTAL_DIM: 37 → 39.
- Zero existing index shifts (no kernel literal sites affected).
- StateResetRegistry entry renamed ISV_SCHEMA_VERSION → ISV_LAYOUT_FINGERPRINT.
- ResetCategory::SchemaContract docstring updated to remove "migration" framing.
- docs/isv-slots.md: updated table + design note for tail placement.

Tests: state_reset_registry 3 unit tests pass with renamed entry.
       cargo check -p ml clean (pre-existing warnings only).

Plan 1 Task 5. Spec §4.A.2 (tail-placement alternative).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-24 12:24:48 +02:00
parent 1353e71ae6
commit fbb8694a0b
4 changed files with 186 additions and 20 deletions

View File

@@ -164,10 +164,17 @@ const ISV_NETWORK_DIM: usize = 23;
/// to ensure no (sample, branch) pair collapses to zero gradient
/// contribution.
///
/// Extending past 37 requires updating the pinned allocation in the
/// Slots [37..39) carry the layout fingerprint (ISV_LAYOUT_FINGERPRINT_LO_INDEX
/// and ISV_LAYOUT_FINGERPRINT_HI_INDEX). These are placed at the tail because
/// slots [0] and [1] are actively written by `isv_signal_update` (Q-drift and
/// gradient-norm EMA respectively). Written by the constructor; checked at
/// checkpoint load. Fail-fast only — no migration path exists. See spec §4.A.2
/// and `LAYOUT_FINGERPRINT_CURRENT` for the structural-hash design rationale.
///
/// Extending past 39 requires updating the pinned allocation in the
/// constructor and any kernel that takes an ISV pointer expecting a
/// specific length.
const ISV_TOTAL_DIM: usize = 37;
const ISV_TOTAL_DIM: usize = 39;
/// Legacy alias preserved for call sites that haven't been audited for the
/// network-vs-total split. New code should pick `ISV_NETWORK_DIM` (for weight
/// tensor sizing) or `ISV_TOTAL_DIM` (for the broadcast bus buffer).
@@ -255,6 +262,79 @@ pub const GRAD_SCALE_LIMIT_INDEX: usize = 35;
/// actively EMA-tracked; treated as a safety floor per
/// `feedback_isv_for_adaptive_bounds.md` carve-out.
pub const IQL_BRANCH_SCALE_FLOOR_INDEX: usize = 36;
/// ISV slot [37] — low 32 bits of the u64 layout fingerprint (stored as raw f32 bits).
///
/// Design note: this is NOT a version number. There is no ordered version space.
/// The fingerprint is a structural hash that automatically changes whenever any
/// slot is added, removed, reordered, or renamed — so there is no natural pairing
/// "v1 -> v2" and no place to write a migrator. Per spec §4.A.2 and
/// `feedback_no_legacy_aliases.md`: backward compat is structurally unwritable,
/// not merely discouraged.
///
/// Placement at the tail (not head) is mandated by slots [0] and [1] being
/// actively written by `isv_signal_update` (Q-drift and gradient-norm EMA).
pub const ISV_LAYOUT_FINGERPRINT_LO_INDEX: usize = 37;
/// ISV slot [38] — high 32 bits of the u64 layout fingerprint (stored as raw f32 bits).
pub const ISV_LAYOUT_FINGERPRINT_HI_INDEX: usize = 38;
/// Canonical alias for the fingerprint slot (the low half).
pub const ISV_LAYOUT_FINGERPRINT_INDEX: usize = ISV_LAYOUT_FINGERPRINT_LO_INDEX;
/// FNV-1a 64-bit hash of a byte slice, usable as a `const fn`.
const fn fnv1a_64(bytes: &[u8]) -> u64 {
const OFFSET: u64 = 0xcbf29ce484222325;
const PRIME: u64 = 0x00000100000001b3;
let mut h: u64 = OFFSET;
let mut i = 0;
while i < bytes.len() {
h ^= bytes[i] as u64;
h = h.wrapping_mul(PRIME);
i += 1;
}
h
}
/// Source bytes for the layout fingerprint. Canonical format:
/// `b"<slot_name>=<index>;<slot_name>=<index>;...;ISV_TOTAL_DIM=<N>"`.
///
/// When adding/removing a slot, add/remove its entry here in the SAME commit
/// that changes the constant list. The fingerprint recomputes automatically.
/// Order matches the constant declaration order in this module.
///
/// Slots [0..12) have no named `pub const` (they are written directly via
/// literal indices in `isv_signal_update`). They are covered here by
/// `SLOT_<N>_<PURPOSE>` entries derived from the `ISV_TOTAL_DIM` docstring
/// and `experience_kernels.cu` write-site comments. Any renumber of these
/// slots changes the seed and invalidates the fingerprint.
const fn layout_fingerprint_seed() -> &'static [u8] {
b"SLOT_0_Q_DRIFT=0;\
SLOT_1_GRAD_NORM_EMA=1;\
SLOT_2_TD_ERR_EMA=2;\
SLOT_3_ENS_VAR_EMA=3;\
SLOT_4_ENS_VAR_VEL=4;\
SLOT_5_REWARD_EMA=5;\
SLOT_6_ATOM_UTIL_EMA=6;\
SLOT_7_LOSS_EMA=7;\
SLOT_8_ADX_EMA=8;\
SLOT_9_REGIME_DISAGREE=9;\
SLOT_10_REGIME_VEL_EMA=10;\
SLOT_11_REGIME_STABILITY=11;\
LEARNING_HEALTH=12;\
Q_MAG_MEAN_QUARTER=13;Q_MAG_MEAN_HALF=14;Q_MAG_MEAN_FULL=15;Q_ABS_REF=16;\
Q_DIR_MEAN_SHORT=17;Q_DIR_MEAN_HOLD=18;Q_DIR_MEAN_LONG=19;Q_DIR_MEAN_FLAT=20;Q_DIR_ABS_REF=21;\
SHARPE_EMA=22;\
V_CENTER_DIR=23;V_HALF_DIR=24;V_CENTER_MAG=25;V_HALF_MAG=26;\
V_CENTER_ORD=27;V_HALF_ORD=28;V_CENTER_URG=29;V_HALF_URG=30;\
GRAD_NORM_TARGET_DIR=31;GRAD_NORM_TARGET_MAG=32;GRAD_NORM_TARGET_ORD=33;GRAD_NORM_TARGET_URG=34;\
GRAD_SCALE_LIMIT=35;IQL_BRANCH_SCALE_FLOOR=36;\
ISV_LAYOUT_FINGERPRINT_LO=37;ISV_LAYOUT_FINGERPRINT_HI=38;\
ISV_TOTAL_DIM=39"
}
/// Compile-time layout fingerprint. Burned into the binary at build time.
/// Checked against the stored value at checkpoint load. Mismatch means retrain required.
pub const LAYOUT_FINGERPRINT_CURRENT: u64 = fnv1a_64(layout_fingerprint_seed());
const ISV_EMB_DIM: usize = 8; // ISV embedding output dimension (FC2 output, gate/gamma input)
/// First ISV tensor index in the flat param buffer.
/// ISV weights (68-79) are online-only — NOT synced to the target network.
@@ -1944,11 +2024,11 @@ pub struct GpuDqnTrainer {
q_dir_bin_means_reduce_kernel: CudaFunction,
// ── ISV core buffers (pinned device-mapped, GPU read/write) ──
// isv_signals_pinned is sized for ISV_TOTAL_DIM (31) — the extra eight
// isv_signals_pinned is sized for ISV_TOTAL_DIM (39) — the extra slots
// slots [23..31] carry per-branch Q-support centres/half-widths. The
// network (w_isv_fc1) and history rotation still consume only the first
// ISV_NETWORK_DIM (23) slots; the tail is broadcast-bus scratchpad only.
isv_signals_pinned: *mut f32, // [ISV_TOTAL_DIM = 31]
isv_signals_pinned: *mut f32, // [ISV_TOTAL_DIM = 39]
isv_signals_dev_ptr: u64,
isv_history_pinned: *mut f32, // [ISV_K * 12 = 48] — history rotates slots [0..11] only
isv_history_dev_ptr: u64,
@@ -8087,10 +8167,11 @@ impl GpuDqnTrainer {
};
// ── ISV core buffers (pinned device-mapped) ──────────────────────
// Allocation uses `ISV_TOTAL_DIM` so slots [23..31] (per-branch
// Q-support centres and half-widths) have backing storage. The
// network-facing path still consumes only the first `ISV_NETWORK_DIM`
// slots — the scratchpad tail is invisible to `w_isv_fc1`.
// Allocation uses `ISV_TOTAL_DIM` so all scratchpad slots [23..39) have
// backing storage. The network-facing path still consumes only the first
// `ISV_NETWORK_DIM` slots — the scratchpad tail is invisible to `w_isv_fc1`.
// Slots [37..39) hold the layout fingerprint (tail placement because [0..2)
// are actively written by `isv_signal_update`).
let (isv_signals_pinned, isv_signals_dev_ptr) = {
let num_bytes = ISV_TOTAL_DIM * std::mem::size_of::<f32>();
let mut host_ptr: *mut std::ffi::c_void = std::ptr::null_mut();
@@ -8131,6 +8212,14 @@ impl GpuDqnTrainer {
// IQL readiness→1 and the losing branch has near-zero
// advantage. Safety bound, not tuning.
*sig_ptr.add(IQL_BRANCH_SCALE_FLOOR_INDEX) = 0.1_f32;
// Layout fingerprint (ISV[37..39)). Compile-time structural hash of
// the slot layout; checkpoint load fails-fast on mismatch.
// Stored as a u64 split across two f32 lanes using raw bit-cast so
// no precision is lost. See LAYOUT_FINGERPRINT_CURRENT docs for why
// this is NOT a version number and why no migration path exists.
let fp = LAYOUT_FINGERPRINT_CURRENT;
*sig_ptr.add(ISV_LAYOUT_FINGERPRINT_LO_INDEX) = f32::from_bits(fp as u32);
*sig_ptr.add(ISV_LAYOUT_FINGERPRINT_HI_INDEX) = f32::from_bits((fp >> 32) as u32);
}
(host_ptr as *mut f32, dev_ptr_out)
};
@@ -8750,7 +8839,7 @@ impl GpuDqnTrainer {
// D1/N1: clone Arc<CudaStream> before it is moved into the struct literal.
let stream_for_snapshots = Arc::clone(&stream);
Ok(Self {
let trainer = Self {
config,
stream,
ptrs,
@@ -9306,7 +9395,12 @@ impl GpuDqnTrainer {
last_distill_active: false,
last_meta_q_pred: 0.5,
q_sample_history: std::collections::VecDeque::with_capacity(64),
})
};
// Self-check: verify the fingerprint we just wrote is readable and matches
// the compile-time constant. This is a construction-time invariant check;
// any restore path that loads persisted ISV state must also call this.
trainer.check_layout_fingerprint()?;
Ok(trainer)
}
/// Reference to the trainer's forked CudaStream.
@@ -9793,6 +9887,39 @@ impl GpuDqnTrainer {
unsafe { *self.isv_signals_pinned.add(index) = value; }
}
/// Read the ISV layout fingerprint from pinned memory and compare to the
/// compile-time `LAYOUT_FINGERPRINT_CURRENT`.
///
/// Returns `Err` on mismatch. NEVER returns `Ok` on mismatch — that would
/// permit silent backward compat, which this design forbids. Treat a null
/// ISV pointer as fingerprint `0`, which never matches any real fingerprint.
///
/// Called at construction (after writing) to self-verify. Must also be called
/// by any restore path that loads persisted ISV state from disk.
pub fn check_layout_fingerprint(&self) -> Result<(), MLError> {
let (lo_bits, hi_bits) = if self.isv_signals_pinned.is_null() {
(0u32, 0u32)
} else if ISV_TOTAL_DIM < ISV_LAYOUT_FINGERPRINT_HI_INDEX + 1 {
(0u32, 0u32)
} else {
unsafe {
let lo = f32::to_bits(*self.isv_signals_pinned.add(ISV_LAYOUT_FINGERPRINT_LO_INDEX));
let hi = f32::to_bits(*self.isv_signals_pinned.add(ISV_LAYOUT_FINGERPRINT_HI_INDEX));
(lo, hi)
}
};
let fp_stored: u64 = (lo_bits as u64) | ((hi_bits as u64) << 32);
if fp_stored != LAYOUT_FINGERPRINT_CURRENT {
return Err(MLError::ModelError(format!(
"ISV layout fingerprint mismatch: stored 0x{:016x}, current code 0x{:016x}. \
Checkpoint layout does not match current code - retrain required. \
(This is a structural fingerprint, not a version number. There is no migration path.)",
fp_stored, LAYOUT_FINGERPRINT_CURRENT
)));
}
Ok(())
}
/// Read atom utilization from q_readback_pinned[6] (one-step lag, written by
/// reduce_current_q_stats). Returns 0.0 if pointer is null.
pub fn read_atom_utilization(&self) -> f32 {

View File

@@ -14,7 +14,7 @@ pub enum ResetCategory {
SoftReset { decay_bars: u32 },
/// Survives across folds (learned weights).
TrainingPersist,
/// Version-stable across runs; touched only by explicit migration.
/// Structurally fixed across runs; fail-fast on mismatch. No migration path exists.
SchemaContract,
}
@@ -142,9 +142,9 @@ impl StateResetRegistry {
},
// ───── Schema-contract state ────────────────────────────────
RegistryEntry {
name: "ISV_SCHEMA_VERSION",
name: "ISV_LAYOUT_FINGERPRINT",
category: ResetCategory::SchemaContract,
description: "ISV[0] — slot-layout schema version (migrated only)",
description: "ISV[37..39) — compile-time structural hash of slot layout; fail-fast only, no migration path",
},
RegistryEntry {
name: "isv_iql_branch_scale_floor",
@@ -199,7 +199,7 @@ mod tests {
assert_eq!(r.category("kelly_stats"), Some(ResetCategory::FoldReset));
assert_eq!(r.category("adaptive_gamma"), Some(ResetCategory::SoftReset { decay_bars: 500 }));
assert_eq!(r.category("network_params"), Some(ResetCategory::TrainingPersist));
assert_eq!(r.category("ISV_SCHEMA_VERSION"), Some(ResetCategory::SchemaContract));
assert_eq!(r.category("ISV_LAYOUT_FINGERPRINT"), Some(ResetCategory::SchemaContract));
}
#[test]

View File

@@ -2,18 +2,42 @@
**Source of truth for ISV bus slot allocations.** Every slot has a named constant in `gpu_dqn_trainer.rs`. This doc is the cross-reference.
**Current `ISV_TOTAL_DIM` as of this plan's start:** 37. Post-full DQN v2 rollout: 72.
**Design: layout fingerprint, not schema version.** ISV[37..39) carries a
compile-time structural hash of the slot layout. Checkpoint load is
fail-fast; no migration functions exist. Backward compat is structurally
unwritable (there is no ordered version space to pair migrations against).
See spec §4.A.2.
**Tail placement rationale:** The fingerprint occupies the tail (`[37..39)`) rather
than the head because `isv_signals[0]` and `isv_signals[1]` are actively written
by `isv_signal_update` (Q-drift EMA and gradient-norm EMA respectively). Inserting
at the head would displace those live signals and require shifting every upstream
literal in `experience_kernels.cu`.
**Current `ISV_TOTAL_DIM`:** 39. Post-full DQN v2 rollout: 72.
| Index | Name constant | Type | Producer | Consumers | Reset-category | Notes |
|---|---|---|---|---|---|---|
| [0..2) | `ISV_LAYOUT_FINGERPRINT_{LO,HI}_INDEX` | u64 (as 2× f32) | construct | load | SchemaContract | Compile-time structural hash of slot layout; fail-fast only, no migration path — **not** a version number. See spec §4.A.2. |
| [1..12) | (pre-existing) | f32 | varies | varies | varies | Legacy — audit during A.2 |
| [0] | `SLOT_0_Q_DRIFT` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | `isv_signals[0]` = (q_mean q_ema) / max(|q_ema|, 0.1) |
| [1] | `SLOT_1_GRAD_NORM_EMA` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | EMA of sqrt(grad_norm²) |
| [2] | `SLOT_2_TD_ERR_EMA` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | EMA of TD-error scalar |
| [3] | `SLOT_3_ENS_VAR_EMA` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | C51 Q-distribution variance EMA (batch-mean atom-spread) |
| [4] | `SLOT_4_ENS_VAR_VEL` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | Delta of slot 3 (variance velocity) |
| [5] | `SLOT_5_REWARD_EMA` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | EMA of per-batch mean reward |
| [6] | `SLOT_6_ATOM_UTIL_EMA` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | EMA of atom utilization fraction |
| [7] | `SLOT_7_LOSS_EMA` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | EMA of total training loss |
| [8] | `SLOT_8_ADX_EMA` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | Batch-mean ADX EMA (regime velocity indicator) |
| [9] | `SLOT_9_REGIME_DISAGREE` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | |norm(ADX) norm(CUSUM)| disagreement signal |
| [10] | `SLOT_10_REGIME_VEL_EMA` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | EMA of regime transition velocity (ADX + CUSUM delta) |
| [11] | `SLOT_11_REGIME_STABILITY` (seed only) | f32 | isv_signal_update | ISV encoder | FoldReset | 1 sigmoid(5 × regime_vel); high = stable regime |
| [12] | `LEARNING_HEALTH_INDEX` | f32 | isv_signal_update | many | FoldReset | health score ∈ [0, 1] |
| [13..17) | `Q_MAG_MEAN_*_INDEX` | f32 | q_mag_means_reduce | c51 kernels | FoldReset | Quarter/Half/Full mag Q-mean EMAs + |Q| ref |
| [17..22) | `Q_DIR_MEAN_*_INDEX` | f32 | q_dir_means_reduce | c51 kernels | FoldReset | Short/Hold/Long/Flat dir Q-mean EMAs + |Q| ref |
| [13..17) | `Q_MAG_MEAN_*_INDEX` | f32 | q_mag_means_reduce | c51 kernels | FoldReset | Quarter/Half/Full mag Q-mean EMAs + \|Q\| ref |
| [17..22) | `Q_DIR_MEAN_*_INDEX` | f32 | q_dir_means_reduce | c51 kernels | FoldReset | Short/Hold/Long/Flat dir Q-mean EMAs + \|Q\| ref |
| [22] | `SHARPE_EMA_INDEX` | f32 | training_loop host | isv_signal_update | FoldReset | Training Sharpe EMA |
| [23..31) | `V_{CENTER,HALF}_{DIR,MAG,ORD,URG}_INDEX` | f32 | update_eval_v_range | adaptive_atoms, warm_start | FoldReset | Per-branch Q-support |
| [31..35) | `GRAD_NORM_TARGET_*_INDEX` | f32 | grad_balance_isv_update | branch_grad_rescale | SoftReset(decay_bars=500) | Per-branch grad-norm target |
| [35] | `GRAD_SCALE_LIMIT_INDEX` | f32 | grad_balance_isv_update | branch_grad_rescale | SoftReset(decay_bars=500) | Scale clamp limit |
| [36] | `IQL_BRANCH_SCALE_FLOOR_INDEX` | f32 | construct (static) | iql_per_branch_advantage | SchemaContract | Per-sample branch_scales floor; safety bound |
| [37..72) | (reserved for DQN v2) | | | | | Allocated incrementally by Plans 2-5 |
| [37] | `ISV_LAYOUT_FINGERPRINT_LO_INDEX` | u32 bits (in f32) | construct | check_layout_fingerprint | SchemaContract | Low 32 bits of u64 FNV-1a structural hash. Fail-fast on mismatch — NOT a version number, no migration path. |
| [38] | `ISV_LAYOUT_FINGERPRINT_HI_INDEX` | u32 bits (in f32) | construct | check_layout_fingerprint | SchemaContract | High 32 bits of u64 FNV-1a structural hash. |
| [39..72) | (reserved for DQN v2) | | | | | Allocated incrementally by Plans 2-5 |

View File

@@ -105,8 +105,23 @@ check_no_todo_fixme() {
fi
}
# DQN v2 Invariant (spec §4.A.2): no ISV migration functions allowed.
# The layout fingerprint is fail-fast only; no migration path exists.
check_no_isv_migrations() {
local bad=$(git diff --cached -U0 -- crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs \
crates/ml/src/trainers/dqn/trainer/ 2>/dev/null | \
grep -E '^\+.*fn\s+(migrate|upgrade)_isv' || true)
if [ -n "$bad" ]; then
echo "❌ Spec §4.A.2 violation: ISV migration functions are forbidden"
echo " (layout fingerprint is fail-fast only; no migration path exists)"
echo "$bad" | sed 's/^/ /'
return 1
fi
}
check_audit_doc_updates || exit 1
check_no_todo_fixme || exit 1
check_no_isv_migrations || exit 1
echo "✅ All pre-commit checks passed!"
echo ""