feat(sp21): T2.2 Phase 7 — E8 curriculum_concentration → PER alpha boost (atomic)

Wires E8's curriculum_weights distribution into the per_update_pa
alpha boost composition via a new scalar signal:
`compute_curriculum_concentration(weights) = 1 - entropy/log(n)`.
Same pattern as Phases 5+6 (E6 winner_concentration + E7
hindsight_magnitude); all three feed the same boost_delta sum.

Plan revision (mirrors Phase 5+6 pattern):
- Original plan: "wire E8 (curriculum weights) → segment sampling
  weights." Literal interpretation requires new curriculum-segment
  abstraction in PER (segments don't exist — flat ring buffer
  today).
- Resolution: signal-driven from the SHAPE of the weights
  distribution (entropy concentration), not the CONTENT (per-segment
  weighted sampling). Feeds existing per_update_pa kernel; no new
  segment-sampling kernel.
- True per-segment PER sampling deferred to Phase 7.5 (mirrors
  Phase 6.5 deferral for E7's literal injection consumer).

Signal semantics (compute_curriculum_concentration):
- Input: Vec<f32> of E8's per-segment weights (sum=1).
- Output: 1 - entropy/log(n) ∈ [0, 1].
  - 0.0 = uniform (segments equally hard)
  - 1.0 = one segment dominates (concentrated difficulty)
- Single-segment trivially → 1.0
- Cold start (empty input) → 0.0 sentinel
- Zero-weight segments skipped (p log p → 0)

ISV slot allocation:
- CURRICULUM_CONCENTRATION_INDEX = 527
- ISV_TOTAL_DIM 527 → 528 (bus extension)
- Layout fingerprint adds SLOT_527 entry

Kernel change (4 lines, no ABI churn):
- per_update_pa reads isv_signals[527] (already-existing arg from
  Phase 5+6)
- curriculum_term = clamp(curric_conc, 0, 1) × 0.1 ∈ [0, 0.1]
- boost_delta upper bound 0.4 → 0.5 to accommodate new term
- Cold-start short-circuit predicate extended to all 3 signals

Producer wireup:
- enrichment.rs: new helper compute_curriculum_concentration;
  EnrichmentResult.curriculum_concentration field;
  run_enrichments populates it; log line extended
- training_loop.rs: post-enrichment block writes ISV[527]

Files changed:
- crates/ml/src/cuda_pipeline/sp21_isv_slots.rs: +1 slot const
- crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs: ISV_TOTAL_DIM
  bump + fingerprint
- crates/ml-dqn/src/per_kernels.cu: 4-line boost composition
  extension
- crates/ml/src/trainers/dqn/trainer/enrichment.rs: helper + field
- crates/ml/src/trainers/dqn/trainer/training_loop.rs: write ISV
- docs/dqn-wire-up-audit.md: 2026-05-11 audit entry

Verification (passing):
- cargo check -p ml --tests --features cuda: 0 errors
- cargo test -p ml --lib sp21_isv_slots: 3/3
- sp20_aggregate_inputs_test: 12/12
- sp20_phase1_4_wireup_test: 2/2
- sp20_emas_compute_test: 4/4
- sp20_controllers_compute_test: 7/7
- sp21_per_trade_predicted_q_test: 3/3
Total: 34 tests, 0 failures.

After this commit (Phase 8 + 6.5 + 7.5):
- Phase 8: signal-drive remaining controller GAINS in enrichment
- Phase 6.5: true E7 hindsight synthetic injection (deferred)
- Phase 7.5: true E8 per-segment PER sampling (deferred)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-10 23:55:46 +02:00
parent a6087a0b23
commit 34d19955ff
6 changed files with 249 additions and 19 deletions

View File

@@ -76,26 +76,33 @@ extern "C" __global__ void per_update_pa(
int ival = __float_as_int(new_prio);
atomicMax((int*)batch_max, ival);
/* SP21 Phase 5+6: compose alpha_boost from E6 (winner concentration)
* and E7 (hindsight magnitude). Both signals at sentinel 0.0 →
* boost_delta = 0 → alpha_eff = alpha (cold-start no-op). Otherwise
* the additive delta is bounded to keep alpha_eff in a sane range:
* - winner_term: (concentration - 1.0) clipped to [0, 2.0], scaled
* by 0.1 → [0, 0.2]. Concentration > 1.0 means top-decile drives
* more profit per trade than average; we want PER to focus more
* on hard cases (alpha up).
/* SP21 Phase 5+6+7: compose alpha_boost from three enrichment signals:
* - E6 winner_concentration (ISV[525]): top-decile pnl / all-mean pnl
* - E7 hindsight_magnitude (ISV[526]): mean |counterfactual_pnl|
* - E8 curriculum_concentration (ISV[527]): 1 - entropy/log(N) ∈ [0,1]
*
* All signals at sentinel 0.0 → boost_delta = 0 → alpha_eff = alpha
* (cold-start no-op). Otherwise additive delta bounded to keep
* alpha_eff in a sane range:
* - winner_term: (concentration - 1.0) clipped to [0, 2.0],
* scaled by 0.1 → [0, 0.2]. Concentration > 1.0 means top-decile
* drives more profit per trade than average.
* - hindsight_term: magnitude clipped to [0, 0.02] (typical
* counterfactual P&L scale ≈ 1-2%), scaled by 10.0 → [0, 0.2].
* Higher magnitude = more missed opportunities = need more
* PER focus.
* - Total delta clipped at 0.4 so alpha_eff ≤ alpha + 0.4. */
* Higher = more missed opportunities = need more PER focus.
* - curriculum_term: concentration ∈ [0, 1] (already normalised),
* scaled by 0.1 → [0, 0.1]. Higher = uneven mastery across
* curriculum segments = more focus on hard transitions.
* - Total delta clipped at 0.5 so alpha_eff ≤ alpha + 0.5. */
float winner_conc = (isv_signals != NULL) ? isv_signals[525] : 0.0f;
float hindsight_mag = (isv_signals != NULL) ? isv_signals[526] : 0.0f;
float curric_conc = (isv_signals != NULL) ? isv_signals[527] : 0.0f;
float boost_delta = 0.0f;
if (winner_conc > 1e-6f || hindsight_mag > 1e-6f) {
float winner_term = fmaxf(0.0f, fminf(2.0f, winner_conc - 1.0f)) * 0.1f;
float hindsight_term = fminf(0.02f, hindsight_mag) * 10.0f;
boost_delta = fminf(0.4f, winner_term + hindsight_term);
if (winner_conc > 1e-6f || hindsight_mag > 1e-6f || curric_conc > 1e-6f) {
float winner_term = fmaxf(0.0f, fminf(2.0f, winner_conc - 1.0f)) * 0.1f;
float hindsight_term = fminf(0.02f, hindsight_mag) * 10.0f;
float curriculum_term = fmaxf(0.0f, fminf(1.0f, curric_conc)) * 0.1f;
boost_delta = fminf(0.5f, winner_term + hindsight_term + curriculum_term);
}
float alpha_eff = alpha + boost_delta;

View File

@@ -2622,7 +2622,7 @@ const ISV_NETWORK_DIM: usize = 23;
/// (and the C-side mirrors in `state_layout.cuh`). Bump this constant in the
/// SAME commit that adds the new range, and update `layout_fingerprint_seed`
/// to register the new slot names.
pub(crate) const ISV_TOTAL_DIM: usize = 527; // SP21 Phase 5+6 adds 2 slots [525..527) for WINNER_CONCENTRATION + HINDSIGHT_MAGNITUDE (PER alpha scaling); Phase 4 added 4 slots [521..525) for BRANCH_LR_SCALE_*; Phase 3 added 1 slot [520..521) for Q_CORRECTION
pub(crate) const ISV_TOTAL_DIM: usize = 528; // SP21 Phase 7 adds 1 slot [527..528) for CURRICULUM_CONCENTRATION (PER alpha scaling, composed with WINNER_CONCENTRATION + HINDSIGHT_MAGNITUDE); Phase 5+6 added 2 slots [525..527); Phase 4 added 4 slots [521..525) for BRANCH_LR_SCALE_*; Phase 3 added 1 slot [520..521) for Q_CORRECTION
/// Legacy alias preserved for call sites that haven't been audited for the
/// network-vs-total split. New code should pick `ISV_NETWORK_DIM` (for weight
/// tensor sizing) or `ISV_TOTAL_DIM` (for the broadcast bus buffer).
@@ -3755,7 +3755,8 @@ const fn layout_fingerprint_seed() -> &'static [u8] {
SLOT_524_BRANCH_LR_SCALE_URGENCY=524;\
SLOT_525_WINNER_CONCENTRATION=525;\
SLOT_526_HINDSIGHT_MAGNITUDE=526;\
ISV_TOTAL_DIM=527;\
SLOT_527_CURRICULUM_CONCENTRATION=527;\
ISV_TOTAL_DIM=528;\
SP19_PRODUCER_HARDCODED_HORIZON_BLEND=sp19_path_b;\
SP14_C_AUX_TRUNK_CONTROL_PLANE=sp14_c_phase_1;\
ALPHA_MACHINERY_DELETED=sp14_c_phase_1;\

View File

@@ -17,6 +17,7 @@
//! | 524 | `BRANCH_LR_SCALE_URGENCY_INDEX` | `enrichment::compute_branch_lr_scale[3]` | `dqn_adam_update_kernel` per-branch sub-launch (urgency tensors 29-32) |
//! | 525 | `WINNER_CONCENTRATION_INDEX` | host-side aggregation of E6 winner P&L + total P&L | `per_update_pa` kernel (PER alpha scaling) |
//! | 526 | `HINDSIGHT_MAGNITUDE_INDEX` | host-side aggregation of E7 counterfactual P&L magnitudes | `per_update_pa` kernel (PER alpha scaling) |
//! | 527 | `CURRICULUM_CONCENTRATION_INDEX` | host-side entropy of E8 curriculum-weights distribution | `per_update_pa` kernel (PER alpha scaling) |
//!
//! ## Bus extension
//!
@@ -127,6 +128,30 @@ pub const WINNER_CONCENTRATION_INDEX: usize = 525;
/// unchanged when both signals are at sentinel.
pub const HINDSIGHT_MAGNITUDE_INDEX: usize = 526;
/// SP21 T2.2 Phase 7 (2026-05-11) — curriculum-weight distribution
/// concentration from E8 (`enrichment::compute_curriculum_weights`).
///
/// Producer: host-side after each validation backtest pass. E8
/// emits per-segment sampling weights (typically `n_segments=8`,
/// normalised to sum=1, where each weight is the inverse-Sharpe
/// of that segment's trades — harder segments get larger weights).
/// `compute_curriculum_concentration` reduces that vector to a
/// scalar `1 - entropy/log(n_segments)` ∈ `[0, 1]`:
///
/// - 0.0 = perfectly uniform weights (all segments equally hard)
/// - 1.0 = one segment dominates (all difficulty concentrated)
///
/// Consumer: `per_update_pa` kernel (PER priority alpha scaling)
/// reads slot 527 device-side and composes it into the existing
/// `boost_delta` alongside `WINNER_CONCENTRATION_INDEX` (E6) and
/// `HINDSIGHT_MAGNITUDE_INDEX` (E7). High curriculum concentration
/// signals the policy has uneven mastery across the data distribution
/// → lift PER alpha to focus more on hard transitions.
///
/// Cold-start: 0.0 sentinel before first val pass — kernel
/// short-circuits to no-op. Per `pearl_first_observation_bootstrap`.
pub const CURRICULUM_CONCENTRATION_INDEX: usize = 527;
/// Convenience accessor: branch index (0..4) → ISV slot index. Used
/// by the launcher's per-branch sub-launch loop to keep the
/// branch-index ↔ ISV-slot mapping in one place.
@@ -146,7 +171,7 @@ pub fn branch_lr_scale_index(branch: usize) -> usize {
pub const SP21_SLOT_START: usize = 520;
/// One past the last SP21 slot.
pub const SP21_SLOT_END: usize = 527;
pub const SP21_SLOT_END: usize = 528;
#[cfg(test)]
mod tests {
@@ -178,6 +203,7 @@ mod tests {
BRANCH_LR_SCALE_URGENCY_INDEX,
WINNER_CONCENTRATION_INDEX,
HINDSIGHT_MAGNITUDE_INDEX,
CURRICULUM_CONCENTRATION_INDEX,
];
let mut sorted = slots.to_vec();
sorted.sort();

View File

@@ -71,6 +71,14 @@ pub(crate) struct EnrichmentResult {
pub hindsight_magnitude: f32,
/// E8: curriculum weights (one per segment).
pub curriculum_weights: Vec<f32>,
/// SP21 T2.2 Phase 7 (2026-05-11) — `1 - entropy/log(n_segments)`
/// ∈ `[0, 1]`. 0 = curriculum weights are uniform (segments equally
/// hard); 1 = one segment dominates (highly concentrated difficulty).
/// Cold start: empty curriculum returns 0.0 sentinel. Consumed by
/// `per_update_pa` via ISV[CURRICULUM_CONCENTRATION_INDEX=527] for
/// per-sample alpha boost (composed multiplicatively with
/// `winner_concentration` + `hindsight_magnitude`).
pub curriculum_concentration: f32,
}
/// Persistent state across epochs for the enrichment module.
@@ -302,6 +310,43 @@ fn compute_hindsight_magnitude(hindsight: &[HindsightExperience]) -> f32 {
/ hindsight.len() as f32
}
/// SP21 T2.2 Phase 7 (2026-05-11) — curriculum-weight concentration.
///
/// Reduces E8's `Vec<f32>` per-segment weights to a single scalar:
/// `1 - entropy/log(n)` where `entropy = -Σ w_i × log(w_i)`. The
/// weights are pre-normalised (sum=1) by `compute_curriculum_weights`,
/// so entropy ∈ `[0, log(n)]` and the normalised concentration falls
/// in `[0, 1]`:
///
/// - 0.0: uniform weights — all curriculum segments equally hard.
/// - 1.0: one segment dominates — all difficulty in a single
/// time-window slice of the data.
///
/// Used by `per_update_pa` for PER alpha boost composition (Phase 7).
/// Cold start: empty input → 0.0 sentinel (no-op per
/// `pearl_first_observation_bootstrap`). Zero weights are skipped
/// in the entropy sum (`p log p → 0` as `p → 0`).
fn compute_curriculum_concentration(weights: &[f32]) -> f32 {
if weights.is_empty() {
return 0.0;
}
let n = weights.len() as f32;
if n < 2.0 {
// Single-segment curriculum is trivially "concentrated"; return 1.0.
return 1.0;
}
let entropy: f32 = weights
.iter()
.filter(|&&w| w > 1e-12)
.map(|&w| -w * w.ln())
.sum();
let max_entropy = n.ln();
if max_entropy <= 1e-12 {
return 0.0;
}
(1.0 - entropy / max_entropy).clamp(0.0, 1.0)
}
/// E6: Winner distillation — bar indices of top-10% trades by P&L.
fn compute_winner_indices(trades: &[EvalTrade]) -> Vec<usize> {
if trades.is_empty() {
@@ -427,11 +472,16 @@ pub(crate) fn run_enrichments(
// E8: Curriculum weights
let curriculum_weights = compute_curriculum_weights(eval_trades, state.n_segments);
// SP21 T2.2 Phase 7 (2026-05-11) — scalar concentration signal
// derived from the curriculum-weight entropy. Feeds PER alpha boost
// alongside E6/E7 signals (see WINNER_CONCENTRATION_INDEX,
// HINDSIGHT_MAGNITUDE_INDEX in cuda_pipeline::sp21_isv_slots).
let curriculum_concentration = compute_curriculum_concentration(&curriculum_weights);
info!(
"Enrichment cycle {} complete: q_corr={:.4}, eps={:.4}, gamma={:?}, \
branch_lr={:?}, agree_thr={:.4}, winners={}, win_conc={:.4}, \
hindsight={}, hindsight_mag={:.4}, curriculum_segs={}",
hindsight={}, hindsight_mag={:.4}, curriculum_segs={}, curric_conc={:.4}",
state.cycle_count,
q_correction,
epsilon,
@@ -443,6 +493,7 @@ pub(crate) fn run_enrichments(
hindsight.len(),
hindsight_magnitude,
curriculum_weights.len(),
curriculum_concentration,
);
EnrichmentResult {
@@ -456,5 +507,6 @@ pub(crate) fn run_enrichments(
hindsight,
hindsight_magnitude,
curriculum_weights,
curriculum_concentration,
}
}

View File

@@ -1601,6 +1601,7 @@ impl DQNTrainer {
use crate::cuda_pipeline::sp21_isv_slots::{
Q_CORRECTION_INDEX, branch_lr_scale_index,
WINNER_CONCENTRATION_INDEX, HINDSIGHT_MAGNITUDE_INDEX,
CURRICULUM_CONCENTRATION_INDEX,
};
fused.trainer().write_isv_signal_at(
Q_CORRECTION_INDEX,
@@ -1631,6 +1632,15 @@ impl DQNTrainer {
HINDSIGHT_MAGNITUDE_INDEX,
result.hindsight_magnitude,
);
// SP21 T2.2 Phase 7 (2026-05-11): curriculum
// concentration feeds the same per_update_pa
// alpha boost composition (alongside E6 winner +
// E7 hindsight signals). Cold-start sentinel
// 0.0 → kernel short-circuits to no-op.
fused.trainer().write_isv_signal_at(
CURRICULUM_CONCENTRATION_INDEX,
result.curriculum_concentration,
);
}
}
}

View File

@@ -14371,3 +14371,137 @@ T2.2 multi-phase scope continues with Phases 7-8 + 6.5:
E2 2.0/0.5/-0.5 epsilon thresholds; etc.).
- Phase 6.5 (deferred): true hindsight synthetic experience injection
via val state retention + `per_insert_pa` extension.
## 2026-05-11 — SP21 T2.2 Phase 7: E8 curriculum_weights → curriculum-concentration PER alpha boost
### Scope (atomic single commit)
Wires E8 (`enrichment::compute_curriculum_weights`) output into
the same `per_update_pa` alpha boost composition that already
consumes E6 (winner_concentration, Phase 5) and E7 (hindsight_
magnitude, Phase 6). The single line added to the kernel reads
`isv_signals[527]` and folds it into `boost_delta` per the same
pattern.
### Plan revision rationale
Original plan: Phase 7 = "wire E8 (curriculum weights) → segment
sampling weights." The literal interpretation requires an entirely
new "curriculum segment" abstraction in PER (segments don't exist
today — PER is a flat ring buffer with priority-proportional
sampling). That's larger scope than what the signal-driven
interpretation provides:
- The per-segment weights have CONTENT (which segment is hardest)
and SHAPE (how concentrated the difficulty distribution is).
- True per-segment PER would consume the CONTENT (segment → buffer
slot mapping, weighted sampling per segment).
- The SHAPE is a single scalar (`1 entropy/log(n)`) that captures
the policy's mastery distribution.
Per `feedback_no_deferrals_for_complementary_fixes`, the SHAPE
signal feeds the same consumer (`per_update_pa`) as Phases 5+6's
E6/E7 signals. Combining them keeps the kernel's boost-composition
logic in one place and saves a kernel ABI churn vs a separate
segment-sampling kernel. The true segment-sampling consumer of
E8's CONTENT is documented as a future Phase 7.5 follow-up (mirrors
the Phase 6.5 deferral for E7's literal injection consumer).
### Signal semantics
**`compute_curriculum_concentration(weights)`**:
- Input: `Vec<f32>` of per-segment weights (E8's
`compute_curriculum_weights` output, normalised so sum=1).
- Output: `1 - entropy/log(n_segments)` clamped to `[0, 1]`.
- `0.0`: uniform weights — segments equally hard.
- `1.0`: one segment dominates — concentrated difficulty.
- Cold start: empty input → 0.0 sentinel.
- Single-segment (`n=1`) → 1.0 (trivially concentrated).
- Zero weights skipped in entropy sum (`p log p → 0` as `p → 0`).
The signal answers: "is the policy's mastery EVEN across the
data distribution, or is it stuck on a particular slice?" Higher
concentration → more reason to boost PER alpha (focus sampling
on hard transitions).
### ISV slot allocation
| Index | Name | Producer | Consumer |
|-------|------|----------|----------|
| 527 | `CURRICULUM_CONCENTRATION_INDEX` | `enrichment::compute_curriculum_concentration` (host, post-val) | `per_update_pa` (PER alpha boost) |
`ISV_TOTAL_DIM` 527 → 528. Layout fingerprint adds
`SLOT_527_CURRICULUM_CONCENTRATION`.
### Kernel change (minimal — 4 lines)
`per_update_pa` in `crates/ml-dqn/src/per_kernels.cu`: the
`boost_delta` composition gains one new term. The kernel was
already extended in Phase 5+6 to take `isv_signals` and short-
circuit on cold-start; Phase 7 adds one more `isv_signals[527]`
read and one more clamped additive term:
```c
float curric_conc = (isv_signals != NULL) ? isv_signals[527] : 0.0f;
...
float curriculum_term = fmaxf(0.0f, fminf(1.0f, curric_conc)) * 0.1f;
boost_delta = fminf(0.5f, winner_term + hindsight_term + curriculum_term);
```
`boost_delta`'s upper bound lifted 0.4 → 0.5 to accommodate the new
term's `[0, 0.1]` range. The cold-start short-circuit predicate
extended to `(winner_conc > 1e-6 || hindsight_mag > 1e-6 ||
curric_conc > 1e-6)`.
NO kernel ABI change — `isv_signals` already added in Phase 5+6.
### Files changed
| File | Status | Purpose |
|------|--------|---------|
| `crates/ml/src/cuda_pipeline/sp21_isv_slots.rs` | +1 slot | `CURRICULUM_CONCENTRATION_INDEX = 527`; `SP21_SLOT_END` 527 → 528; uniqueness test extended |
| `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` | ISV bump | `ISV_TOTAL_DIM` 527 → 528; fingerprint adds 1 SLOT entry |
| `crates/ml-dqn/src/per_kernels.cu` | 1 new term | `boost_delta` composition reads `isv_signals[527]`; upper bound 0.4 → 0.5; cold-start predicate extended |
| `crates/ml/src/trainers/dqn/trainer/enrichment.rs` | +1 helper + 1 field | `compute_curriculum_concentration` helper; `EnrichmentResult.curriculum_concentration: f32`; `run_enrichments` populates it; log line extended |
| `crates/ml/src/trainers/dqn/trainer/training_loop.rs` | Producer | After enrichment block, writes ISV[527] from `result.curriculum_concentration` |
| `docs/dqn-wire-up-audit.md` | This entry | 2026-05-11 audit log |
### Pearls + invariants honoured
- `feedback_no_partial_refactor` — ISV slot + bus bump + kernel
body + producer wireup migrate atomically.
- `feedback_no_stubs``result.curriculum_concentration`
consumed for real by `per_update_pa`.
- `pearl_first_observation_bootstrap` — sentinel 0.0 short-circuits
kernel to no-op until E8 emits a real value.
- `pearl_controller_anchors_isv_driven` — the 0.1 scale factor on
`curriculum_term` is a baseline magnitude bound (Invariant 1
carve-out), not an anchor; SIGNAL drives the value via ISV[527].
- `feedback_isv_for_adaptive_bounds` — adaptive PER alpha sharpness
IS adaptive bound; lives in ISV.
### Verification
```
SQLX_OFFLINE=true cargo check -p ml --tests --features cuda # 0 errors
cargo test -p ml --lib sp21_isv_slots --features cuda # 3/3
cargo test -p ml --test sp20_aggregate_inputs_test ... # 12/12
cargo test -p ml --test sp20_phase1_4_wireup_test ... # 2/2
cargo test -p ml --test sp20_emas_compute_test ... # 4/4
cargo test -p ml --test sp20_controllers_compute_test ... # 7/7
cargo test -p ml --test sp21_per_trade_predicted_q_test ... # 3/3
```
Total: 34 tests, 0 failures. Behavioral gate: HEALTH_DIAG
enrichment log line now includes `curric_conc=…` on every val
pass. PER alpha lift sums three signals after warmup.
### After this commit lands
T2.2 multi-phase scope continues with Phases 8 + 6.5 + 7.5:
- Phase 8: signal-drive remaining controller GAINS (E5 0.9/1.1
steps; E2 2.0/0.5/-0.5 epsilon thresholds; E3 0.85/0.98 gamma
clamps; E4 0.5/2.0 LR scale clamps).
- Phase 6.5 (deferred): true E7 hindsight synthetic injection.
- Phase 7.5 (deferred): true E8 per-segment PER sampling via
bar-index → segment mapping on replay tuples.