spec(sp11): reframe curiosity as feature + add permanent floor

User correction: curiosity is the *fix* for the ep1-peak overfitting
pathology, not a hazard to defend against. Reframed §7 from "Risks"
to "Design notes" — curiosity bound is a signal-relative scale, not
a defensive cap.

Fix the contradiction this exposed in the formula: previous
`curiosity_pressure = stagnant_or_worse * curiosity_bound` went to
zero when improving, which would cancel the always-on exploration
the §7 narrative now relies on. Replace with permanent-floor pattern
per pearl_blend_formulas_must_have_permanent_floor:

  curiosity_floor    = 0.2 * curiosity_bound  (CURIOSITY_PERMANENT_FRACTION)
  curiosity_dynamic  = stagnant_or_worse * curiosity_bound
  curiosity_pressure = max(curiosity_dynamic, curiosity_floor)

Now curiosity is always ≥ 20% of bound (anti-overfitting baseline)
and rises toward the bound when stagnant (stagnation breaker).
Updated unit-test guidance to assert pressure > 0 even at z=+10.

CURIOSITY_PERMANENT_FRACTION=0.2 added to Invariant-1 fraction list.
Saboteur-rising-with-improvement reframed as adversarial-load feature
rather than over-stress risk.
This commit is contained in:
jgrusewski
2026-05-04 00:08:39 +02:00
parent a72a23e6b1
commit d49fbbe4e2

View File

@@ -136,8 +136,14 @@ for (int c = 0; c < 6; c++) {
float curiosity_bound = isv[PNL_REWARD_MAGNITUDE_EMA] * 0.3f; // 0.3 is Invariant-1 fraction
scratch[scratch_curiosity_bound] = curiosity_bound;
// Curiosity pressure: scales [0, curiosity_bound] by (1 - improving)
float curiosity_pressure = stagnant_or_worse * curiosity_bound;
// Curiosity pressure: ALWAYS-ON exploration signal scaled by stagnation.
// When improving: floor preserves baseline exploration (anti-overfitting).
// When stagnating: rises toward curiosity_bound to break out of fixed point.
// Permanent floor pattern per pearl_blend_formulas_must_have_permanent_floor —
// max() form, not blend, so curiosity never zeros even when policy is winning.
float curiosity_floor = CURIOSITY_PERMANENT_FRACTION * curiosity_bound; // 0.2 = Invariant-1 fraction
float curiosity_dynamic = stagnant_or_worse * curiosity_bound;
float curiosity_pressure = fmaxf(curiosity_dynamic, curiosity_floor);
scratch[scratch_curiosity_pressure] = curiosity_pressure;
// Saboteur intensity: continuous mapping of z → [SABOTEUR_MIN, SABOTEUR_MAX]
@@ -157,11 +163,13 @@ float adaptive_floor = fmaxf(WEIGHT_HARD_FLOOR, 0.5f * min_grad_ratio); // 0.5
scratch[scratch_weight_floor] = adaptive_floor;
```
The constants `0.3f` (curiosity fraction), `0.5f` (floor fraction), and
`0.1f` (engagement floor) are Invariant-1 fractions: they bound rates, not
regimes, and remain valid across all regimes. The thresholds themselves
(what counts as "improving", "stagnant", "regressing") are entirely
encoded in the Z-score and sigmoid — no hardcoded threshold survives.
The constants `0.3f` (curiosity fraction of PnL), `0.5f` (floor fraction
of min grad ratio), `0.2f` (CURIOSITY_PERMANENT_FRACTION — exploration
floor as fraction of curiosity_bound), and `0.1f` (engagement floor) are
Invariant-1 fractions: they bound rates, not regimes, and remain valid
across all regimes. The thresholds themselves (what counts as "improving",
"stagnant", "regressing") are entirely encoded in the Z-score and sigmoid
— no hardcoded threshold survives.
### 3.5 Consumer migrations
@@ -234,7 +242,7 @@ Plus modifications to existing kernels:
## 6. Tests
- Unit test: synthetic z=0 → curiosity at midpoint; z>0 → curiosity drops; z<0 → curiosity max
- Unit test: synthetic z=0 → curiosity at midpoint; z<0 → curiosity rises toward curiosity_bound; z>0 → curiosity drops to floor (0.2 × curiosity_bound), NOT zero — assert `curiosity_pressure > 0` even at z=+10
- Unit test: weight floor ≥ WEIGHT_HARD_FLOOR always
- Unit test: saboteur_intensity ∈ [SABOTEUR_MIN, SABOTEUR_MAX]
- Contract test: 20 new slots have FoldReset entries + dispatch arms
@@ -242,19 +250,34 @@ Plus modifications to existing kernels:
- L40S smoke: 5-epoch run shows weights drifting from initial sentinels,
curiosity oscillating with stagnation signal, saboteur_intensity reactive
## 7. Risks and open questions
## 7. Design notes and open questions
- **Risk 1**: Curiosity bonus could destabilize a working policy. Mitigated
by `CURIOSITY_BOUND_INDEX` = 0.3 × PnL EMA — curiosity is always at most
30% of PnL signal magnitude. Permanent floor pattern means it never
zeros out either, so we always have some exploration.
- **Risk 2**: Per-component weight reweighting could starve a component.
- **Curiosity is a feature, not a hazard**: The current pathology is that
the model converges to a fixed point at ep1 and then declines. Curiosity-
driven exploration is the corrective mechanism — we *want* the model to
keep exploring before settling. The `CURIOSITY_BOUND_INDEX = 0.3 × PnL
EMA` is not a defensive cap against curiosity; it's a signal-relative
scale that keeps the bonus magnitude commensurate with the PnL signal so
it doesn't drown out the primary objective. The permanent floor pattern
(per `pearl_blend_formulas_must_have_permanent_floor`) ensures curiosity
never fully zeros out — exploration pressure is maintained even after
convergence, which is exactly what addresses the ep1-peak overfitting.
In other words: more curiosity when stagnant (stagnation breaker), some
curiosity always (overfitting prevention).
- **Component starvation guard**: Per-component weight reweighting could in
principle starve a contributor. Mitigated by adaptive floor
`≥ 0.5 × min_grad_ratio` ensuring even the least-contributing component
keeps a non-trivial budget.
Mitigated by adaptive floor `≥ 0.5 × min_grad_ratio` ensuring even the
least-contributing component keeps a non-trivial budget.
- **Risk 3**: Saboteur intensity rising on improvement could over-stress the
policy. Mitigated by [0.5, 2.0] hard bound and engagement-rate self-
correction (per `pearl_engagement_rate_self_correction`).
- **Risk 4**: First-fold cold-start: with only 1-2 val emits available, the
- **Saboteur intensity adapts both directions**: When val sharpe is
improving, saboteur intensity rises to keep the policy under realistic
adversarial load (you don't want a model that only works when execution
is perfect). When stagnating, intensity drops so the policy gets a
cleaner gradient. The [0.5, 2.0] hard bound and engagement-rate self-
correction (per `pearl_engagement_rate_self_correction`) keep this
bounded.
- **First-fold cold-start**: With only 1-2 val emits available, the
Z-score is noisy. Mitigated by Pearl A sentinel-bootstrap (first
observation replaces sentinel directly) and Pearl D Wiener-α (smooth as
data accumulates).