fix(sp21): Return v3.1 — drop short-rollout guard for volume bars (atomic)
Single-file fix to `compute_epoch_financials`: remove the `if n_returns_f >= bars_per_year` short-rollout fallback that left v2 semantics in place for sub-year rollouts. For Foxhunt's volume bars, `bars_per_day ≈ 34_496` → `bars_per_year ≈ 8.69M`, while a training epoch produces `n_returns ≈ 4.10M`. The guard fired on every production epoch, so the v3 CAGR fix was a no-op. Diagnosis chain: - v3 commit (2937da889) merged the n_returns >= bars_per_year guard - Smoke v4 (commit62b5a50e8, workflow train-frv8x) epoch 1 showed Return=+2.963e2% — bit-identical to v1's pre-fix output - Hypothesis 1 (cache poisoning): ruled out — ensure-binary log shows "Cache MISS: compiling binaries for 62b5a50e8" and ml crate was recompiled fresh - Hypothesis 2 (different commit): ruled out — workflow params confirm commit-sha =62b5a50e8= current HEAD - Hypothesis 3 (bars_per_year mismatch): confirmed — v4 log emits "Bars per day (from data): 34496" which makes bars_per_year > n_returns and triggers the v2 fallback inside the v3 branch Fix: unconditional CAGR. The log-space clamp [-23, +20] bounds the display in all edge cases (tests with tiny n_returns extrapolate aggressively; the clamp caps at exp(20) - 1 ≈ +4.85e8%). Expected v5 epoch 1 Return: ~+1.770e3% (was +2.963e2% under v4). The new value is the *actual annualized* projection: 1377% over a 0.47-year rollout. Overfit cycles cap at +4.85e8% (was +e19%). Tests: - cargo test -p ml --lib financials → 7/7 - Sign-only assertions in test cases (all > 0.0) — no regressions Files changed: - crates/ml/src/trainers/dqn/financials.rs: 1 conditional removed, comment block updated with v3 → v3.1 history - docs/dqn-wire-up-audit.md: diagnosis + fix entry for 2026-05-11 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -80,13 +80,15 @@ pub(crate) fn compute_epoch_financials(
|
||||
// bars): annualized = exp(45.93 × 19_656 / 4_096_000) − 1 ≈ 24.7% —
|
||||
// a reasonable, interpretable number.
|
||||
//
|
||||
// History (do not regress to either earlier form):
|
||||
// History (do not regress to any earlier form):
|
||||
// v1 (pre-2026-04): mean_per_bar × bars_per_year (arithmetic
|
||||
// annualization) — produced -51,234% on negative per-bar means.
|
||||
// v2 (2026-04..2026-05-11): log_growth.exp() − 1 (total compounded,
|
||||
// no annualization) — produced +e19% on long rollouts.
|
||||
// v3 (this commit, 2026-05-11): annualized compounded — bounded by
|
||||
// construction and operationally meaningful.
|
||||
// v3 (2026-05-11): annualized compounded for n_returns >= bars_per_year,
|
||||
// fallback to v2 below it — but the fallback subsumed all production
|
||||
// runs on volume bars (bars_per_year ≈ 8.7M > n_returns ≈ 4.1M).
|
||||
// v3.1 (2026-05-11): unconditional CAGR — the clamp handles edge cases.
|
||||
//
|
||||
// Lower bound: -100% (log_growth.exp() − 1 ≥ -1 when log_growth ≥
|
||||
// log(1e-10) ≈ -23 per the per-bar clamp). Upper bound: practically
|
||||
@@ -112,20 +114,25 @@ pub(crate) fn compute_epoch_financials(
|
||||
// annualization factor exceeds 1, which would extrapolate a
|
||||
// brief positive run to absurd annualized magnitudes (a 5-bar
|
||||
// test with 1.5% gain would annualize to e8% on 1-min bars).
|
||||
// For sub-year rollouts, report TOTAL compounded growth
|
||||
// instead — same `log_growth.exp() − 1` as the v2 formula,
|
||||
// bounded by the per-bar floor to ±100% per the inflation
|
||||
// audit's WR=46% honest meter discipline.
|
||||
// **v3.1 (2026-05-11)**: dropped the `n_returns >= bars_per_year`
|
||||
// short-rollout guard. The guard fell back to raw `log_growth`
|
||||
// (= v2 formula) whenever the rollout was shorter than a trading
|
||||
// year. For Foxhunt's volume bars the data extracts
|
||||
// `bars_per_day ≈ 34_496` → `bars_per_year ≈ 8.7M`, but a single
|
||||
// training epoch produces `n_returns ≈ 4.1M` step returns. Result:
|
||||
// every production epoch took the v2 branch and the v3 fix was a
|
||||
// no-op — smoke v4 epoch 1 (commit 62b5a50e8) reproduced v1's
|
||||
// `Return=+2.963e2%` exactly.
|
||||
//
|
||||
// Unconditional CAGR is the correct invariant: "what annualized
|
||||
// return is this policy producing?" works for any rollout length.
|
||||
// Tests (very small n_returns) get extrapolated (a 5-bar +1.5%
|
||||
// test extrapolates to e8% nominally) — the log-space clamp
|
||||
// `[-23, +20]` bounds the display magnitude, and the test cases
|
||||
// below assert ordering / sign, not exact magnitudes.
|
||||
let n_returns_f = n_returns as f64;
|
||||
if log_growth.is_finite() && n_returns_f > 0.0 && bars_per_year > 0.0 {
|
||||
let log_scaled = if n_returns_f >= bars_per_year {
|
||||
// Production rollout (typically ~4M bars vs ~20K bars/year)
|
||||
// — annualize for CAGR semantics.
|
||||
log_growth * bars_per_year / n_returns_f
|
||||
} else {
|
||||
// Short rollout (tests, warmup) — report total compounded.
|
||||
log_growth
|
||||
};
|
||||
let log_scaled = log_growth * bars_per_year / n_returns_f;
|
||||
// Sanity bounds: lower at -23 (≈ -100% by construction
|
||||
// from per-bar `(1+r).max(1e-10)` floor), upper at +20
|
||||
// (≈ exp(20) − 1 ≈ +4.85e8% — beyond this magnitude the
|
||||
|
||||
@@ -15349,3 +15349,84 @@ validate the full SP21 cascade end-to-end. Watch:
|
||||
(Phase 8 ISV-driven gains)
|
||||
- evaluate_baseline reproduces val_Sharpe / val_PF within 5%
|
||||
tolerance on the same checkpoint+window.
|
||||
|
||||
## 2026-05-11 — Return v3.1: drop short-rollout guard, unconditional CAGR (atomic)
|
||||
|
||||
### Scope (atomic single commit)
|
||||
|
||||
One-file fix to `compute_epoch_financials`: drop the
|
||||
`if n_returns_f >= bars_per_year` short-rollout fallback. v3
|
||||
left the v2 formula in place for sub-year rollouts, but for
|
||||
Foxhunt's **volume bars** the data extracts
|
||||
`bars_per_day ≈ 34_496` → `bars_per_year ≈ 8.7M`, while a
|
||||
single training epoch produces `n_returns ≈ 4.1M`. Result: the
|
||||
v3 guard fired for *every* production epoch, so the v3 fix was
|
||||
a no-op. Smoke v4 (commit 62b5a50e8, workflow train-frv8x)
|
||||
epoch 1 reproduced v1's `Return=+2.963e2%` bit-identically.
|
||||
|
||||
### Diagnosis
|
||||
|
||||
- v1 epoch 1 stats: Sharpe=70.40, log_growth ≈ ln(3.963) ≈ 1.377
|
||||
- v3 expectation (assuming 5-min bars, bars_per_year=98_280):
|
||||
`exp(1.377 × 98280/4.1M) - 1 ≈ +3.36% (= +3.359e0%)`
|
||||
- v4 actual: `Return=+2.963e2%` (same as v1)
|
||||
- Volume bar diagnostic in log:
|
||||
`Bars per day (from data): 34496 (17835714 total bars)` →
|
||||
`bars_per_year = 34_496 × 252 = 8.69M > n_returns = 4.1M` →
|
||||
guard fires → v2 fallback path
|
||||
|
||||
### Fix
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let log_scaled = if n_returns_f >= bars_per_year {
|
||||
log_growth * bars_per_year / n_returns_f
|
||||
} else {
|
||||
log_growth
|
||||
};
|
||||
```
|
||||
with unconditional CAGR:
|
||||
```rust
|
||||
let log_scaled = log_growth * bars_per_year / n_returns_f;
|
||||
```
|
||||
|
||||
The log-space clamp `[-23, +20]` still bounds the display.
|
||||
Tests (very small `n_returns`) extrapolate aggressively (a 5-bar
|
||||
test produces clamp-bounded ~e8% display) but all test
|
||||
assertions are sign-only (`> 0.0`), so no test regresses.
|
||||
|
||||
### Expected v5 epoch 1 Return
|
||||
|
||||
- log_growth ≈ 1.377 (same training-side stats as v1)
|
||||
- log_scaled = 1.377 × 8.69M / 4.1M = 2.92
|
||||
- exp(2.92) - 1 ≈ 17.7 → `Return=+1.770e3%` (≈ 1770% annualized)
|
||||
|
||||
Higher baseline than 296% because the metric now actually
|
||||
extrapolates a 0.47-year rollout to a per-year CAGR. The clamp
|
||||
caps overfit overflow at ~`+4.852e8%` (was `+8.73e19%` under v2).
|
||||
|
||||
### Files changed
|
||||
|
||||
- `crates/ml/src/trainers/dqn/financials.rs`: drop guard, update
|
||||
comment block (v3 → v3.1 history line)
|
||||
- `docs/dqn-wire-up-audit.md`: this entry
|
||||
|
||||
### Pearls + invariants honoured
|
||||
|
||||
- **feedback_no_quickfixes**: minimal scope, restores v3's original
|
||||
CAGR intent without changing other semantics
|
||||
- **feedback_no_legacy_aliases**: removes the now-misnamed
|
||||
"short-rollout guard" rather than renaming or wrapping it
|
||||
- **pearl_audit_first_caught_5_of_7_hypotheses_were_wrong**:
|
||||
hypothesis-driven diagnosis (cache poisoning suspected →
|
||||
ruled out by `git show` + workflow commit-sha check →
|
||||
actual root cause: volume-bar `bars_per_year` mismatch)
|
||||
|
||||
### Verification
|
||||
|
||||
```
|
||||
cargo test -p ml --lib financials # 7/7
|
||||
```
|
||||
|
||||
No CUDA build needed for this fix (financials.rs is CPU-only).
|
||||
Production validation deferred to v5 smoke dispatch.
|
||||
|
||||
Reference in New Issue
Block a user