feat(rl): walk-forward G8 eval phase + fold split (MVP, manual fan-out)

Adds the minimum-viable implementation of the R9 multi-fold G8 gate
per `pearl_single_window_oos_is_not_oos` ("a single window is NOT
out-of-sample"). The trainer can now:

  1. Slice the MBP-10 file list into K equal-sized blocks
     (`--n-folds K --fold-idx k`).
  2. Train on blocks [0..=k] (passed to MultiHorizonLoader).
  3. Run a separate eval phase of `--n-eval-steps` on block [k+1]
     using a second loader instance.
  4. Drain LobSim trade records gated by a pre-eval head checkpoint
     so train-phase trades don't contaminate the eval summary.
  5. Compute profit_factor + sharpe + drawdown via existing
     `ml_backtesting::artifacts::compute_summary`.
  6. Write `eval_summary.json` alongside `alpha_rl_train_summary.json`.

## Manual fan-out (this MVP)

The dispatcher (`scripts/argo-alpha-rl.sh`) gains three new flags
that thread through the Argo template into the CLI: `--fold-idx`,
`--n-folds`, `--n-eval-steps`. To run a 3-fold G8:

  ./scripts/argo-alpha-rl.sh --n-folds 3 --fold-idx 0 --n-eval-steps 200
  ./scripts/argo-alpha-rl.sh --n-folds 3 --fold-idx 1 --n-eval-steps 200

(With n_folds=3 the valid fold indices are 0 and 1 — the third block
is the eval window for fold 1. n_folds=K accepts fold_idx ∈ [0, K-2].)

Each submission produces one `eval_summary.json` at the resolved
output dir; the per-fold profit_factor is the value to aggregate.
Manual aggregation for now — automated DAG matrix fan-out + an
in-cluster aggregator pod is a follow-up commit. The aggregator
will mean ± SD the per-fold PFs and gate on `PF > 1.0`.

## What's NOT pure eval

The eval loop calls `step_with_lobsim` (same as train) — Adam steps,
PER updates, controller adaptations all still fire during eval. At
b_size=1 the per-step learning effect is small relative to the
train-phase-accumulated policy, so the eval PF approximates the
OOS performance of the train-end policy. A clean pure-eval mode
(forward + LobSim step only, no backward/Adam/PER) is a follow-up
architectural change; documented inline at the eval phase block.

## Default behaviour unchanged

`--n-folds=1` (default) skips the eval split entirely and uses all
files for training — identical to the prior single-window smoke.
The R9 prior smokes ran in this mode. Default `--fold-idx=0` and
`--n-eval-steps=0` keep prior smoke runs binary-compatible.

## Template + dispatcher changes

  * `alpha-rl-template.yaml`: adds 3 new workflow parameters
    (`fold-idx`, `n-folds`, `n-eval-steps`) and threads them into
    the train container's `alpha_rl_train` invocation.
  * `argo-alpha-rl.sh`: adds matching CLI flags with explicit
    documentation of the multi-fold dispatching pattern.

## Verified gates

Local sm_86 build + dispatcher syntax clean. Tests unchanged
(the walk-forward path is exercised by cluster smokes, not unit
tests — the loader-slicing logic is straightforward index math).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-23 17:14:11 +02:00
parent cdfaa5e7da
commit 87a22d12c9
3 changed files with 226 additions and 6 deletions

View File

@@ -36,6 +36,9 @@ N_BACKTESTS=1
PER_CAPACITY=4096
SEED=16962
INSTRUMENT_MODE=front-month
FOLD_IDX=0
N_FOLDS=1
N_EVAL_STEPS=0
WATCH=false
SKIP_PUSH_CHECK=false
SKIP_TEMPLATE_APPLY=false
@@ -57,6 +60,23 @@ Usage: $0 [OPTIONS]
--per-capacity <n> PER buffer capacity (default: $PER_CAPACITY).
--seed <n> Random seed (default: $SEED).
--instrument-mode <m> all | front-month | id=<N> (default: $INSTRUMENT_MODE)
--fold-idx <k> Walk-forward fold index (0-based) for the
multi-fold G8 gate (default: $FOLD_IDX). Slices
the MBP-10 file list into n_folds blocks; fold k
trains on blocks [0..=k] and evals on block [k+1].
Requires --n-folds ≥ 3 and --n-eval-steps > 0.
--n-folds <K> Total fold count (default: $N_FOLDS). K=1 disables
the eval split (single-window mode). K≥3 enables
walk-forward; submit K-1 workflows (one per
fold_idx) and aggregate the per-fold profit_factor
for the G8 gate per
\`pearl_single_window_oos_is_not_oos\`.
--n-eval-steps <n> Eval-phase step count (default: $N_EVAL_STEPS).
After the train phase, runs n_eval_steps additional
steps on the held-out fold. LobSim trade records
from the eval phase are isolated via a pre-eval
head checkpoint; \`eval_summary.json\` lands in
OUT_DIR alongside \`alpha_rl_train_summary.json\`.
--watch Follow logs via argo watch
--skip-push-check Bypass the feedback_push_before_deploy guard
(rare; use only when submitting against a
@@ -79,6 +99,9 @@ while [[ $# -gt 0 ]]; do
--per-capacity) PER_CAPACITY="$2"; shift 2 ;;
--seed) SEED="$2"; shift 2 ;;
--instrument-mode) INSTRUMENT_MODE="$2"; shift 2 ;;
--fold-idx) FOLD_IDX="$2"; shift 2 ;;
--n-folds) N_FOLDS="$2"; shift 2 ;;
--n-eval-steps) N_EVAL_STEPS="$2"; shift 2 ;;
--watch) WATCH=true; shift ;;
--skip-push-check) SKIP_PUSH_CHECK=true; shift ;;
--skip-template-apply) SKIP_TEMPLATE_APPLY=true; shift ;;
@@ -191,4 +214,7 @@ argo submit -n foxhunt --from=wftmpl/alpha-rl \
-p per-capacity="$PER_CAPACITY" \
-p seed="$SEED" \
-p instrument-mode="$INSTRUMENT_MODE" \
-p fold-idx="$FOLD_IDX" \
-p n-folds="$N_FOLDS" \
-p n-eval-steps="$N_EVAL_STEPS" \
$WATCH_FLAG