34586dad68
Rename binary alpha_compose_backtest → alpha_baseline and remove the boolean flags whose features are now mandatory: --c51 (always C51 distributional Q) --temporal (always Mamba2 temporal encoder) --isv-continual (controller always fires per eval episode) --regime-scale (vol-EMA regime defense always on) --pruned-actions (FALSIFIED 2026-05-15 per pearl_action_pruning_falsified) Every dependent code path was stripped, not just gated: - Linear-Q kernels (lq_fwd, lq_grad, munch_kernel) and their cubin loads are gone — C51 is the only Q-network. - Single-env push_kernel / h_store_kernel loads removed; the backtest has been batched-parallel-env since T14 and only the _batched variants are called here. (The smoke binary still uses single-env variants because one env per episode is its job.) - Dead transition buffers removed: states_dev, next_states_dev, actions_dev, rewards_dev, dones_dev, q_current_dev, q_next_dev, target_dev, single_state_dev, single_q_dev, probs_current_dev, probs_next_dev, m_dev, single_probs_dev, single-env state_pinned, action_pinned, window_tensor, h_enriched_buf_dev. - Dead constants and helpers: PRUNED_ACTIONS, N_WEIGHTS, N_BIASES, epsilon_greedy, epsilon_greedy_gated. End-to-end verification on the existing Q1 fxcache (rebuild was OOM locally; full multi-quarter validation is the next phase): cost=0.0000 best τ=0.250 Sharpe_ann=+36.83 win=0.984 trades/ep=83.3 cost=0.0625 best τ=0.250 Sharpe_ann=+38.53 win=0.996 trades/ep=83.2 cost=0.1250 best τ=0.250 Sharpe_ann=+38.37 win=0.994 trades/ep=83.3 cost=0.2500 best τ=0.250 Sharpe_ann=+34.24 win=0.990 trades/ep=85.6 cost=0.5000 best τ=0.250 Sharpe_ann=+31.83 win=0.946 trades/ep=84.8 Numbers track the prior T16-flag config (within stochastic noise), confirming the conditional-stripping was a pure simplification — no behavioral change, just a smaller, honester binary. Also updated: - scripts/alpha_pipeline.sh — A/B conditions collapse to fixed-cost vs cost-randomized training (the only opt-in left). - scripts/walk_forward_cv.sh — drop legacy flags, pass --window-k only. - crates/ml/src/env/loaders.rs — module doc-comment updated. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
80 lines
2.7 KiB
Bash
Executable File
80 lines
2.7 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Walk-forward CV for the Phase E.4.A.T10 compose backtest.
|
|
#
|
|
# Slides a fixed-size window across the fxcache. Each fold trains DQN
|
|
# from scratch on the front of the window and evaluates on the back.
|
|
# Fold C is the cleanest stacker-OOS test (eval bars 1.62M..1.9M lie
|
|
# entirely past the stacker training cut at 1.57M).
|
|
#
|
|
# Outputs per-fold JSON to /tmp/cv_fold_*.json. Run from repo root.
|
|
|
|
set -euo pipefail
|
|
|
|
FXCACHE="${FXCACHE:-/home/jgrusewski/Work/foxhunt/test_data/feature-cache/9297017b6db6795f75e57be8aefb03e45e6427513f42c08e22521e58fa025d5e.fxcache}"
|
|
ALPHA="${ALPHA:-config/ml/alpha_logits_cache.bin}"
|
|
FILL="${FILL:-config/ml/alpha_fill_coeffs.json}"
|
|
BIN="${BIN:-./target/release/examples/alpha_baseline}"
|
|
|
|
WINDOW=700000
|
|
TRAIN_FRAC=0.6 # 420K train, 280K eval per fold
|
|
|
|
declare -a FOLDS=(
|
|
"A:0"
|
|
"B:600000"
|
|
"C:1200000"
|
|
)
|
|
|
|
for fold_spec in "${FOLDS[@]}"; do
|
|
name="${fold_spec%%:*}"
|
|
offset="${fold_spec##*:}"
|
|
out="/tmp/cv_fold_${name}.json"
|
|
echo "===================================================================="
|
|
echo "Fold ${name}: offset=${offset} window=${WINDOW} train_frac=${TRAIN_FRAC}"
|
|
echo "===================================================================="
|
|
SQLX_OFFLINE=true RUST_LOG=info "$BIN" \
|
|
--fxcache-path "$FXCACHE" \
|
|
--alpha-cache "$ALPHA" \
|
|
--fill-coeffs "$FILL" \
|
|
--data-start-offset "$offset" \
|
|
--max-snapshots "$WINDOW" \
|
|
--train-frac "$TRAIN_FRAC" \
|
|
--window-k 16 \
|
|
--out-path "$out"
|
|
done
|
|
|
|
echo
|
|
echo "===================================================================="
|
|
echo "Walk-forward summary (best Sharpe_ann per cost, per fold)"
|
|
echo "===================================================================="
|
|
python3 - <<'PY'
|
|
import json, statistics
|
|
folds = ["A", "B", "C"]
|
|
rows = []
|
|
for f in folds:
|
|
with open(f"/tmp/cv_fold_{f}.json") as fh:
|
|
j = json.load(fh)
|
|
by_cost = {}
|
|
for b in j["bins"]:
|
|
c = b["cost"]
|
|
if c not in by_cost or b["sharpe_annualised"] > by_cost[c][1]:
|
|
by_cost[c] = (b["threshold"], b["sharpe_annualised"], b["win_rate"], b["avg_n_trades"])
|
|
rows.append((f, by_cost))
|
|
|
|
costs = sorted(rows[0][1].keys())
|
|
print(f"{'cost':>8} " + " ".join(f"fold-{f}" for f, _ in rows))
|
|
for c in costs:
|
|
cells = []
|
|
for _, by_cost in rows:
|
|
tau, sa, wr, tp = by_cost[c]
|
|
cells.append(f"τ={tau:.2f} S={sa:+6.2f} ({wr*100:.0f}%/{tp:.0f})")
|
|
print(f"{c:>8.4f} " + " ".join(cells))
|
|
|
|
print()
|
|
print("Per-cost mean Sharpe_ann ± stddev across folds:")
|
|
for c in costs:
|
|
sas = [by_cost[c][1] for _, by_cost in rows]
|
|
m = statistics.mean(sas)
|
|
sd = statistics.stdev(sas) if len(sas) > 1 else 0.0
|
|
print(f" cost={c:.4f} mean Sharpe_ann = {m:+7.2f} ± {sd:5.2f}")
|
|
PY
|