From 5fb0089cd28270d29bd27cd8defe7d44342f6d2c Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Tue, 7 Apr 2026 13:43:04 +0200 Subject: [PATCH] =?UTF-8?q?spec:=20f32=20backtest=20pipeline=20=E2=80=94?= =?UTF-8?q?=20eliminate=20bf16=20financial=20arithmetic?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit bf16 has $32 resolution at $5K equity. Transaction costs ($12.50), tick PnL ($12.50), and cumulative returns are all below resolution. All previous hyperopt evaluations used garbage metrics. Co-Authored-By: Claude Opus 4.6 (1M context) --- ...2026-04-07-f32-backtest-pipeline-design.md | 83 +++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-07-f32-backtest-pipeline-design.md diff --git a/docs/superpowers/specs/2026-04-07-f32-backtest-pipeline-design.md b/docs/superpowers/specs/2026-04-07-f32-backtest-pipeline-design.md new file mode 100644 index 000000000..d2d8ea45c --- /dev/null +++ b/docs/superpowers/specs/2026-04-07-f32-backtest-pipeline-design.md @@ -0,0 +1,83 @@ +# f32 Backtest Pipeline — Eliminate bf16 Financial Arithmetic + +## Goal + +Promote the entire GPU backtest pipeline from bf16 to f32. bf16 has $32 resolution at $5,000 equity — transaction costs ($12.50), tick-level PnL ($12.50-$50), and cumulative returns are all below bf16 resolution. This produces garbage Sharpe scores (quantized to ~0.004) and incorrect drawdown (100% from rounding errors). All previous hyperopt evaluations were computed with broken bf16 metrics. + +## Scope + +Three CUDA kernels + one Rust host file. Zero bf16 in the backtest pipeline after this change. + +## Changes + +### 1. `backtest_env_kernel.cu` — both `backtest_env_step` and `backtest_env_step_batch` + +| Buffer | Before | After | +|---|---|---| +| `prices` | `__nv_bfloat16*` | `float*` | +| `portfolio_state` | `__nv_bfloat16*` | `float*` | +| `step_rewards` | `__nv_bfloat16*` | `float*` | +| `step_returns` | `__nv_bfloat16*` | `float*` | + +- Remove ALL `bf16()`, `__bfloat162float()`, `bf16_zero()`, `bf16_one()`, `bf16_fabs()`, `bf16_fmax()` calls +- Pure `float` arithmetic throughout — no type conversions +- `PORTFOLIO_STATE_SIZE` stays 8, just `float` instead of `__nv_bfloat16` +- Shared memory in `backtest_env_step`: `shmem_pf[256 * PORTFOLIO_STATE_SIZE]` → `float` (doubles shmem but still <48KB) +- `done_flags` stays `int*`, `actions_history` stays `int*` + +### 2. `backtest_metrics_kernel.cu` — `compute_backtest_metrics` + +| Buffer | Before | After | +|---|---|---| +| `step_returns` | `__nv_bfloat16*` | `float*` | +| `portfolio_state` | `__nv_bfloat16*` | `float*` | +| `metrics_out` | `__nv_bfloat16*` | `float*` | +| `annualization_factor` | `__nv_bfloat16` | `float` | + +- All shared memory arrays: `__nv_bfloat16` → `float` +- All accumulators: already pattern `local_sum += r` but `r` was bf16 — now `float` +- All `bf16_*()` helper calls → native float ops (`fabsf`, `fmaxf`, `sqrtf`) +- Shared memory doubles in size (2B → 4B per element) — verify within 48KB limit + +### 3. `gpu_backtest_evaluator.rs` — Rust host + +| Field | Before | After | +|---|---|---| +| `prices_buf` | `CudaSlice` | `CudaSlice` | +| `portfolio_buf` | `CudaSlice` | `CudaSlice` | +| `step_rewards_buf` | `CudaSlice` | `CudaSlice` | +| `step_returns_buf` | `CudaSlice` | `CudaSlice` | + +- `init_portfolio_state()` → remove `clone_htod_f32_to_bf16`, use `stream.clone_htod::` directly +- `reset_evaluation_state()` → same +- Price upload in `new()` → remove bf16 conversion, upload f32 directly +- `annualization_factor` parameter: `half::bf16::from_f32(...)` → direct `f32` +- `metrics_out` parsing: already reads as f32 from `metrics_host: Vec` — just change allocation +- Remove `clone_htod_f32_to_bf16` calls for prices, portfolio, step_returns +- `launch_gather` kernel: reads from `features_buf` (bf16 market features) → keep bf16 for features (not financial data), OR promote to f32 too + +### 4. States/features buffer (decision needed) + +`features_buf` and `states_buf` contain market features (z-scored indicators). These are used as neural network INPUT — bf16 is fine for forward inference. Keep bf16 for features. + +`prices_buf` is SEPARATE from features — it contains raw [open, high, low, close] for PnL computation. This goes to f32. + +## What stays bf16 + +- `features_buf` / `states_buf` — market features for cuBLAS forward (tensor core input) +- Training pipeline (forward/backward GEMMs) — intentional bf16 for performance +- Experience collector rewards — individual scalars, not accumulated + +## Testing + +- Smoke tests: verify Sharpe is no longer quantized to bf16 grid +- Compare: old bf16 Sharpe vs new f32 Sharpe on same data +- Verify: transaction costs are now non-zero in the backtest +- H100: deploy and check val_loss varies meaningfully across epochs + +## Impact + +- All previous hyperopt results are INVALID (bf16 Sharpe was noise) +- Hyperopt must be restarted from scratch after this fix +- Expected: val_loss will show meaningful variation, TPE can actually optimize +- Expected: max_dd will be realistic (not 100% from rounding errors)