spec: f32 backtest pipeline — eliminate bf16 financial arithmetic

bf16 has $32 resolution at $5K equity. Transaction costs ($12.50),
tick PnL ($12.50), and cumulative returns are all below resolution.
All previous hyperopt evaluations used garbage metrics.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-07 13:43:04 +02:00
parent 26ffa7ad72
commit 5fb0089cd2

View File

@@ -0,0 +1,83 @@
# f32 Backtest Pipeline — Eliminate bf16 Financial Arithmetic
## Goal
Promote the entire GPU backtest pipeline from bf16 to f32. bf16 has $32 resolution at $5,000 equity — transaction costs ($12.50), tick-level PnL ($12.50-$50), and cumulative returns are all below bf16 resolution. This produces garbage Sharpe scores (quantized to ~0.004) and incorrect drawdown (100% from rounding errors). All previous hyperopt evaluations were computed with broken bf16 metrics.
## Scope
Three CUDA kernels + one Rust host file. Zero bf16 in the backtest pipeline after this change.
## Changes
### 1. `backtest_env_kernel.cu` — both `backtest_env_step` and `backtest_env_step_batch`
| Buffer | Before | After |
|---|---|---|
| `prices` | `__nv_bfloat16*` | `float*` |
| `portfolio_state` | `__nv_bfloat16*` | `float*` |
| `step_rewards` | `__nv_bfloat16*` | `float*` |
| `step_returns` | `__nv_bfloat16*` | `float*` |
- Remove ALL `bf16()`, `__bfloat162float()`, `bf16_zero()`, `bf16_one()`, `bf16_fabs()`, `bf16_fmax()` calls
- Pure `float` arithmetic throughout — no type conversions
- `PORTFOLIO_STATE_SIZE` stays 8, just `float` instead of `__nv_bfloat16`
- Shared memory in `backtest_env_step`: `shmem_pf[256 * PORTFOLIO_STATE_SIZE]``float` (doubles shmem but still <48KB)
- `done_flags` stays `int*`, `actions_history` stays `int*`
### 2. `backtest_metrics_kernel.cu` — `compute_backtest_metrics`
| Buffer | Before | After |
|---|---|---|
| `step_returns` | `__nv_bfloat16*` | `float*` |
| `portfolio_state` | `__nv_bfloat16*` | `float*` |
| `metrics_out` | `__nv_bfloat16*` | `float*` |
| `annualization_factor` | `__nv_bfloat16` | `float` |
- All shared memory arrays: `__nv_bfloat16``float`
- All accumulators: already pattern `local_sum += r` but `r` was bf16 — now `float`
- All `bf16_*()` helper calls → native float ops (`fabsf`, `fmaxf`, `sqrtf`)
- Shared memory doubles in size (2B → 4B per element) — verify within 48KB limit
### 3. `gpu_backtest_evaluator.rs` — Rust host
| Field | Before | After |
|---|---|---|
| `prices_buf` | `CudaSlice<half::bf16>` | `CudaSlice<f32>` |
| `portfolio_buf` | `CudaSlice<half::bf16>` | `CudaSlice<f32>` |
| `step_rewards_buf` | `CudaSlice<half::bf16>` | `CudaSlice<f32>` |
| `step_returns_buf` | `CudaSlice<half::bf16>` | `CudaSlice<f32>` |
- `init_portfolio_state()` → remove `clone_htod_f32_to_bf16`, use `stream.clone_htod::<f32>` directly
- `reset_evaluation_state()` → same
- Price upload in `new()` → remove bf16 conversion, upload f32 directly
- `annualization_factor` parameter: `half::bf16::from_f32(...)` → direct `f32`
- `metrics_out` parsing: already reads as f32 from `metrics_host: Vec<f32>` — just change allocation
- Remove `clone_htod_f32_to_bf16` calls for prices, portfolio, step_returns
- `launch_gather` kernel: reads from `features_buf` (bf16 market features) → keep bf16 for features (not financial data), OR promote to f32 too
### 4. States/features buffer (decision needed)
`features_buf` and `states_buf` contain market features (z-scored indicators). These are used as neural network INPUT — bf16 is fine for forward inference. Keep bf16 for features.
`prices_buf` is SEPARATE from features — it contains raw [open, high, low, close] for PnL computation. This goes to f32.
## What stays bf16
- `features_buf` / `states_buf` — market features for cuBLAS forward (tensor core input)
- Training pipeline (forward/backward GEMMs) — intentional bf16 for performance
- Experience collector rewards — individual scalars, not accumulated
## Testing
- Smoke tests: verify Sharpe is no longer quantized to bf16 grid
- Compare: old bf16 Sharpe vs new f32 Sharpe on same data
- Verify: transaction costs are now non-zero in the backtest
- H100: deploy and check val_loss varies meaningfully across epochs
## Impact
- All previous hyperopt results are INVALID (bf16 Sharpe was noise)
- Hyperopt must be restarted from scratch after this fix
- Expected: val_loss will show meaningful variation, TPE can actually optimize
- Expected: max_dd will be realistic (not 100% from rounding errors)