jgrusewski b43413e4d3 feat(sp18 v2 P1.T1+T2): pre-dispatch consumer-audit script + pre-commit hook
Phase 1 Tasks 1.1 + 1.2 of docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md.

Per the plan's audit-first design: surface every live reference to the
SP13/SP16 Hold-cost-scale chain (D-leg) and the TD(λ) `q_next = rewards`
self-bootstrap (B-leg) BEFORE atomic deletion. This catches the SP17-style
"missed quantile_q_select consumer" failure mode where a sweeping refactor
left a dangling reference that compiled but broke at runtime.

What lands

- `scripts/audit_sp18_consumers.sh` — 17-section grep that walks every
  consumer pattern from the plan's locked checklist (D-leg slot 380,
  461, [462..468) HCS_*, hold_cost_scale_update kernel, hold_rate_observer
  kernel retained chain, build.rs cubin manifest, state_layout.cuh
  mirror constants, state_reset_registry entries, plus B-leg
  td_lambda_kernel launch sites, q_next origin, rewards_out consumers,
  PER priority sites, replay buffer schema, c51_loss target-Q origin,
  target_params_buf consumers, PopArt slot 63 references). Three modes:
  default (full grep output), `--fingerprint` (per-section per-file hit
  count for diff-able snapshots), `--check` (diff fingerprint vs locked
  snapshot in docs/sp18-wireup-audit.md, exit 1 on drift).

- `docs/sp18-wireup-audit.md` — Phase 1 Task 1.1 outcome with two
  sections of import:
    1. The plan's locked checklist (8 entries the plan author explicitly
       identified as deletion targets).
    2. The 10 ADDITIONAL CONSUMERS surfaced by the audit (A1-A10) that
       the plan-author missed. Per the task input's halt-on-drift
       directive, Phase 1 Tasks 1.3-1.5 (atomic deletion) are HALTED
       pending human review of the expanded scope. The audit doc is the
       spec-amendment record.
    3. A locked fingerprint snapshot the pre-commit hook uses for drift
       detection on subsequent commits.
  B-leg verification confirms B-DD4 (no PER migration) + B-DD1 (target_
  params_buf reusable) + the single q_next bootstrap site at
  gpu_experience_collector.rs:4143.

- `scripts/pre-commit-hook.sh` — Invariant 7 list extended with the new
  audit doc; new `check_sp18_consumer_audit` step runs the audit script
  in `--check` mode whenever a commit touches the chain files
  (experience_kernels.cu, hold_*_kernel.cu, state_layout.cuh, sp1[3-8]_
  isv_slots.rs, gpu_dqn_trainer.rs, gpu_aux_trunk.rs, gpu_experience_
  collector.rs, state_reset_registry.rs, training_loop.rs, build.rs,
  sp1[3-8]_oracle_tests.rs). Drift triggers a hook failure with a
  pointer to the regeneration command. This is a generalisation of
  Invariant 7 and addresses Open Q-B from the spec (audit-as-pre-commit
  hook for SP-chain consumer drift).

Findings (HALT trigger)

The audit found 10 consumers NOT in the plan's locked 8-site checklist:
A1 — gpu_aux_trunk.rs:1240-1323 HoldCostScaleUpdateOps struct + impl
     (84 lines; the plan said launcher was in gpu_dqn_trainer but the
     real struct/impl lives in gpu_aux_trunk; trainer just has a thin
     forwarding method)
A2 — sp14_oracle_tests.rs:2174-2920 11 GPU oracle tests (~750 lines)
     directly exercising the deleted kernel via include_bytes!
     (sp16_phase2_hold_cost_scale_climbs_with_overrun + 10 others)
A3 — training_loop.rs:8971-9043 7 dispatch arms in reset_named_state
     (slot 461 + HCS_* slots 462-467); contract test
     every_fold_and_soft_reset_entry_has_dispatch_arm requires
     atomic deletion alongside registry entries
A4 — gpu_dqn_trainer.rs:23200-23216 constructor block writing
     HOLD_COST_BASE to slot 380 (RETAINED per DD7c, comment requires
     update)
A5 — gpu_dqn_trainer.rs:617, 2245-2256 doc-block prose
A6 — sp13_isv_slots.rs:75-77 HOLD_COST_CONTROLLER_GAIN/FLOOR/CEIL
     constants (HOLD_COST_BASE retained for A4)
A7 — gpu_experience_collector.rs:5579-5585 stale comment doc-ref
A8 — sp5_isv_slots.rs:325-327 comment doc-ref
A9 — state_reset_registry.rs:1138-1271 7 already-RETIRED entries
     promote to FULL DELETION
A10 — state_reset_registry.rs:2256-2308 lock_sp18_v2_pp4_retired_chain
      contract test asserts retired entries STILL EXIST; must be
      deleted/rewritten when entries are removed

None of these are architectural surprises — they're straight extensions
of the atomic-deletion scope. But per the task input's halt-on-drift
directive ("If your audit surfaces ANY additional consumer, halt and
report — do NOT proceed with deletion ad-hoc"), Phase 1 Tasks 1.3-1.5
HALT pending human sign-off on the expanded scope.

Path forward (next phase)

Either expand the atomic-deletion commit scope to cover all 18 sites
(8 plan + 10 audit-surfaced) — recommended per `feedback_no_partial_
refactor`; the audit doc serves as the spec-amendment record — or
human-review the audit doc and explicitly approve/reject each A* entry.

The audit doc + script + hook are the pre-condition for either path
and ship together as Tasks 1.1 + 1.2 of Phase 1. Tasks 1.3-1.6 await
human go-ahead.

Branch is at INTERIM STATE: NOT runnable for L40S smoke between this
commit and the post-Phase-1.6 close-out. The interim is purely-additive
(audit script + hook + doc); the actual deletion that creates the
"3 reward sites missing Hold cost" interim from the plan's Task 1.5
has NOT yet been performed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 02:23:57 +02:00

Foxhunt

Production HFT trading system in Rust.

Architecture

The workspace contains 32 crates organized as follows:

Core Libraries (16)

Crate Purpose
trading_engine Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing
risk VaR, Kelly, circuit breakers, kill switches, compliance
risk-data Risk data types and shared structures
trading-data Trading data types
ml DQN Rainbow, PPO, TFT, Mamba2, ensemble inference
ml-data ML data types and feature definitions
data Market data ingestion and storage
backtesting Replay engine, strategy tester
adaptive-strategy Ensemble execution, microstructure analysis
common Shared types, resilience, error handling
storage S3 and local model storage
model_loader Model serialization and loading
market-data Market data feed handlers
database PostgreSQL access layer (SQLx)
config Configuration management
tli CLI commands and tooling

Services (8)

Service Purpose
backtesting_service gRPC backtesting service
broker_gateway_service FIX routing, broker connectivity
trading_service Core trading operations
ml_training_service Model training orchestration
data_acquisition_service Market data acquisition
trading_agent_service Autonomous trading agents
api_gateway gRPC API gateway with auth
web-gateway Axum REST + WebSocket gateway

Frontend

web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.

Building

# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace

# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib

# Clippy
SQLX_OFFLINE=true cargo clippy --workspace

ML Models

Four production model architectures on Candle v0.9.1 with CUDA:

  • DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
  • PPO -- Proximal Policy Optimization with GAE and LSTM policies
  • TFT -- Temporal Fusion Transformer for multi-horizon forecasting
  • Mamba2 -- State space model for sequence prediction

Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.

Infrastructure

  • Git: Gitea at git.fxhnt.ai (Tailscale-only), Scaleway DEV1-S
  • Observability: OpenTelemetry OTLP (env OTEL_EXPORTER_OTLP_ENDPOINT)
  • Database: PostgreSQL with SQLx offline mode for CI

License

Proprietary. All rights reserved.

Description
No description provided
Readme 849 MiB
Languages
Rust 88.2%
Cuda 7.7%
Python 1.3%
Shell 1.1%
PLpgSQL 0.8%
Other 0.8%