# DQN 45 Factored Action Restoration via Branching DQN

**Date**: 2026-03-11
**Status**: Approved

## Summary

Enable `use_branching: true` by default in DQNConfig, restoring 45 factored actions
(5 exposure x 3 order_type x 3 urgency) via the existing BranchingDuelingQNetwork.
Fix wiring gaps so all action selection paths route through branching when enabled.

## Motivation

- DQN was reduced from 45 to 5 actions during debugging; this was never intended as permanent
- At tick-level HFT, order type and urgency have learnable microstructure signal
  (queue priority, adverse selection, fill probability)
- The deterministic OrderRouter discards information the agent could exploit
- Branching DQN outputs 11 Q-values (5+3+3) instead of 45, avoiding curse of dimensionality
- PPO already validates the 45-action space works on same data

## Changes

### 1. DQNConfig defaults
- `use_branching: true` (was `false`)
- `num_actions: 5` stays (exposure branch size, not total actions)

### 2. Action selection paths (dqn.rs)
- `select_action()` — already handles branching (lines 1623-1656) ✓
- `select_action_with_confidence()` — add branching path (currently OrderRouter only)
- `select_action_inference()` — add branching path (currently OrderRouter only)

### 3. Agent action selection (agent.rs)
- `select_action_factored()` — route through branching instead of OrderRouter

### 4. Experience encoding
- `Experience.action: u8` — already sufficient (0-44 fits in u8)
- `FactoredAction::to_index()` / `from_index()` already exist

### 5. Count bonus (count_bonus.rs)
- Track per-branch counts [5][3][3] instead of flat [5]

### 6. CUDA header (common_device_functions.cuh)
- `DQN_NUM_ACTIONS=5` stays (exposure branch)
- Add `DQN_ORDER_ACTIONS=3`, `DQN_URGENCY_ACTIONS=3`, `DQN_TOTAL_ACTIONS=45`
- Experience kernel already has `decompose_actions_batch_gpu()` for branching

### 7. Tests
- Update ~8 test sites with hardcoded `num_actions = 5`

## Non-changes
- BranchingDuelingQNetwork — already complete with C51, NoisyNet, tensor core alignment
- BranchingConfig — already defaults to [5, 3, 3]
- Training loss path — already has branching loss with per-branch decomposition
- OrderRouter — kept for backward compat and as constraint enforcer, no longer primary decision path
