foxhunt/database/migrations/NOTIFY_CHANNELS_REFERENCE.md

# PostgreSQL NOTIFY Channels Reference

**Wave 71 Agent 7 - Database Migration Implementation**

This document describes the PostgreSQL NOTIFY channels available for hot-reload configuration management in the Foxhunt HFT system.

---

## Available NOTIFY Channels

### 1. Configuration Change Channels

#### `config_changed_trading`
**Purpose:** Trading service configuration updates
**Trigger:** Updates to `config_settings` where category includes 'trading'
**Payload Format:**
```json
{
  "operation": "UPDATE",
  "table": "config_settings",
  "timestamp": 1696348234.567,
  "config_key": "risk.max_daily_loss",
  "old_value": "150000",
  "new_value": "160000"
}
```

**Listening Example:**
```sql
LISTEN config_changed_trading;
```

---

#### `config_changed_backtesting`
**Purpose:** Backtesting service configuration updates
**Trigger:** Updates to `config_settings` where category includes 'backtesting'
**Use Case:** Hot-reload backtesting parameters without service restart

---

#### `config_changed_ml_training`
**Purpose:** ML training service configuration updates
**Trigger:** Updates to `config_settings` where category includes 'ml_training' or 'ml'
**Use Case:** Update model training hyperparameters, batch sizes, learning rates

---

#### `config_changed_api_gateway`
**Purpose:** API Gateway configuration updates
**Trigger:** Updates to `config_settings` where category includes 'api_gateway' or 'auth'
**Use Case:** Update rate limits, JWT expiration, CORS settings

---

#### `config_changed_global`
**Purpose:** Global system configuration updates
**Trigger:** Updates to `config_settings` where category = 'global'
**Use Case:** System-wide settings affecting all services

---

### 2. Permission Change Channels

#### `permissions_changed`
**Purpose:** RBAC permission and role updates
**Trigger:** Changes to `roles`, `permissions`, `role_permissions`, or `user_roles` tables
**Payload Format:**
```json
{
  "operation": "INSERT",
  "table": "user_roles",
  "timestamp": 1696348234.567,
  "user_id": "e7488f2f-d4d8-4918-b311-e7b52bb0eae0",
  "role_id": "d4203937-acda-42fc-964b-96421b3a6998"
}
```

**Use Cases:**
- User role assignments
- Permission additions/removals
- Role permission updates
- Real-time authorization cache invalidation

**Listening Example:**
```sql
LISTEN permissions_changed;
```

---

### 3. Model Configuration Channels

#### `model_config_changed`
**Purpose:** ML model configuration updates
**Trigger:** Updates to `model_config` table (when implemented)
**Use Case:** ML model version updates, S3 path changes, cache invalidation

---

## Usage Patterns

### Service Integration Pattern

```rust
// In your Rust service
use tokio_postgres::{AsyncMessage, Client};

async fn listen_for_config_changes(client: &mut Client) {
    client.execute("LISTEN config_changed_trading", &[]).await.unwrap();
    client.execute("LISTEN permissions_changed", &[]).await.unwrap();

    loop {
        match client.recv().await {
            Ok(AsyncMessage::Notification(notification)) => {
                match notification.channel() {
                    "config_changed_trading" => {
                        let payload: ConfigChangePayload = serde_json::from_str(notification.payload()).unwrap();
                        reload_config(payload).await;
                    }
                    "permissions_changed" => {
                        let payload: PermissionChangePayload = serde_json::from_str(notification.payload()).unwrap();
                        invalidate_permission_cache(payload).await;
                    }
                    _ => {}
                }
            }
            _ => {}
        }
    }
}
```

---

## Testing NOTIFY Channels

### Test 1: Config Change Notification

**Terminal 1 (Listener):**
```bash
PGPASSWORD=foxhunt_dev_password psql -h localhost -U foxhunt -d foxhunt << 'EOF'
LISTEN config_changed_trading;
-- Keep this session open and wait for notifications
SELECT pg_sleep(300); -- Wait 5 minutes
EOF
```

**Terminal 2 (Trigger):**
```bash
PGPASSWORD=foxhunt_dev_password psql -h localhost -U foxhunt -d foxhunt << 'EOF'
UPDATE config_settings
SET config_value = '"999999"'::jsonb
WHERE config_key LIKE '%daily_loss%'
LIMIT 1;
EOF
```

**Expected Result:**
Terminal 1 receives: `Asynchronous notification "config_changed_trading" received from server process with PID 12345.`

---

### Test 2: Permission Change Notification

**Terminal 1 (Listener):**
```bash
PGPASSWORD=foxhunt_dev_password psql -h localhost -U foxhunt -d foxhunt -c "LISTEN permissions_changed;"
```

**Terminal 2 (Trigger):**
```bash
PGPASSWORD=foxhunt_dev_password psql -h localhost -U foxhunt -d foxhunt << 'EOF'
INSERT INTO user_roles (user_id, role_id)
SELECT
    (SELECT id FROM users WHERE username = 'test_trader'),
    (SELECT id FROM roles WHERE name = 'analyst');
EOF
```

**Expected Result:**
Terminal 1 receives notification with JSON payload containing user_id, role_id, operation, and timestamp.

---

## Trigger Functions

### `notify_config_change()`
**Attached To:**
- `config_settings` (INSERT, UPDATE, DELETE)
- `config_environment_overrides` (INSERT, UPDATE, DELETE)

**Logic:**
1. Extracts service scope from category path
2. Builds JSON payload with old/new values
3. Sends NOTIFY to `config_changed_{service}` channel

---

### `notify_permission_change()`
**Attached To:**
- `role_permissions` (INSERT, UPDATE, DELETE)
- `user_roles` (INSERT, UPDATE, DELETE)
- `permissions` (INSERT, UPDATE, DELETE)
- `roles` (INSERT, UPDATE, DELETE)

**Logic:**
1. Detects which table triggered the change
2. Builds JSON payload with appropriate IDs
3. Sends NOTIFY to `permissions_changed` channel

---

### `notify_model_config_change()`
**Attached To:**
- `model_config` (INSERT, UPDATE, DELETE) - when table exists

**Logic:**
1. Extracts model name and version
2. Builds JSON payload with S3 path information
3. Sends NOTIFY to `model_config_changed` channel

---

## Production Considerations

### Performance
- NOTIFY is non-blocking and lightweight
- Payloads limited to 8000 bytes (JSON should be compact)
- Use connection pooling to avoid excessive LISTEN connections

### Reliability
- NOTIFY is **not persistent** - messages are lost if no listeners
- Services should periodically poll configuration as fallback
- Use PostgreSQL replication for HA NOTIFY delivery

### Security
- NOTIFY messages are visible to all listeners on the same database
- Don't include sensitive data (passwords, API keys) in payloads
- Use row-level security on config tables if needed

---

## Migration Summary

| Migration | Tables Created | Triggers Added | Channels |
|-----------|----------------|----------------|----------|
| 009 | 4 | 2 | 0 |
| 017 | 4 | 1 | 0 |
| 018 | 4 | 4 | 1 (permissions_changed) |
| 019 | 0 | 6 | 5 (config_changed_*) |
| **Total** | **12** | **13** | **6** |

---

## Quick Reference

```bash
# List all NOTIFY triggers
PGPASSWORD=foxhunt_dev_password psql -h localhost -U foxhunt -d foxhunt -c "
SELECT trigger_name, event_object_table
FROM information_schema.triggers
WHERE trigger_name LIKE '%notify%'
ORDER BY event_object_table;"

# Test NOTIFY functionality
bash /home/jgrusewski/Work/foxhunt/database/migrations/test_notify_functionality.sh

# Listen to all channels (debugging)
PGPASSWORD=foxhunt_dev_password psql -h localhost -U foxhunt -d foxhunt << 'EOF'
LISTEN config_changed_trading;
LISTEN config_changed_backtesting;
LISTEN config_changed_ml_training;
LISTEN config_changed_api_gateway;
LISTEN config_changed_global;
LISTEN permissions_changed;
SELECT pg_sleep(600); -- Wait 10 minutes
EOF
```

---

**Created:** 2025-10-03
**Wave:** 71 Agent 7
**Status:** Production Ready