Files
foxhunt/docs/troubleshooting/database-issues.md
jgrusewski 7458f1be01 feat(wave12): E2E validation complete - 225-feature pipeline ready
 Validation Results:
- PPO training: 24.2s (1 epoch, 950 samples, dim=225)
- Feature extraction: 105μs/bar (9.5x faster than target)
- Model checkpoint: 293KB (147KB actor + 146KB critic)
- GPU memory: 145MB used (96.4% headroom)
- Zero dimension mismatches

📊 Success Criteria (5/5):
 Feature dimension = 225 (Wave C 201 + Wave D 24)
 Model state_dim = 225
 Training completed without errors
 Checkpoint saved successfully
 No dimension mismatch errors

📁 Training Data Ready:
- ES.FUT: 2.9MB, 180 days
- NQ.FUT: 4.4MB, 180 days
- 6E.FUT: 2.8MB, 180 days
- ZN.FUT: 65KB, 90 days (clean)

🚀 Next: Full production model retraining (4 models, ~10min GPU time)

🤖 Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-22 22:48:04 +02:00

1.8 KiB

Database Issues Troubleshooting Guide

Last Updated: 2025-10-22


Connection Issues

Symptom: could not connect to server

Diagnosis:

# Check PostgreSQL is running
kubectl get pods -n foxhunt -l app=postgresql

# Check service endpoint
kubectl get endpoints -n foxhunt foxhunt-postgresql

# Test connectivity
kubectl run -it --rm psql-test --image=postgres:15 --restart=Never -- \
    psql -h foxhunt-postgresql.foxhunt.svc.cluster.local -U foxhunt -d foxhunt

Resolution:

# Restart PostgreSQL
kubectl delete pod foxhunt-postgresql-0 -n foxhunt

# Verify secrets
kubectl get secret foxhunt-secrets -n foxhunt -o yaml

Slow Queries

Symptom: Queries taking >1s

Diagnosis:

-- Find slow queries
SELECT query, mean_exec_time, calls 
FROM pg_stat_statements 
ORDER BY mean_exec_time DESC 
LIMIT 10;

-- Analyze query plan
EXPLAIN ANALYZE SELECT * FROM orders WHERE symbol='ES.FUT' ORDER BY created_at DESC LIMIT 100;

Resolution:

-- Create index
CREATE INDEX CONCURRENTLY idx_orders_symbol_created_at ON orders(symbol, created_at DESC);

-- Update statistics
ANALYZE orders;

-- Increase work_mem for complex queries
SET work_mem = '256MB';

Deadlocks

Symptom: deadlock detected

Diagnosis:

-- Check deadlock log
SELECT * FROM pg_stat_activity WHERE wait_event_type = 'Lock';

-- Enable deadlock logging
ALTER SYSTEM SET deadlock_timeout = '1s';
ALTER SYSTEM SET log_lock_waits = on;
SELECT pg_reload_conf();

Resolution:

-- Access tables in consistent order
BEGIN;
SELECT * FROM orders WHERE id = 1 FOR UPDATE;  -- Always lock orders first
SELECT * FROM positions WHERE order_id = 1 FOR UPDATE;  -- Then positions
COMMIT;

End of Database Issues Troubleshooting Guide