BAOLIFE DATABASE ARCHITECTURE - QUICK SUMMARY ============================================== CRITICAL FINDINGS: ================== 1. CONNECTION MANAGEMENT - HIGH RISK - Dual connection systems (database.py pool vs functions.py singleton) - functions.py doesn't use the new pool - Cursors NEVER explicitly closed → resource leaks - Single blocking connection at module import time 2. QUERY PATTERNS - CRITICAL ISSUE - Entire 2862-line player object serialized as BLOB (~500KB-2MB) on EVERY save - 40-50+ saves per player session = 20GB+ per 1000 players per month - Dual serialization: both JSON AND pickle stored (redundant) 3. N+1 QUERIES - CRITICAL ISSUE - Server startup: 1 query to get all player IDs + N queries to load each (100 players = 101 queries, 30+ sec startup) - Random name generation: 2 queries with RAND() per NPC (full table scans) 4. MISSING INDEXES - HIGH ISSUE - lifesim_savegames table structure NOT documented in migrations - RAND() queries full-scan 500+ rows for 1 result - messages table missing indexes on player/date 5. TRANSACTION MANAGEMENT - HIGH ISSUE - No transaction context; manual commit only - No rollback handling; inconsistent state possible under failure - New transactions.py not integrated with functions.py 6. CONCURRENCY - HIGH RISK - Pool limit: 32 connections (too low for any scale) - Global mydb variable creates bottleneck - Lost update race condition: concurrent saves overwrite each other PERFORMANCE ESTIMATES: ====================== - Per save: 515-1270ms (serialization + network + DB) - Per load: 111-355ms (deserialization + network + DB) - Startup with 100 players: 30+ seconds - Startup with 1000 players: 5+ minutes - Max concurrent players: ~32 (limited by connection pool) - Database growth: 3.9GB per active player per month KEY FILE LOCATIONS: =================== - Connection pool: /home/user/lichun/ws/database.py - Legacy connection: /home/user/lichun/ws/functions.py (lines 25-47) - saveGame/loadGame: /home/user/lichun/ws/functions.py (lines 446-475) - Game loop: /home/user/lichun/ws/app.py (initLifeSim, iterateGames) - Transaction context: /home/user/lichun/ws/database/transactions.py - Config: /home/user/lichun/ws/config.py - Migrations: /home/user/lichun/ws/migrations/*.sql IMMEDIATE FIXES (Priority Order): ================================== 1. Close cursors: Add mycursor.close() after every query in functions.py 2. Implement pooling: Make functions.py use database.py pool 3. Use transactions: Wrap saveGame/loadGame with transaction context manager 4. Add indexes: Document and create PRIMARY KEY on lifesim_savegames 5. Remove dual serialization: Keep only pickle OR JSON, not both 6. Optimize RAND(): Fix "ORDER BY RAND()" pattern - use LIMIT first SECURITY ISSUE: =============== - Hardcoded password in /home/user/lichun/ws/tests/test_db_setup.py line 37 - Password: H8g6gRA2r/h$[t{6 - Risk: CRITICAL - exposed in source control FULL ANALYSIS: ============== See: /home/user/lichun/DATABASE_ARCHITECTURE_ANALYSIS.md (12,000+ words) Contains: Detailed code examples, scalability assessment, recommendations