22787b3e0a
- Generate full AI-consumable docs (docs/ai/): system overview, architecture, module cheatsheet, API contracts, data model, build guide, decision log, glossary, and open questions (deep tier coverage) - Add PROBLEM_REPORT.md: categorized bug/risk summary - Add DETAILED_CODE_REVIEW.md: full line-by-line review of all 15 backend files, documenting 4 fatal issues, 5 critical deployment bugs, 4 security vulnerabilities, and 6 architecture defects with prioritized fix plan
219 lines
11 KiB
Markdown
219 lines
11 KiB
Markdown
---
|
||
generated_by: repo-insight
|
||
version: 1
|
||
created: 2026-03-03
|
||
last_updated: 2026-03-03
|
||
source_commit: 0d9dffa
|
||
coverage: standard
|
||
---
|
||
|
||
# 02 — Module Cheatsheet
|
||
|
||
## Purpose
|
||
Module-by-module index: file path, role, key public interfaces, and dependencies.
|
||
|
||
## TL;DR
|
||
- Backend has 20 Python modules; signal_engine.py is the largest and most complex (~1000+ lines).
|
||
- Frontend has 2 TypeScript lib files + 9 pages + 6 components.
|
||
- `db.py` is the only shared infrastructure module; all other backend modules import from it.
|
||
- `signal_engine.py` is the core business logic module; `live_executor.py` and `risk_guard.py` are independent processes that only use `db.py` and direct PG connections.
|
||
- Strategy configs are external JSON; no code changes needed to tune weights/thresholds.
|
||
|
||
## Canonical Facts
|
||
|
||
### Backend Modules
|
||
|
||
#### `backend/main.py` — FastAPI HTTP API
|
||
- **Role**: Primary HTTP API server; rate/snapshot/history proxy; aggTrade query endpoints; signal history.
|
||
- **Key interfaces**:
|
||
- `GET /api/health` — liveness check (public)
|
||
- `GET /api/rates` — live Binance premiumIndex for 4 symbols (public, 3 s cache)
|
||
- `GET /api/snapshots` — rate snapshot history from PG (auth required)
|
||
- `GET /api/kline` — candlestick bars aggregated from `rate_snapshots` (auth required)
|
||
- `GET /api/stats` — 7-day funding rate stats per symbol (auth required, 60 s cache)
|
||
- `GET /api/stats/ytd` — YTD annualized stats (auth required, 3600 s cache)
|
||
- `GET /api/history` — 7-day raw funding rate history (auth required, 60 s cache)
|
||
- `GET /api/signals/history` — `signal_logs` table (auth required)
|
||
- `GET /api/trades/meta` — `agg_trades_meta` (auth required)
|
||
- `GET /api/trades/summary` — aggregated OHLCV from `agg_trades` (auth required)
|
||
- Many more: paper trades, signals v52, live trades, live config, position sync, etc. (full list in saved tool output)
|
||
- **Deps**: `auth.py`, `db.py`, `httpx`, `asyncio`
|
||
- **Background task**: `background_snapshot_loop()` writes `rate_snapshots` every 2 s.
|
||
|
||
#### `backend/signal_engine.py` — V5 Signal Engine (PM2 worker)
|
||
- **Role**: Core signal computation loop; 15 s interval; in-memory rolling-window indicators; scored signal output.
|
||
- **Key classes**:
|
||
- `TradeWindow(window_ms)` — rolling CVD/VWAP calculator using `deque`; props: `cvd`, `vwap`
|
||
- `ATRCalculator(period_ms, length)` — 5-min candle ATR; props: `atr`, `atr_percentile`
|
||
- `SymbolState` — per-symbol state container holding `TradeWindow` ×3, `ATRCalculator`, large-order percentile deques
|
||
- **Key functions**:
|
||
- `load_strategy_configs() -> list[dict]` — reads JSON files from `strategies/`
|
||
- `fetch_market_indicators(symbol) -> dict` — reads `market_indicators` table
|
||
- `fetch_new_trades(symbol, last_id) -> list` — reads new rows from `agg_trades`
|
||
- `save_indicator(ts, symbol, result, strategy)` — writes to `signal_indicators`
|
||
- `paper_open_trade(...)` — inserts `paper_trades` row
|
||
- `paper_check_positions(symbol, price, now_ms)` — checks TP/SL for paper positions
|
||
- `main()` — entry point; calls `load_historical()` then enters main loop
|
||
- **Deps**: `db.py`, `json`, `collections.deque`
|
||
|
||
#### `backend/live_executor.py` — Live Trade Executor (PM2 worker)
|
||
- **Role**: Listens on PG `NOTIFY new_signal`; places Binance Futures market orders; writes `live_trades`.
|
||
- **Key functions**:
|
||
- `reload_live_config(conn)` — refreshes `RISK_PER_TRADE_USD`, `MAX_POSITIONS` from `live_config` every 60 s
|
||
- `binance_request(session, method, path, params)` — HMAC-signed Binance API call
|
||
- **Config**: `TRADE_ENV` (`testnet`/`production`), `LIVE_STRATEGIES`, `RISK_PER_TRADE_USD`, `MAX_POSITIONS`
|
||
- **Deps**: `psycopg2`, `aiohttp`, HMAC signing
|
||
|
||
#### `backend/risk_guard.py` — Risk Circuit-Breaker (PM2 worker)
|
||
- **Role**: Every 5 s; monitors PnL, balance, data freshness, hold timeouts; writes circuit-break flags.
|
||
- **Key classes**: `RiskState` — holds `status` (`normal`/`warning`/`circuit_break`), loss counters
|
||
- **Key functions**:
|
||
- `check_daily_loss(conn)` — sums `pnl_r` from today's `live_trades`
|
||
- `check_unrealized_loss(session, risk_usd_dynamic)` — queries Binance positions API
|
||
- `check_balance(session)` — queries Binance account balance
|
||
- `check_data_freshness(conn)` — checks `market_indicators` recency
|
||
- `check_hold_timeout(session, conn)` — force-closes positions held >70 min
|
||
- `trigger_circuit_break(session, conn, reason, action)` — writes to `live_events`, may flat positions
|
||
- `check_auto_resume()` — re-enables trading after cooldown
|
||
- `check_emergency_commands(session, conn)` — watches for manual DB commands
|
||
- **Deps**: `trade_config.py`, `aiohttp`, `psycopg2`
|
||
|
||
#### `backend/db.py` — Database Layer
|
||
- **Role**: All PG connectivity; schema creation; partition management.
|
||
- **Key interfaces**:
|
||
- Sync (psycopg2): `get_sync_conn()`, `sync_execute()`, `sync_executemany()`
|
||
- Async (asyncpg): `async_fetch()`, `async_fetchrow()`, `async_execute()`
|
||
- Cloud SQL sync pool: `get_cloud_sync_conn()` (non-fatal on failure)
|
||
- `init_schema()` — creates all tables from `SCHEMA_SQL`
|
||
- `ensure_partitions()` — creates `agg_trades_YYYYMM` partitions for current+next 2 months
|
||
- **Deps**: `asyncpg`, `psycopg2`
|
||
|
||
#### `backend/auth.py` — JWT Auth + Registration
|
||
- **Role**: FastAPI router at `/api`; register/login/refresh/logout endpoints.
|
||
- **Key interfaces**:
|
||
- `POST /api/register` — invite-code gated registration
|
||
- `POST /api/login` — returns `access_token` + `refresh_token`
|
||
- `POST /api/refresh` — token refresh
|
||
- `POST /api/logout` — revokes refresh token
|
||
- `GET /api/me` — current user info
|
||
- `get_current_user` — FastAPI `Depends` injector; validates Bearer JWT
|
||
- **Token storage**: HMAC-SHA256 hand-rolled JWT (no PyJWT); refresh tokens stored in `refresh_tokens` table.
|
||
- **Deps**: `db.py`, `hashlib`, `hmac`, `secrets`
|
||
|
||
#### `backend/agg_trades_collector.py` — AggTrades Collector (PM2 worker)
|
||
- **Role**: Streams Binance `aggTrade` WebSocket events; batch-inserts into `agg_trades` partitioned table; maintains `agg_trades_meta`.
|
||
- **Key functions**: `ws_collect(symbol)`, `rest_catchup(symbol, from_id)`, `continuity_check()`, `flush_buffer(symbol, trades)`
|
||
- **Deps**: `db.py`, `websockets`/`httpx`
|
||
|
||
#### `backend/market_data_collector.py` — Market Data Collector (PM2 worker)
|
||
- **Role**: Collects Binance market indicators (LS ratio, OI, coinbase premium, funding rate) via REST polling; stores in `market_indicators` JSONB.
|
||
- **Key class**: `MarketDataCollector`
|
||
- **Deps**: `db.py`, `httpx`
|
||
|
||
#### `backend/liquidation_collector.py` — Liquidation Collector (PM2 worker)
|
||
- **Role**: Streams Binance liquidation WS; aggregates into `liquidation_events` and `liquidation_agg` tables.
|
||
- **Key functions**: `ensure_table()`, `save_liquidation()`, `save_aggregated()`, `run()`
|
||
- **Deps**: `db.py`, `websockets`
|
||
|
||
#### `backend/backtest.py` — Offline Backtester
|
||
- **Role**: Replays `agg_trades` from PG to simulate signal engine and measure strategy performance.
|
||
- **Key classes**: `Position`, `BacktestEngine`
|
||
- **Key functions**: `load_trades()`, `run_backtest()`, `main()`
|
||
- **Deps**: `db.py`
|
||
|
||
#### `backend/trade_config.py` — Symbol / Qty Config
|
||
- **Role**: Constants for symbols and Binance qty precision.
|
||
- **Deps**: none
|
||
|
||
#### `backend/admin_cli.py` — Admin CLI
|
||
- **Role**: CLI for invite-code and user management (gen_invite, list_invites, ban_user, set_admin).
|
||
- **Deps**: `db.py`
|
||
|
||
#### `backend/subscriptions.py` — Subscription Query Helper
|
||
- **Role**: Helpers for querying signal history (used internally).
|
||
- **Deps**: `db.py`
|
||
|
||
#### `backend/paper_monitor.py` — Paper Trade Monitor
|
||
- **Role**: Standalone script to print paper trade status.
|
||
- **Deps**: `db.py`
|
||
|
||
#### `backend/signal_pusher.py` — Discord Notifier (PM2 worker)
|
||
- **Role**: Polls `signal_indicators` for high-score events; pushes Discord webhook notifications.
|
||
- **Deps**: `db.py`, `httpx`
|
||
|
||
#### `backend/position_sync.py` — Position Sync
|
||
- **Role**: Syncs live positions between `live_trades` table and Binance account state.
|
||
- **Deps**: `db.py`, `aiohttp`
|
||
|
||
#### `backend/fix_historical_pnl.py` — PnL Fix Script
|
||
- **Role**: One-time migration to recalculate historical PnL in `paper_trades`.
|
||
- **Deps**: `db.py`
|
||
|
||
### Frontend Modules
|
||
|
||
#### `frontend/lib/api.ts` — API Client
|
||
- **Role**: Typed `api` object with all backend endpoint wrappers; distinguishes public vs. protected fetches.
|
||
- **Interfaces exported**: `RateData`, `RatesResponse`, `HistoryPoint`, `HistoryResponse`, `StatsResponse`, `SignalHistoryItem`, `SnapshotItem`, `KBar`, `KlineResponse`, `YtdStatsResponse`, `api` object
|
||
- **Deps**: `lib/auth.tsx` (`authFetch`)
|
||
|
||
#### `frontend/lib/auth.tsx` — Auth Context
|
||
- **Role**: React context for current user; `useAuth()` hook; `authFetch()` with access-token injection and auto-refresh.
|
||
- **Deps**: Next.js router, `localStorage`
|
||
|
||
#### `frontend/app/` Pages
|
||
| Page | Route | Description |
|
||
|------|-------|-------------|
|
||
| `page.tsx` | `/` | Main dashboard: rates, kline, history, signal log |
|
||
| `dashboard/page.tsx` | `/dashboard` | (inferred) extended dashboard |
|
||
| `signals/page.tsx` | `/signals` | Signal history view (V5.1) |
|
||
| `signals-v52/page.tsx` | `/signals-v52` | Signal history view (V5.2) |
|
||
| `paper/page.tsx` | `/paper` | Paper trades view (V5.1) |
|
||
| `paper-v52/page.tsx` | `/paper-v52` | Paper trades view (V5.2) |
|
||
| `live/page.tsx` | `/live` | Live trades view |
|
||
| `history/page.tsx` | `/history` | Funding rate history |
|
||
| `kline/page.tsx` | `/kline` | Kline chart page |
|
||
| `trades/page.tsx` | `/trades` | aggTrades summary |
|
||
| `server/page.tsx` | `/server` | Server status / metrics |
|
||
| `about/page.tsx` | `/about` | About page |
|
||
| `login/page.tsx` | `/login` | Login form |
|
||
| `register/page.tsx` | `/register` | Registration form |
|
||
|
||
#### `frontend/components/`
|
||
| Component | Role |
|
||
|-----------|------|
|
||
| `Navbar.tsx` | Top navigation bar |
|
||
| `Sidebar.tsx` | Sidebar navigation |
|
||
| `AuthHeader.tsx` | Auth-aware header with user info |
|
||
| `RateCard.tsx` | Displays current funding rate for one asset |
|
||
| `StatsCard.tsx` | Displays 7d mean and annualized stats |
|
||
| `FundingChart.tsx` | Funding rate chart component |
|
||
| `LiveTradesCard.tsx` | Live trades summary card |
|
||
|
||
## Interfaces / Dependencies
|
||
|
||
### Key import graph (backend)
|
||
```
|
||
main.py → auth.py, db.py
|
||
signal_engine.py → db.py
|
||
live_executor.py → psycopg2 direct (no db.py module import)
|
||
risk_guard.py → trade_config.py, psycopg2 direct
|
||
backtest.py → db.py
|
||
agg_trades_collector.py → db.py
|
||
market_data_collector.py → db.py
|
||
liquidation_collector.py → db.py
|
||
admin_cli.py → db.py
|
||
```
|
||
|
||
## Unknowns & Risks
|
||
- [inference] Content of `frontend/app/dashboard/`, `signals/`, `paper/`, `live/` pages not read; role described from filename convention.
|
||
- [unknown] `signal_pusher.py` Discord webhook env var name not confirmed.
|
||
- [inference] `position_sync.py` exact interface not read; role inferred from name and listing.
|
||
|
||
## Source Refs
|
||
- `backend/main.py` — all API route definitions
|
||
- `backend/signal_engine.py:170-285` — `TradeWindow`, `ATRCalculator`, `SymbolState`
|
||
- `backend/auth.py:23-23` — router prefix `/api`
|
||
- `backend/db.py:35-157` — all public DB functions
|
||
- `frontend/lib/api.ts:103-116` — `api` export object
|
||
- `frontend/lib/auth.tsx` — auth context (not fully read)
|