arbitrage-engine/docs/ai/00-system-overview.md

---
generated_by: repo-insight
version: 1
created: 2026-03-03
last_updated: 2026-03-03
source_commit: 0d9dffa
coverage: standard
---

# 00 — System Overview

## Purpose
High-level description of the arbitrage-engine project: what it does, its tech stack, repo layout, and entry points.

## TL;DR
- **Domain**: Crypto perpetual futures funding-rate arbitrage monitoring and short-term trading signal engine.
- **Strategy**: Hold spot long + perpetual short to collect funding rates every 8 h; plus a CVD/ATR-based short-term directional signal engine (V5.x).
- **Backend**: Python / FastAPI + independent PM2 worker processes; PostgreSQL (local + Cloud SQL dual-write).
- **Frontend**: Next.js 16 / React 19 / TypeScript SPA, charting via `lightweight-charts` + Recharts.
- **Targets**: BTC, ETH, XRP, SOL perpetual contracts on Binance USDC-M futures.
- **Deployment**: PM2 process manager on a GCP VM; frontend served via Next.js; backend accessible at `https://arb.zhouyangclaw.com`.
- **Auth**: JWT (access 24 h + refresh 7 d) + invite-code registration gating.
- **Trading modes**: Paper (simulated), Live (Binance Futures testnet or production via `TRADE_ENV`).

## Canonical Facts

### Repo Layout
```
arbitrage-engine/
├── backend/                 # Python FastAPI API + all worker processes
│   ├── main.py              # FastAPI app entry point (uvicorn)
│   ├── signal_engine.py     # V5 signal engine (PM2 worker, 15 s loop)
│   ├── live_executor.py     # Live trade executor (PM2 worker)
│   ├── risk_guard.py        # Risk circuit-breaker (PM2 worker)
│   ├── market_data_collector.py  # Binance WS market data (PM2 worker)
│   ├── agg_trades_collector.py   # Binance aggTrades WS collector (PM2 worker)
│   ├── liquidation_collector.py  # Binance liquidation WS collector (PM2 worker)
│   ├── signal_pusher.py     # Discord signal notifier (PM2 worker)
│   ├── db.py                # Dual-pool PostgreSQL layer (psycopg2 sync + asyncpg async)
│   ├── auth.py              # JWT auth + invite-code registration router
│   ├── trade_config.py      # Symbol / qty precision constants
│   ├── backtest.py          # Offline backtest engine
│   ├── paper_monitor.py     # Paper trade monitoring helper
│   ├── admin_cli.py         # CLI for invite / user management
│   ├── subscriptions.py     # Signal subscription query helper
│   ├── paper_config.json    # Paper trading runtime toggle
│   ├── strategies/          # JSON strategy configs (v51_baseline, v52_8signals)
│   ├── ecosystem.dev.config.js  # PM2 process definitions
│   └── logs/                # Rotating log files
├── frontend/                # Next.js app
│   ├── app/                 # App Router pages
│   ├── components/          # Reusable UI components
│   ├── lib/api.ts           # Typed API client
│   └── lib/auth.tsx         # Auth context + token refresh logic
├── docs/                    # Documentation (including docs/ai/)
├── scripts/                 # Utility scripts
└── signal-engine.log        # Live log symlink / output file
```

### Primary Language & Frameworks
| Layer | Technology |
|-------|-----------|
| Backend API | Python 3.x, FastAPI, uvicorn |
| DB access | asyncpg (async), psycopg2 (sync) |
| Frontend | TypeScript, Next.js 16, React 19 |
| Styling | Tailwind CSS v4 |
| Charts | lightweight-charts 5.x, Recharts 3.x |
| Process manager | PM2 (via `ecosystem.dev.config.js`) |
| Database | PostgreSQL (local + Cloud SQL dual-write) |

### Entry Points
| Process | File | Role |
|---------|------|------|
| HTTP API | `backend/main.py` | FastAPI on uvicorn |
| Signal engine | `backend/signal_engine.py` | 15 s indicator loop |
| Trade executor | `backend/live_executor.py` | PG NOTIFY listener → Binance API |
| Risk guard | `backend/risk_guard.py` | 5 s circuit-breaker loop |
| Market data | `backend/market_data_collector.py` | Binance WS → `market_indicators` table |
| aggTrades collector | `backend/agg_trades_collector.py` | Binance WS → `agg_trades` partitioned table |
| Liquidation collector | `backend/liquidation_collector.py` | Binance WS → liquidation tables |
| Signal pusher | `backend/signal_pusher.py` | DB → Discord push |
| Frontend | `frontend/` | Next.js dev/prod server |

### Monitored Symbols
`BTCUSDT`, `ETHUSDT`, `XRPUSDT`, `SOLUSDT` (Binance USDC-M Futures)

### Environment Variables (key ones)
| Variable | Default | Description |
|----------|---------|-------------|
| `PG_HOST` | `127.0.0.1` | Local PG host |
| `PG_DB` | `arb_engine` | Database name |
| `PG_USER` / `PG_PASS` | `arb` / `arb_engine_2026` | PG credentials |
| `CLOUD_PG_HOST` | `10.106.0.3` | Cloud SQL host |
| `CLOUD_PG_ENABLED` | `true` | Enable dual-write |
| `JWT_SECRET` | (testnet default set) | JWT signing key |
| `TRADE_ENV` | `testnet` | `testnet` or `production` |
| `LIVE_STRATEGIES` | `["v52_8signals"]` | Active live trading strategies |
| `RISK_PER_TRADE_USD` | `2` | USD risk per trade |

## Interfaces / Dependencies
- **External API**: Binance USDC-M Futures REST (`https://fapi.binance.com/fapi/v1`) and WebSocket.
- **Discord**: Webhook for signal notifications (via `signal_pusher.py`).
- **CORS origins**: `https://arb.zhouyangclaw.com`, `http://localhost:3000`, `http://localhost:3001`.

## Unknowns & Risks
- [inference] PM2 `ecosystem.dev.config.js` not read in this pass; exact process restart policies and env injection not confirmed.
- [inference] `.env` file usage confirmed via `python-dotenv` calls in live modules, but `.env.example` absent.
- [unknown] Deployment pipeline (CI/CD) not present in repo.

## Source Refs
- `backend/main.py:1-27` — FastAPI app init, CORS, SYMBOLS
- `backend/signal_engine.py:1-16` — V5 architecture docstring
- `backend/live_executor.py:1-10` — live executor architecture comment
- `backend/risk_guard.py:1-12` — risk guard circuit-break rules
- `backend/db.py:14-30` — PG/Cloud SQL env config
- `frontend/package.json` — frontend dependencies
- `frontend/lib/api.ts:1-116` — typed API client