arbitrage-engine/docs/ai/00-system-overview.md

generated_by	version	created	last_updated	source_commit	coverage
repo-insight	1	2026-03-03	2026-03-03	0d9dffa	standard

00 — System Overview

Purpose

High-level description of the arbitrage-engine project: what it does, its tech stack, repo layout, and entry points.

TL;DR

• Domain: Crypto perpetual futures funding-rate arbitrage monitoring and short-term trading signal engine.
• Strategy: Hold spot long + perpetual short to collect funding rates every 8 h; plus a CVD/ATR-based short-term directional signal engine (V5.x).
• Backend: Python / FastAPI + independent PM2 worker processes; PostgreSQL (local + Cloud SQL dual-write).
• Frontend: Next.js 16 / React 19 / TypeScript SPA, charting via lightweight-charts + Recharts.
• Targets: BTC, ETH, XRP, SOL perpetual contracts on Binance USDC-M futures.
• Deployment: PM2 process manager on a GCP VM; frontend served via Next.js; backend accessible at https://arb.zhouyangclaw.com.
• Auth: JWT (access 24 h + refresh 7 d) + invite-code registration gating.
• Trading modes: Paper (simulated), Live (Binance Futures testnet or production via TRADE_ENV).

Canonical Facts

Repo Layout

arbitrage-engine/
├── backend/                 # Python FastAPI API + all worker processes
│   ├── main.py              # FastAPI app entry point (uvicorn)
│   ├── signal_engine.py     # V5 signal engine (PM2 worker, 15 s loop)
│   ├── live_executor.py     # Live trade executor (PM2 worker)
│   ├── risk_guard.py        # Risk circuit-breaker (PM2 worker)
│   ├── market_data_collector.py  # Binance WS market data (PM2 worker)
│   ├── agg_trades_collector.py   # Binance aggTrades WS collector (PM2 worker)
│   ├── liquidation_collector.py  # Binance liquidation WS collector (PM2 worker)
│   ├── signal_pusher.py     # Discord signal notifier (PM2 worker)
│   ├── db.py                # Dual-pool PostgreSQL layer (psycopg2 sync + asyncpg async)
│   ├── auth.py              # JWT auth + invite-code registration router
│   ├── trade_config.py      # Symbol / qty precision constants
│   ├── backtest.py          # Offline backtest engine
│   ├── paper_monitor.py     # Paper trade monitoring helper
│   ├── admin_cli.py         # CLI for invite / user management
│   ├── subscriptions.py     # Signal subscription query helper
│   ├── paper_config.json    # Paper trading runtime toggle
│   ├── strategies/          # JSON strategy configs (v51_baseline, v52_8signals)
│   ├── ecosystem.dev.config.js  # PM2 process definitions
│   └── logs/                # Rotating log files
├── frontend/                # Next.js app
│   ├── app/                 # App Router pages
│   ├── components/          # Reusable UI components
│   ├── lib/api.ts           # Typed API client
│   └── lib/auth.tsx         # Auth context + token refresh logic
├── docs/                    # Documentation (including docs/ai/)
├── scripts/                 # Utility scripts
└── signal-engine.log        # Live log symlink / output file

Primary Language & Frameworks

Layer	Technology
Backend API	Python 3.x, FastAPI, uvicorn
DB access	asyncpg (async), psycopg2 (sync)
Frontend	TypeScript, Next.js 16, React 19
Styling	Tailwind CSS v4
Charts	lightweight-charts 5.x, Recharts 3.x
Process manager	PM2 (via ecosystem.dev.config.js)
Database	PostgreSQL (local + Cloud SQL dual-write)

Entry Points

Process	File	Role
HTTP API	backend/main.py	FastAPI on uvicorn
Signal engine	backend/signal_engine.py	15 s indicator loop
Trade executor	backend/live_executor.py	PG NOTIFY listener → Binance API
Risk guard	backend/risk_guard.py	5 s circuit-breaker loop
Market data	backend/market_data_collector.py	Binance WS → market_indicators table
aggTrades collector	backend/agg_trades_collector.py	Binance WS → agg_trades partitioned table
Liquidation collector	backend/liquidation_collector.py	Binance WS → liquidation tables
Signal pusher	backend/signal_pusher.py	DB → Discord push
Frontend	frontend/	Next.js dev/prod server

Monitored Symbols

BTCUSDT, ETHUSDT, XRPUSDT, SOLUSDT (Binance USDC-M Futures)

Environment Variables (key ones)

Variable	Default	Description
PG_HOST	127.0.0.1	Local PG host
PG_DB	arb_engine	Database name
PG_USER / PG_PASS	arb / arb_engine_2026	PG credentials
CLOUD_PG_HOST	10.106.0.3	Cloud SQL host
CLOUD_PG_ENABLED	true	Enable dual-write
JWT_SECRET	(testnet default set)	JWT signing key
TRADE_ENV	testnet	testnet or production
LIVE_STRATEGIES	["v52_8signals"]	Active live trading strategies
RISK_PER_TRADE_USD	2	USD risk per trade

Interfaces / Dependencies

• External API: Binance USDC-M Futures REST (https://fapi.binance.com/fapi/v1) and WebSocket.
• Discord: Webhook for signal notifications (via signal_pusher.py).
• CORS origins: https://arb.zhouyangclaw.com, http://localhost:3000, http://localhost:3001.

Unknowns & Risks

• [inference] PM2 ecosystem.dev.config.js not read in this pass; exact process restart policies and env injection not confirmed.
• [inference] .env file usage confirmed via python-dotenv calls in live modules, but .env.example absent.
• [unknown] Deployment pipeline (CI/CD) not present in repo.

Source Refs

• backend/main.py:1-27 — FastAPI app init, CORS, SYMBOLS
• backend/signal_engine.py:1-16 — V5 architecture docstring
• backend/live_executor.py:1-10 — live executor architecture comment
• backend/risk_guard.py:1-12 — risk guard circuit-break rules
• backend/db.py:14-30 — PG/Cloud SQL env config
• frontend/package.json — frontend dependencies
• frontend/lib/api.ts:1-116 — typed API client