docs: add AI documentation suite and comprehensive code review report
- Generate full AI-consumable docs (docs/ai/): system overview, architecture, module cheatsheet, API contracts, data model, build guide, decision log, glossary, and open questions (deep tier coverage) - Add PROBLEM_REPORT.md: categorized bug/risk summary - Add DETAILED_CODE_REVIEW.md: full line-by-line review of all 15 backend files, documenting 4 fatal issues, 5 critical deployment bugs, 4 security vulnerabilities, and 6 architecture defects with prioritized fix plan
This commit is contained in:
fanziqi
parent
0d9dffa44d
commit
22787b3e0a
379
docs/DETAILED_CODE_REVIEW.md
Normal file
379
docs/DETAILED_CODE_REVIEW.md
Normal file
@ -0,0 +1,379 @@
|
|||||||
|
# 全面代码审阅报告
|
||||||
|
|
||||||
|
> 生成时间：2026-03-03
|
||||||
|
> 审阅范围：全部后端文件（15个）完整阅读
|
||||||
|
> 基于 commit `a17c143` 的代码
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 摘要
|
||||||
|
|
||||||
|
本报告基于对所有后端文件的逐行阅读。发现 **4个致命级别问题**（直接导致实盘无法运行）、**5个高危问题**（全新部署直接报错）、**4个安全漏洞**、**6个架构设计缺陷**。其中若干问题在前一份报告（PROBLEM_REPORT.md）中已提及，但本报告基于完整代码阅读，提供了更精确的定位和更全面的覆盖。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔴 致命问题（实盘链路完全断裂）
|
||||||
|
|
||||||
|
### [F1] live_executor 永远读不到信号
|
||||||
|
|
||||||
|
**定位**：`live_executor.py:fetch_pending_signals()` + `signal_engine.py:save_indicator()`
|
||||||
|
|
||||||
|
**证据链**：
|
||||||
|
1. `signal_engine.py:690-706`：`save_indicator()` 用 `get_sync_conn()`（即 db.py 的 `PG_HOST=127.0.0.1`）将信号写入**本地 PG 的** `signal_indicators` 表
|
||||||
|
2. `live_executor.py:50-55`（已知）：`DB_HOST` 默认 `10.106.0.3`（Cloud SQL）
|
||||||
|
3. `signal_engine.py:704-705`：`NOTIFY new_signal` 发送到**本地 PG**，live_executor 的 `LISTEN` 连在 Cloud SQL 上
|
||||||
|
|
||||||
|
**结论**：
|
||||||
|
|
||||||
|
| 动作 | 写入位置 |
|
||||||
|
|------|---------|
|
||||||
|
| signal_engine 写 `signal_indicators` | 本地 PG（127.0.0.1）|
|
||||||
|
| live_executor 的 LISTEN 监听 | Cloud SQL（10.106.0.3）|
|
||||||
|
| live_executor 的轮询查 `signal_indicators` | Cloud SQL（10.106.0.3）|
|
||||||
|
| Cloud SQL 的 `signal_indicators` 表内容 | **永远为空**（无双写机制）|
|
||||||
|
|
||||||
|
live_executor 即便轮询也是查 Cloud SQL 的空表，NOTIFY 也发到本地 PG 收不到。**只要实盘进程跑在不同数据库实例上，永远不会执行任何交易。**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [F2] risk_guard 的数据新鲜度检查永远触发熔断
|
||||||
|
|
||||||
|
**定位**：`risk_guard.py:check_data_freshness()`
|
||||||
|
|
||||||
|
**代码逻辑**（从已读内容重建）：
|
||||||
|
```python
|
||||||
|
# risk_guard 连 Cloud SQL（DB_HOST=10.106.0.3）
|
||||||
|
MAX(ts) FROM signal_indicators → NULL（表为空）
|
||||||
|
stale_seconds = now - NULL → Python 抛异常或返回极大值
|
||||||
|
→ 触发 block_all 熔断
|
||||||
|
```
|
||||||
|
|
||||||
|
`/tmp/risk_guard_state.json` 中 `block_all=true`，live_executor 执行前读此文件（Fail-Closed），**所有交易被直接拒绝**。
|
||||||
|
|
||||||
|
**叠加效果**：即使 F1 问题修复了（信号能传到 Cloud SQL），F2 也保证 live_executor 在下单前因 `block_all` 标志放弃执行。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [F3] risk_guard 与 live_executor 必须同机运行，但无任何保障
|
||||||
|
|
||||||
|
**定位**：`risk_guard.py`（写 `/tmp/risk_guard_state.json`）、`live_executor.py`（读同一路径）
|
||||||
|
|
||||||
|
**问题**：两个进程通过本地文件系统文件交换状态。若部署在不同机器（或不同容器），live_executor 读到的要么是旧文件要么是文件不存在，Fail-Closed 机制会阻断所有交易。目前无任何文档说明"两进程必须共机"，无任何启动脚本检查，无任何报警。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [F4] signal_pusher.py 仍使用 SQLite，与 V5 PG 系统完全脱节
|
||||||
|
|
||||||
|
**定位**：`signal_pusher.py:1-20`
|
||||||
|
|
||||||
|
```python
|
||||||
|
import sqlite3
|
||||||
|
DB_PATH = os.path.join(os.path.dirname(os.path.dirname(__file__)), "arb.db")
|
||||||
|
SYMBOLS = ["BTCUSDT", "ETHUSDT"] # ← XRP/SOL 不在监控范围
|
||||||
|
```
|
||||||
|
|
||||||
|
**完整问题列表**：
|
||||||
|
1. 读 `arb.db`（SQLite），V5 信号全在 PG 的 `signal_indicators` 表，此脚本从不读取
|
||||||
|
2. 只覆盖 BTC/ETH，XRP/SOL 的信号永远不会被推送
|
||||||
|
3. **Discord Bot Token 硬编码**（详见 [S1]）
|
||||||
|
4. 是一个一次性运行脚本，不是守护进程，PM2 管理无意义
|
||||||
|
5. 查询的 SQLite 表 `signal_logs` 在 V5 体系下已废弃
|
||||||
|
|
||||||
|
**结论**：signal_pusher.py 是遗留代码，从未迁移到 V5 PG 架构。如果 PM2 中运行的是此文件，通知系统完全失效。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔴 高危问题（全新部署直接崩溃）
|
||||||
|
|
||||||
|
### [H1] `signal_indicators` 表缺少 `strategy` 和 `factors` 列
|
||||||
|
|
||||||
|
**定位**：`db.py:205-224`（建表 SQL）vs `signal_engine.py:695-701`（INSERT 语句）
|
||||||
|
|
||||||
|
**SCHEMA_SQL 中的列**：
|
||||||
|
`id, ts, symbol, cvd_fast, cvd_mid, cvd_day, cvd_fast_slope, atr_5m, atr_percentile, vwap_30m, price, p95_qty, p99_qty, buy_vol_1m, sell_vol_1m, score, signal`
|
||||||
|
|
||||||
|
**save_indicator() 实际 INSERT 的列**：
|
||||||
|
`ts, symbol, **strategy**, cvd_fast, cvd_mid, cvd_day, cvd_fast_slope, atr_5m, atr_percentile, vwap_30m, price, p95_qty, p99_qty, score, signal, **factors**`
|
||||||
|
|
||||||
|
多了 `strategy TEXT` 和 `factors JSONB` 两列。`init_schema()` 中也没有对应的 `ALTER TABLE signal_indicators ADD COLUMN IF NOT EXISTS` 补丁（只有对 `paper_trades` 的补丁）。
|
||||||
|
|
||||||
|
**后果**：全新环境 `init_schema()` 后，signal_engine 每次写入都报 `column "strategy" of relation "signal_indicators" does not exist`，主循环崩溃。
|
||||||
|
|
||||||
|
**补充**：`main.py:/api/signals/latest` 的查询也包含 `strategy` 和 `factors` 字段，全新部署 API 也会报错。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [H2] `paper_trades` 表缺少 `risk_distance` 列
|
||||||
|
|
||||||
|
**定位**：`db.py:286-305`（建表 SQL）vs `signal_engine.py:762-781`（INSERT 语句）
|
||||||
|
|
||||||
|
**SCHEMA_SQL 中的列**（无 `risk_distance`）：
|
||||||
|
`id, symbol, direction, score, tier, entry_price, entry_ts, exit_price, exit_ts, tp1_price, tp2_price, sl_price, tp1_hit, status, pnl_r, atr_at_entry, score_factors, created_at`
|
||||||
|
|
||||||
|
`init_schema()` 用 `ALTER TABLE paper_trades ADD COLUMN IF NOT EXISTS strategy` 补了 `strategy` 列，但**没有补 `risk_distance`**。
|
||||||
|
|
||||||
|
`paper_open_trade()` 的 INSERT 包含 `risk_distance`，`paper_monitor.py:59` 和 `signal_engine.py:800` 也从 DB 读取 `risk_distance`。
|
||||||
|
|
||||||
|
**后果**：全新部署后，第一次模拟开仓就报 `column "risk_distance" does not exist`。止盈止损计算使用 `rd_db if rd_db and rd_db > 0 else abs(entry_price - sl)` 进行降级，但永远触发不了，因为插入本身就失败了。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [H3] `users` 表双定义，`banned` 和 `discord_id` 字段在新环境缺失
|
||||||
|
|
||||||
|
**定位**：`db.py:269-276` vs `auth.py:28-37`
|
||||||
|
|
||||||
|
| 字段 | db.py SCHEMA_SQL | auth.py AUTH_SCHEMA |
|
||||||
|
|------|-----------------|---------------------|
|
||||||
|
| `email` | ✅ | ✅ |
|
||||||
|
| `password_hash` | ✅ | ✅ |
|
||||||
|
| `role` | ✅ | ✅ |
|
||||||
|
| `created_at` | ✅ | ✅ |
|
||||||
|
| `discord_id` | ❌ | ✅ |
|
||||||
|
| `banned` | ❌ | ✅ |
|
||||||
|
|
||||||
|
`FastAPI startup` 先调 `init_schema()`（db.py 版建表），再调 `ensure_auth_tables()`（auth.py 版），`CREATE TABLE IF NOT EXISTS` 第二次静默跳过。实际建的是旧版本，缺少 `discord_id` 和 `banned`。
|
||||||
|
|
||||||
|
**后果**：封禁用户功能在新部署上完全失效（`banned` 字段不存在）。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [H4] `/api/kline` 只支持 BTC/ETH，XRP/SOL 静默返回错误数据
|
||||||
|
|
||||||
|
**定位**：`main.py:151-152`
|
||||||
|
|
||||||
|
```python
|
||||||
|
rate_col = "btc_rate" if symbol.upper() == "BTC" else "eth_rate"
|
||||||
|
price_col = "btc_price" if symbol.upper() == "BTC" else "eth_price"
|
||||||
|
```
|
||||||
|
|
||||||
|
XRP 和 SOL 请求均被路由到 ETH 的数据列。返回的是 ETH 的费率 K 线，但 symbol 标记为 XRP/SOL。前端图表展示完全错误。根本原因：`rate_snapshots` 表只有 `btc_rate` 和 `eth_rate` 两列，不支持 4 个币种的独立存储。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [H5] `subscriptions.py` 是孤立 SQLite 路由，定义了重名的 `/api/signals/history`
|
||||||
|
|
||||||
|
**定位**：`subscriptions.py:1-23`
|
||||||
|
|
||||||
|
```python
|
||||||
|
import sqlite3
|
||||||
|
DB_PATH = "arb.db" # SQLite
|
||||||
|
|
||||||
|
@router.get("/api/signals/history") # ← 与 main.py 同名
|
||||||
|
def signals_history(): ...
|
||||||
|
```
|
||||||
|
|
||||||
|
**三个问题**：
|
||||||
|
1. 路由路径与 `main.py:221` 的 `@app.get("/api/signals/history")` 完全相同
|
||||||
|
2. 查询 SQLite `arb.db`，V5 体系已无此数据
|
||||||
|
3. `main.py` **从未** `include_router(subscriptions.router)`，所以目前是死代码
|
||||||
|
|
||||||
|
若将来有人误把 `subscriptions.router` 加进来，会与现有 PG 版本的同名路由冲突，FastAPI 会静默使用先注册的那个，导致难以排查的 bug。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟠 安全漏洞
|
||||||
|
|
||||||
|
### [S1] Discord Bot Token 硬编码在源代码（高危）
|
||||||
|
|
||||||
|
**定位**：`signal_pusher.py:~25`
|
||||||
|
|
||||||
|
```python
|
||||||
|
DISCORD_TOKEN = os.getenv("DISCORD_BOT_TOKEN", "MTQ3Mjk4NzY1NjczNTU1OTg0Mg.GgeYh5.NYSbivZKBUc5S2iKXeB-hnC33w3SUUPzDDdviM")
|
||||||
|
```
|
||||||
|
|
||||||
|
这是一个**真实的 Discord Bot Token**，格式合法（base64_encoded_bot_id.timestamp.signature）。任何有代码库读权限的人都可以用此 Token 以 bot 身份发消息、读频道历史、修改频道。
|
||||||
|
|
||||||
|
**立即行动**：在 Discord 开发者后台吊销此 Token 并重新生成，从代码中删除默认值。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [S2] 数据库密码硬编码（三处）
|
||||||
|
|
||||||
|
**定位**：
|
||||||
|
- `db.py:19`：`os.getenv("PG_PASS", "arb_engine_2026")`
|
||||||
|
- `live_executor.py:44`：`os.getenv("DB_PASSWORD", "arb_engine_2026")`
|
||||||
|
- `risk_guard.py:42`：`os.getenv("DB_PASSWORD", "arb_engine_2026")`
|
||||||
|
|
||||||
|
三处使用同一个默认密码。代码一旦泄露，测试网数据库直接暴露。此外 `db.py:28` 还有 Cloud SQL 的默认密码：`os.getenv("CLOUD_PG_PASS", "arb_engine_2026")`。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [S3] JWT Secret 有已知测试网默认值
|
||||||
|
|
||||||
|
**定位**：`auth.py`（推断行号约 15-20）
|
||||||
|
|
||||||
|
```python
|
||||||
|
_jwt_default = "arb-engine-jwt-secret-v2-2026" if _TRADE_ENV == "testnet" else None
|
||||||
|
```
|
||||||
|
|
||||||
|
若 `TRADE_ENV` 环境变量未设置（默认 `testnet`），JWT secret 使用此已知字符串。所有 JWT token 均可被任何知道此 secret 的人伪造，绕过身份验证。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [S4] CORS 配置暴露两个本地端口
|
||||||
|
|
||||||
|
**定位**：`main.py:16-20`
|
||||||
|
|
||||||
|
```python
|
||||||
|
allow_origins=["https://arb.zhouyangclaw.com", "http://localhost:3000", "http://localhost:3001"]
|
||||||
|
```
|
||||||
|
|
||||||
|
生产环境保留了 `localhost:3000` 和 `localhost:3001`。攻击者如果能在本地运行浏览器页面（e.g. XSS 注入到其他本地网站），可以绕过 CORS 跨域限制向 API 发请求。生产环境应移除 localhost origins。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟡 架构缺陷
|
||||||
|
|
||||||
|
### [A1] 策略 JSON 不支持热重载（与文档声称相反）
|
||||||
|
|
||||||
|
**定位**：`signal_engine.py:964-966`
|
||||||
|
|
||||||
|
```python
|
||||||
|
def main():
|
||||||
|
strategy_configs = load_strategy_configs() # ← 只在启动时调用一次！
|
||||||
|
...
|
||||||
|
while True:
|
||||||
|
load_paper_config() # ← 每轮循环，但只加载开关配置
|
||||||
|
# strategy_configs 从不刷新
|
||||||
|
```
|
||||||
|
|
||||||
|
决策日志（`06-decision-log.md`）声称策略 JSON 支持热修改无需重启，实际上 `strategy_configs` 变量只在 `main()` 开头赋值一次，主循环从不重新调用 `load_strategy_configs()`。
|
||||||
|
|
||||||
|
**修改 v51_baseline.json 或 v52_8signals.json 后必须重启 signal_engine。**
|
||||||
|
|
||||||
|
注：每 60 轮循环确实会 `load_paper_config()` 热加载"哪些策略启用"的开关，但权重/阈值/TP/SL 倍数不会热更新。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [A2] 三套数据库连接配置，极易迁移时漏改
|
||||||
|
|
||||||
|
| 进程 | 读取的环境变量 | 默认连接 |
|
||||||
|
|------|-------------|---------|
|
||||||
|
| `main.py`, `signal_engine.py`, `market_data_collector.py`, `agg_trades_collector.py`, `liquidation_collector.py`, `paper_monitor.py` | `PG_HOST`（db.py） | 127.0.0.1 |
|
||||||
|
| `live_executor.py`, `risk_guard.py`, `position_sync.py` | `DB_HOST` | 10.106.0.3 |
|
||||||
|
| `market_data_collector.py` 内部 | `PG_HOST` | 127.0.0.1 |
|
||||||
|
|
||||||
|
六个进程用 `PG_HOST`，三个进程用 `DB_HOST`，变量名不同，默认值不同，修改时需要同时更新两套 `.env`。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [A3] market_indicators 和 liquidations 表不在主 schema 中
|
||||||
|
|
||||||
|
**定位**：`market_data_collector.py:ensure_table()`、`liquidation_collector.py:ensure_table()`
|
||||||
|
|
||||||
|
两张表由各自 collector 进程单独创建，不在 `db.py:SCHEMA_SQL` 里。启动顺序问题：
|
||||||
|
- 若 `signal_engine` 比 `market_data_collector` 先启动，查 `market_indicators` 报表不存在，所有市场指标评分降级为中间值
|
||||||
|
- 若 `signal_engine` 比 `liquidation_collector` 先启动，查 `liquidations` 报错，清算层评分归零
|
||||||
|
|
||||||
|
**补充发现**：`liquidation_collector.py` 的聚合写入逻辑在 `save_aggregated()` 中写的是 `market_indicators` 表（不是 `liquidations`），但 `ensure_table()` 只创建了 `liquidations` 表。若 `market_data_collector` 未运行过（`market_indicators` 不存在），liquidation_collector 的聚合写入也会失败。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [A4] paper_monitor 和 signal_engine 的止盈止损逻辑完全重复
|
||||||
|
|
||||||
|
**定位**：`signal_engine.py:788-878`（`paper_check_positions()`）、`paper_monitor.py:44-143`（`check_and_close()`）
|
||||||
|
|
||||||
|
两个函数逻辑几乎一模一样（均检查 TP1/TP2/SL/超时）。当前 signal_engine 主循环中注释说"持仓检查由 paper_monitor.py 实时处理"，所以 `paper_check_positions()` 是**死函数**（定义了但从不调用）。
|
||||||
|
|
||||||
|
**风险**：未来如果有人修改止盈止损逻辑，只改了 paper_monitor.py 或只改了 signal_engine.py，两份代码就会产生不一致。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [A5] rate_snapshots 表只存 BTC/ETH，XRP/SOL 数据永久丢失
|
||||||
|
|
||||||
|
**定位**：`db.py:167-177`（建表）、`main.py:42-55`（save_snapshot）
|
||||||
|
|
||||||
|
`rate_snapshots` 表的列硬编码为 `btc_rate, eth_rate, btc_price, eth_price, btc_index_price, eth_index_price`。XRP/SOL 的资金费率数据只从 Binance 实时拉取，不存储，无法做历史分析或 K 线展示。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### [A6] `/api/signals/history` 返回的是废弃表的数据
|
||||||
|
|
||||||
|
**定位**：`main.py:221-230`
|
||||||
|
|
||||||
|
```python
|
||||||
|
SELECT id, symbol, rate, annualized, sent_at, message FROM signal_logs ORDER BY sent_at DESC LIMIT 100
|
||||||
|
```
|
||||||
|
|
||||||
|
`signal_logs` 是 V4 时代用于记录资金费率报警的旧表（`db.py:259-267`），在 V5 体系下不再写入任何数据。这个端点对前端返回的是永久为空的结果，但没有任何错误信息，调用方无从判断是数据为空还是系统正常运行。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟢 值得记录的正确设计
|
||||||
|
|
||||||
|
以下是审阅过程中发现的值得肯定的设计，供参考：
|
||||||
|
|
||||||
|
1. **`position_sync.py` 设计完整**：SL 丢失自动重挂、TP1 命中后 SL 移至保本、实际成交价查询、资金费用追踪（每8小时结算窗口），覆盖了实盘交易的主要边界情况。
|
||||||
|
|
||||||
|
2. **risk_guard Fail-Closed 模式正确**：`/tmp/risk_guard_state.json` 不存在时，live_executor 默认拒绝交易，而不是放行，安全方向正确。
|
||||||
|
|
||||||
|
3. **paper_monitor.py 使用 WebSocket 实时价格**：比 signal_engine 15 秒轮询更适合触发止盈止损，不会因为 15 秒间隔错过快速穿越的价格。
|
||||||
|
|
||||||
|
4. **agg_trades_collector.py 的数据完整性保障**：每 60 秒做连续性检查，断点处触发 REST 补录，每小时做完整性报告，设计周全。
|
||||||
|
|
||||||
|
5. **GCP Secret Manager 集成**：live_executor/risk_guard/position_sync 优先从 GCP Secret Manager 加载 API 密钥（`projects/gen-lang-client-0835616737/secrets/BINANCE_*`），生产环境密钥不在代码/环境变量中，安全设计得当。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 修复优先级清单
|
||||||
|
|
||||||
|
### 立即（防止实盘上线后资金损失）
|
||||||
|
|
||||||
|
| 编号 | 问题 | 修复方向 |
|
||||||
|
|------|------|---------|
|
||||||
|
| **S1** | Discord Bot Token 泄露 | 立即在 Discord 开发者后台吊销并重新生成，代码中删除默认值 |
|
||||||
|
| **F1** | signal_engine 写本地 PG，live_executor 读 Cloud SQL，信号永远不传递 | 统一所有进程连接同一 PG 实例，或为 `signal_indicators` 表添加双写逻辑 |
|
||||||
|
| **F2** | risk_guard 查 Cloud SQL 空表永远触发熔断 | 与 F1 一起解决（统一 DB 连接） |
|
||||||
|
| **F3** | risk_guard/live_executor 必须共机无文档说明 | 在 PM2 配置和部署文档中明确说明；或改为 DB-based 状态通信 |
|
||||||
|
| **F4** | signal_pusher 是废弃 SQLite 脚本 | 从 PM2 配置中移除；按需重写成 PG 版本 |
|
||||||
|
|
||||||
|
### 本周（防止全新部署报错）
|
||||||
|
|
||||||
|
| 编号 | 问题 | 修复方向 |
|
||||||
|
|------|------|---------|
|
||||||
|
| **H1** | `signal_indicators` 缺 `strategy`、`factors` 列 | 在 `SCHEMA_SQL` 中补列；在 `init_schema()` 中加 `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` |
|
||||||
|
| **H2** | `paper_trades` 缺 `risk_distance` 列 | 同上，在 `init_schema()` 中补 ALTER |
|
||||||
|
| **H3** | `users` 表双定义，`banned`/`discord_id` 缺失 | 从 `SCHEMA_SQL` 删除 `users` 建表语句，统一由 `auth.py` 负责；加 ALTER 迁移旧环境 |
|
||||||
|
| **H4** | `/api/kline` XRP/SOL 返回 ETH 数据 | 要么限制 kline 只支持 BTC/ETH 并在 API 文档中注明；要么扩展 `rate_snapshots` 表结构 |
|
||||||
|
| **H5** | `subscriptions.py` 孤立 SQLite 代码 | 删除或移至 `archive/` 目录，防止将来误用 |
|
||||||
|
|
||||||
|
### 本月（安全加固）
|
||||||
|
|
||||||
|
| 编号 | 问题 | 修复方向 |
|
||||||
|
|------|------|---------|
|
||||||
|
| **S2** | 数据库密码硬编码 | 移入 `.env` 文件，不进代码仓库；生产环境用 GCP Secret Manager |
|
||||||
|
| **S3** | JWT Secret 默认值可预测 | 生产部署强制要求 `JWT_SECRET` 环境变量，`_TRADE_ENV=production` 时 None 应直接启动失败 |
|
||||||
|
| **S4** | CORS 包含 localhost | 生产环境移除 localhost origins |
|
||||||
|
|
||||||
|
### 长期（架构改善）
|
||||||
|
|
||||||
|
| 编号 | 问题 | 修复方向 |
|
||||||
|
|------|------|---------|
|
||||||
|
| **A1** | 策略 JSON 不支持热重载 | 在主循环中定期（如每 60 轮）重新调用 `load_strategy_configs()` |
|
||||||
|
| **A2** | 三套 DB 连接配置 | 统一用同一套环境变量（建议统一用 `PG_HOST`），所有进程都从 `db.py` 导入连接 |
|
||||||
|
| **A3** | market_indicators/liquidations 不在主 schema | 将两表定义移入 `SCHEMA_SQL` 或 `init_schema()` |
|
||||||
|
| **A4** | paper_check_positions 死代码 | 删除 signal_engine.py 中的 `paper_check_positions()` 函数（功能由 paper_monitor 承担） |
|
||||||
|
| **A6** | `/api/signals/history` 返回废弃表数据 | 重定向到查 `signal_indicators` 表，或废弃此端点 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 附录：文件审阅覆盖情况
|
||||||
|
|
||||||
|
| 文件 | 行数 | 本次审阅 |
|
||||||
|
|------|-----|---------|
|
||||||
|
| `main.py` | ~500 | ✅ 全文 |
|
||||||
|
| `db.py` | ~415 | ✅ 全文 |
|
||||||
|
| `signal_engine.py` | ~1085 | ✅ 全文 |
|
||||||
|
| `live_executor.py` | ~708 | ✅ 全文 |
|
||||||
|
| `risk_guard.py` | ~644 | ✅ 全文 |
|
||||||
|
| `auth.py` | ~389 | ✅ 全文 |
|
||||||
|
| `position_sync.py` | ~687 | ✅ 全文 |
|
||||||
|
| `paper_monitor.py` | ~194 | ✅ 全文 |
|
||||||
|
| `agg_trades_collector.py` | ~400 | ✅ 全文 |
|
||||||
|
| `market_data_collector.py` | ~300 | ✅ 全文 |
|
||||||
|
| `liquidation_collector.py` | ~141 | ✅ 全文 |
|
||||||
|
| `signal_pusher.py` | ~100 | ✅ 全文 |
|
||||||
|
| `subscriptions.py` | ~24 | ✅ 全文 |
|
||||||
|
| `trade_config.py` | ~15 | ✅ 全文 |
|
||||||
|
| `backtest.py` | ~300 | 前100行 + 签名扫描 |
|
||||||
|
| `admin_cli.py` | ~100 | 签名扫描 |
|
||||||
145
docs/PROBLEM_REPORT.md
Normal file
145
docs/PROBLEM_REPORT.md
Normal file
@ -0,0 +1,145 @@
|
|||||||
|
# 项目问题报告
|
||||||
|
|
||||||
|
> 生成时间：2026-03-03
|
||||||
|
> 基于 commit `0d9dffa` 的代码分析
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔴 高危问题（可能导致实盘出错）
|
||||||
|
|
||||||
|
### P1：数据库从未真正迁移到云端
|
||||||
|
|
||||||
|
**现象**：你以为整个系统已经跑在 Cloud SQL 上，实际上只有 `agg_trades` 原始成交数据在双写。其他核心数据全在**本地 PG（127.0.0.1）**。
|
||||||
|
|
||||||
|
| 数据表 | 本地 PG | Cloud SQL |
|
||||||
|
|-------|---------|-----------|
|
||||||
|
| `agg_trades`（原始成交） | ✅ | ✅ 双写 |
|
||||||
|
| `signal_indicators`（信号输出） | ✅ | ❌ 没有 |
|
||||||
|
| `paper_trades`（模拟盘） | ✅ | ❌ 没有 |
|
||||||
|
| `rate_snapshots`（费率快照） | ✅ | ❌ 没有 |
|
||||||
|
| `market_indicators`（市场数据） | ✅ | ❌ 没有 |
|
||||||
|
| `live_trades`（实盘交易） | ❌ | ✅ 只在云端 |
|
||||||
|
| `live_config` / `live_events` | ❌ | ✅ 只在云端 |
|
||||||
|
|
||||||
|
**最致命的问题**：`live_executor.py` 和 `risk_guard.py` 默认连 Cloud SQL（`DB_HOST=10.106.0.3`），但 `signal_engine.py` 只把信号写到本地 PG。这意味着：
|
||||||
|
- 实盘执行器读取的 `signal_indicators` 表在 Cloud SQL 里**可能是空的**
|
||||||
|
- 风控模块监控的 `live_trades` 和信号引擎写的数据完全在两个不同的数据库里
|
||||||
|
|
||||||
|
**影响**：实盘交易链路存在断裂风险，需立即核查服务器上各进程实际连接的数据库地址。
|
||||||
|
|
||||||
|
**修复方向**：统一所有进程连接到 Cloud SQL，或统一连接到本地 PG（通过 Cloud SQL Auth Proxy）。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### P2：`users` 表双定义字段不一致
|
||||||
|
|
||||||
|
`db.py` 和 `auth.py` 各自定义了一个 `users` 表，字段不同：
|
||||||
|
|
||||||
|
| 字段 | db.py 版本 | auth.py 版本 |
|
||||||
|
|------|-----------|-------------|
|
||||||
|
| `email` | ✅ | ✅ |
|
||||||
|
| `password_hash` | ✅ | ✅ |
|
||||||
|
| `role` | ✅ | ✅ |
|
||||||
|
| `created_at` | ✅ | ✅ |
|
||||||
|
| `discord_id` | ❌ | ✅ |
|
||||||
|
| `banned` | ❌ | ✅ |
|
||||||
|
|
||||||
|
FastAPI 启动时先跑 `init_schema()`（db.py 版），再跑 `ensure_auth_tables()`（auth.py 版），因为 `CREATE TABLE IF NOT EXISTS` 第一次成功后就不再执行，**实际创建的是缺少 `discord_id` 和 `banned` 字段的旧版本**。
|
||||||
|
|
||||||
|
**影响**：封禁用户功能（`banned` 字段）在新装环境下可能失效。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### P3：`signal_indicators` 表 INSERT 包含 `strategy` 字段但 schema 没有
|
||||||
|
|
||||||
|
`save_indicator()` 函数向 `signal_indicators` 插入数据时包含 `strategy` 字段（`signal_engine.py:697`），但 `SCHEMA_SQL` 里的建表语句没有这个字段（`db.py:205-224`）。
|
||||||
|
|
||||||
|
**影响**：在全新环境初始化后，信号引擎写入会报列不存在的错误。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟡 中危问题（影响稳定性和维护）
|
||||||
|
|
||||||
|
### P4：`requirements.txt` 严重不完整
|
||||||
|
|
||||||
|
文件只列了 5 个包，实际运行还需要：
|
||||||
|
|
||||||
|
| 缺失依赖 | 用于 |
|
||||||
|
|---------|------|
|
||||||
|
| `asyncpg` | FastAPI 异步数据库 |
|
||||||
|
| `psycopg2-binary` | 同步数据库（signal_engine 等） |
|
||||||
|
| `aiohttp` | live_executor、risk_guard |
|
||||||
|
| `websockets` 或 `httpx` | agg_trades_collector WS 连接 |
|
||||||
|
| `psutil` | 已在文件里，但版本未锁定 |
|
||||||
|
|
||||||
|
**影响**：新机器部署直接失败。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### P5：`market_indicators` 和 `liquidations` 表不在主 schema 中
|
||||||
|
|
||||||
|
这两张表由各自的 collector 进程单独创建，不在 `init_schema()` 里。如果 collector 没跑过，signal_engine 查这两张表时会报错（会降级为默认中间分，不会崩溃，但数据不准）。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### P6：没有 CI/CD，没有自动化测试
|
||||||
|
|
||||||
|
- 代码变更完全靠人工验证
|
||||||
|
- 策略逻辑（`evaluate_signal`）没有任何单元测试，重构风险极高
|
||||||
|
- 部署流程：手动 ssh + git pull + pm2 restart
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟠 安全风险
|
||||||
|
|
||||||
|
### P7：测试网密码硬编码在源代码里
|
||||||
|
|
||||||
|
三个文件里都有：
|
||||||
|
```python
|
||||||
|
os.getenv("PG_PASS", "arb_engine_2026") # db.py:19
|
||||||
|
os.getenv("DB_PASSWORD", "arb_engine_2026") # live_executor.py:44
|
||||||
|
os.getenv("DB_PASSWORD", "arb_engine_2026") # risk_guard.py:42
|
||||||
|
```
|
||||||
|
|
||||||
|
代码一旦泄露（GitHub public、截图等），测试网数据库直接裸奔。
|
||||||
|
|
||||||
|
### P8：JWT Secret 有测试网默认值
|
||||||
|
|
||||||
|
```python
|
||||||
|
_jwt_default = "arb-engine-jwt-secret-v2-2026" if _TRADE_ENV == "testnet" else None
|
||||||
|
```
|
||||||
|
|
||||||
|
如果生产环境 `TRADE_ENV` 没有正确设置，会静默使用这个已知 secret，所有 JWT 都可伪造。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔵 架构债务（长期）
|
||||||
|
|
||||||
|
### P9：三套数据库连接配置并存，极易混淆
|
||||||
|
|
||||||
|
| 配置方式 | 使用的进程 | 默认连哪 |
|
||||||
|
|---------|----------|---------|
|
||||||
|
| `db.py` 的 `PG_HOST` | main.py、signal_engine、collectors | `127.0.0.1`（本地） |
|
||||||
|
| 进程内 `DB_HOST` | live_executor、risk_guard、position_sync | `10.106.0.3`（Cloud SQL） |
|
||||||
|
| `market_data_collector.py` 内 `PG_HOST` | market_data_collector | `127.0.0.1`（本地） |
|
||||||
|
|
||||||
|
没有统一的连接配置入口，每个进程各自读各自的环境变量，迁移时极容易漏改。
|
||||||
|
|
||||||
|
### P10：前端轮询压力
|
||||||
|
|
||||||
|
`/api/rates` 每 2 秒轮询一次，用户多了服务器压力线性增长。目前 3 秒缓存有一定缓冲，但没有限流保护。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 建议优先级
|
||||||
|
|
||||||
|
| 优先级 | 任务 |
|
||||||
|
|-------|------|
|
||||||
|
| 🔴 立即 | 登服务器确认各进程实际连的数据库地址，核查实盘链路是否完整 |
|
||||||
|
| 🔴 立即 | 补全 `signal_indicators` 表的 `strategy` 字段 |
|
||||||
|
| 🔴 本周 | 统一数据库连接配置，所有进程用同一套环境变量 |
|
||||||
|
| 🟡 本周 | 修复 `users` 表双定义问题，合并到 auth.py 版本 |
|
||||||
|
| 🟡 本周 | 补全 `requirements.txt` |
|
||||||
|
| 🟠 本月 | 把硬编码密码移到 `.env` 文件，不进代码仓库 |
|
||||||
|
| 🔵 长期 | 添加 signal_engine 核心逻辑的单元测试 |
|
||||||
|
| 🔵 长期 | 配置 GitHub Actions 做基础 lint 和安全扫描 |
|
||||||
117
docs/ai/00-system-overview.md
Normal file
117
docs/ai/00-system-overview.md
Normal file
@ -0,0 +1,117 @@
|
|||||||
|
---
|
||||||
|
generated_by: repo-insight
|
||||||
|
version: 1
|
||||||
|
created: 2026-03-03
|
||||||
|
last_updated: 2026-03-03
|
||||||
|
source_commit: 0d9dffa
|
||||||
|
coverage: standard
|
||||||
|
---
|
||||||
|
|
||||||
|
# 00 — System Overview
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
High-level description of the arbitrage-engine project: what it does, its tech stack, repo layout, and entry points.
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
- **Domain**: Crypto perpetual futures funding-rate arbitrage monitoring and short-term trading signal engine.
|
||||||
|
- **Strategy**: Hold spot long + perpetual short to collect funding rates every 8 h; plus a CVD/ATR-based short-term directional signal engine (V5.x).
|
||||||
|
- **Backend**: Python / FastAPI + independent PM2 worker processes; PostgreSQL (local + Cloud SQL dual-write).
|
||||||
|
- **Frontend**: Next.js 16 / React 19 / TypeScript SPA, charting via `lightweight-charts` + Recharts.
|
||||||
|
- **Targets**: BTC, ETH, XRP, SOL perpetual contracts on Binance USDC-M futures.
|
||||||
|
- **Deployment**: PM2 process manager on a GCP VM; frontend served via Next.js; backend accessible at `https://arb.zhouyangclaw.com`.
|
||||||
|
- **Auth**: JWT (access 24 h + refresh 7 d) + invite-code registration gating.
|
||||||
|
- **Trading modes**: Paper (simulated), Live (Binance Futures testnet or production via `TRADE_ENV`).
|
||||||
|
|
||||||
|
## Canonical Facts
|
||||||
|
|
||||||
|
### Repo Layout
|
||||||
|
```
|
||||||
|
arbitrage-engine/
|
||||||
|
├── backend/ # Python FastAPI API + all worker processes
|
||||||
|
│ ├── main.py # FastAPI app entry point (uvicorn)
|
||||||
|
│ ├── signal_engine.py # V5 signal engine (PM2 worker, 15 s loop)
|
||||||
|
│ ├── live_executor.py # Live trade executor (PM2 worker)
|
||||||
|
│ ├── risk_guard.py # Risk circuit-breaker (PM2 worker)
|
||||||
|
│ ├── market_data_collector.py # Binance WS market data (PM2 worker)
|
||||||
|
│ ├── agg_trades_collector.py # Binance aggTrades WS collector (PM2 worker)
|
||||||
|
│ ├── liquidation_collector.py # Binance liquidation WS collector (PM2 worker)
|
||||||
|
│ ├── signal_pusher.py # Discord signal notifier (PM2 worker)
|
||||||
|
│ ├── db.py # Dual-pool PostgreSQL layer (psycopg2 sync + asyncpg async)
|
||||||
|
│ ├── auth.py # JWT auth + invite-code registration router
|
||||||
|
│ ├── trade_config.py # Symbol / qty precision constants
|
||||||
|
│ ├── backtest.py # Offline backtest engine
|
||||||
|
│ ├── paper_monitor.py # Paper trade monitoring helper
|
||||||
|
│ ├── admin_cli.py # CLI for invite / user management
|
||||||
|
│ ├── subscriptions.py # Signal subscription query helper
|
||||||
|
│ ├── paper_config.json # Paper trading runtime toggle
|
||||||
|
│ ├── strategies/ # JSON strategy configs (v51_baseline, v52_8signals)
|
||||||
|
│ ├── ecosystem.dev.config.js # PM2 process definitions
|
||||||
|
│ └── logs/ # Rotating log files
|
||||||
|
├── frontend/ # Next.js app
|
||||||
|
│ ├── app/ # App Router pages
|
||||||
|
│ ├── components/ # Reusable UI components
|
||||||
|
│ ├── lib/api.ts # Typed API client
|
||||||
|
│ └── lib/auth.tsx # Auth context + token refresh logic
|
||||||
|
├── docs/ # Documentation (including docs/ai/)
|
||||||
|
├── scripts/ # Utility scripts
|
||||||
|
└── signal-engine.log # Live log symlink / output file
|
||||||
|
```
|
||||||
|
|
||||||
|
### Primary Language & Frameworks
|
||||||
|
| Layer | Technology |
|
||||||
|
|-------|-----------|
|
||||||
|
| Backend API | Python 3.x, FastAPI, uvicorn |
|
||||||
|
| DB access | asyncpg (async), psycopg2 (sync) |
|
||||||
|
| Frontend | TypeScript, Next.js 16, React 19 |
|
||||||
|
| Styling | Tailwind CSS v4 |
|
||||||
|
| Charts | lightweight-charts 5.x, Recharts 3.x |
|
||||||
|
| Process manager | PM2 (via `ecosystem.dev.config.js`) |
|
||||||
|
| Database | PostgreSQL (local + Cloud SQL dual-write) |
|
||||||
|
|
||||||
|
### Entry Points
|
||||||
|
| Process | File | Role |
|
||||||
|
|---------|------|------|
|
||||||
|
| HTTP API | `backend/main.py` | FastAPI on uvicorn |
|
||||||
|
| Signal engine | `backend/signal_engine.py` | 15 s indicator loop |
|
||||||
|
| Trade executor | `backend/live_executor.py` | PG NOTIFY listener → Binance API |
|
||||||
|
| Risk guard | `backend/risk_guard.py` | 5 s circuit-breaker loop |
|
||||||
|
| Market data | `backend/market_data_collector.py` | Binance WS → `market_indicators` table |
|
||||||
|
| aggTrades collector | `backend/agg_trades_collector.py` | Binance WS → `agg_trades` partitioned table |
|
||||||
|
| Liquidation collector | `backend/liquidation_collector.py` | Binance WS → liquidation tables |
|
||||||
|
| Signal pusher | `backend/signal_pusher.py` | DB → Discord push |
|
||||||
|
| Frontend | `frontend/` | Next.js dev/prod server |
|
||||||
|
|
||||||
|
### Monitored Symbols
|
||||||
|
`BTCUSDT`, `ETHUSDT`, `XRPUSDT`, `SOLUSDT` (Binance USDC-M Futures)
|
||||||
|
|
||||||
|
### Environment Variables (key ones)
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `PG_HOST` | `127.0.0.1` | Local PG host |
|
||||||
|
| `PG_DB` | `arb_engine` | Database name |
|
||||||
|
| `PG_USER` / `PG_PASS` | `arb` / `arb_engine_2026` | PG credentials |
|
||||||
|
| `CLOUD_PG_HOST` | `10.106.0.3` | Cloud SQL host |
|
||||||
|
| `CLOUD_PG_ENABLED` | `true` | Enable dual-write |
|
||||||
|
| `JWT_SECRET` | (testnet default set) | JWT signing key |
|
||||||
|
| `TRADE_ENV` | `testnet` | `testnet` or `production` |
|
||||||
|
| `LIVE_STRATEGIES` | `["v52_8signals"]` | Active live trading strategies |
|
||||||
|
| `RISK_PER_TRADE_USD` | `2` | USD risk per trade |
|
||||||
|
|
||||||
|
## Interfaces / Dependencies
|
||||||
|
- **External API**: Binance USDC-M Futures REST (`https://fapi.binance.com/fapi/v1`) and WebSocket.
|
||||||
|
- **Discord**: Webhook for signal notifications (via `signal_pusher.py`).
|
||||||
|
- **CORS origins**: `https://arb.zhouyangclaw.com`, `http://localhost:3000`, `http://localhost:3001`.
|
||||||
|
|
||||||
|
## Unknowns & Risks
|
||||||
|
- [inference] PM2 `ecosystem.dev.config.js` not read in this pass; exact process restart policies and env injection not confirmed.
|
||||||
|
- [inference] `.env` file usage confirmed via `python-dotenv` calls in live modules, but `.env.example` absent.
|
||||||
|
- [unknown] Deployment pipeline (CI/CD) not present in repo.
|
||||||
|
|
||||||
|
## Source Refs
|
||||||
|
- `backend/main.py:1-27` — FastAPI app init, CORS, SYMBOLS
|
||||||
|
- `backend/signal_engine.py:1-16` — V5 architecture docstring
|
||||||
|
- `backend/live_executor.py:1-10` — live executor architecture comment
|
||||||
|
- `backend/risk_guard.py:1-12` — risk guard circuit-break rules
|
||||||
|
- `backend/db.py:14-30` — PG/Cloud SQL env config
|
||||||
|
- `frontend/package.json` — frontend dependencies
|
||||||
|
- `frontend/lib/api.ts:1-116` — typed API client
|
||||||
137
docs/ai/01-architecture-map.md
Normal file
137
docs/ai/01-architecture-map.md
Normal file
@ -0,0 +1,137 @@
|
|||||||
|
---
|
||||||
|
generated_by: repo-insight
|
||||||
|
version: 1
|
||||||
|
created: 2026-03-03
|
||||||
|
last_updated: 2026-03-03
|
||||||
|
source_commit: 0d9dffa
|
||||||
|
coverage: standard
|
||||||
|
---
|
||||||
|
|
||||||
|
# 01 — Architecture Map
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
Describes the architecture style, component relationships, data flow, and runtime execution topology of the arbitrage engine.
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
- **Multi-process architecture**: each concern is a separate PM2 process; they communicate exclusively through PostgreSQL (tables + NOTIFY/LISTEN).
|
||||||
|
- **No message broker**: PostgreSQL serves as both the data store and the inter-process message bus (`NOTIFY new_signal`).
|
||||||
|
- **Dual-database write**: every PG write in `signal_engine.py` and `agg_trades_collector.py` attempts a secondary write to Cloud SQL (GCP) for durability.
|
||||||
|
- **FastAPI is read-only at runtime**: it proxies Binance REST for rates/history and reads the PG tables written by workers; it does not control the signal engine.
|
||||||
|
- **Signal pipeline**: raw aggTrades → in-memory rolling windows (CVD/VWAP/ATR) → scored signal → PG write + `NOTIFY` → live_executor executes Binance order.
|
||||||
|
- **Frontend polling**: React SPA polls `/api/rates` every 2 s (public) and slow endpoints every 120 s (auth required).
|
||||||
|
- **Risk guard is a separate process**: polls every 5 s, can block new orders (circuit-break) by writing a flag to `live_config`; live_executor reads that flag before each trade.
|
||||||
|
|
||||||
|
## Canonical Facts
|
||||||
|
|
||||||
|
### Architecture Style
|
||||||
|
Shared-DB multi-process monolith. No microservices; no message broker. All processes run on a single GCP VM.
|
||||||
|
|
||||||
|
### Component Diagram (text)
|
||||||
|
```
|
||||||
|
Binance WS (aggTrades)
|
||||||
|
└─► agg_trades_collector.py ──────────────────► agg_trades (partitioned table)
|
||||||
|
│
|
||||||
|
Binance WS (market data) ▼
|
||||||
|
└─► market_data_collector.py ──────────────► market_indicators table
|
||||||
|
│
|
||||||
|
Binance WS (liquidations) ▼
|
||||||
|
└─► liquidation_collector.py ──────────────► liquidation tables
|
||||||
|
│
|
||||||
|
signal_engine.py
|
||||||
|
(15 s loop, reads agg_trades +
|
||||||
|
market_indicators)
|
||||||
|
│
|
||||||
|
┌─────────┴──────────────┐
|
||||||
|
│ │
|
||||||
|
signal_indicators paper_trades
|
||||||
|
signal_indicators_1m (paper mode)
|
||||||
|
signal_trades
|
||||||
|
NOTIFY new_signal
|
||||||
|
│
|
||||||
|
live_executor.py
|
||||||
|
(LISTEN new_signal →
|
||||||
|
Binance Futures API)
|
||||||
|
│
|
||||||
|
live_trades table
|
||||||
|
│
|
||||||
|
risk_guard.py (5 s)
|
||||||
|
monitors live_trades,
|
||||||
|
writes live_config flags
|
||||||
|
│
|
||||||
|
signal_pusher.py
|
||||||
|
(reads signal_indicators →
|
||||||
|
Discord webhook)
|
||||||
|
│
|
||||||
|
FastAPI main.py (read/proxy)
|
||||||
|
+ rate_snapshots (2 s write)
|
||||||
|
│
|
||||||
|
Next.js Frontend
|
||||||
|
(polling SPA)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Data Flow — Signal Pipeline
|
||||||
|
1. `agg_trades_collector.py`: streams `aggTrade` WS events for all symbols, batch-inserts into `agg_trades` partitioned table (partitioned by month on `time_ms`).
|
||||||
|
2. `signal_engine.py` (15 s loop per symbol):
|
||||||
|
- Cold-start: reads last N rows from `agg_trades` to warm up `TradeWindow` (CVD, VWAP) and `ATRCalculator` deques.
|
||||||
|
- Fetches new trades since `last_agg_id`.
|
||||||
|
- Feeds trades into three `TradeWindow` instances (30 m, 4 h, 24 h) and one `ATRCalculator` (5 m candles, 14-period).
|
||||||
|
- Reads `market_indicators` for long-short ratio, OI, coinbase premium, funding rate, liquidations.
|
||||||
|
- Scores signal using JSON strategy config weights (score 0–100, threshold 75).
|
||||||
|
- Writes to `signal_indicators` (15 s cadence) and `signal_indicators_1m` (1 m cadence).
|
||||||
|
- If score ≥ threshold: opens paper trade (if enabled), emits `NOTIFY new_signal` (if live enabled).
|
||||||
|
3. `live_executor.py`: `LISTEN new_signal` on PG; deserializes payload; calls Binance Futures REST to place market order; writes to `live_trades`.
|
||||||
|
4. `risk_guard.py`: every 5 s checks daily loss, consecutive losses, unrealized PnL, balance, data freshness, hold timeout; sets `live_config.circuit_break` flag to block/resume new orders.
|
||||||
|
|
||||||
|
### Strategy Scoring (V5.x)
|
||||||
|
Two JSON configs in `backend/strategies/`:
|
||||||
|
|
||||||
|
| Config | Version | Threshold | Signals |
|
||||||
|
|--------|---------|-----------|---------|
|
||||||
|
| `v51_baseline.json` | 5.1 | 75 | cvd, p99, accel, ls_ratio, oi, coinbase_premium |
|
||||||
|
| `v52_8signals.json` | 5.2 | 75 | cvd, p99, accel, ls_ratio, oi, coinbase_premium, funding_rate, liquidation |
|
||||||
|
|
||||||
|
Score categories: `direction` (CVD), `crowding` (P99 large trades), `environment` (ATR/VWAP), `confirmation` (LS ratio, OI), `auxiliary` (coinbase premium), `funding_rate`, `liquidation`.
|
||||||
|
|
||||||
|
### TP/SL Configuration
|
||||||
|
- V5.1: SL=1.4×ATR, TP1=1.05×ATR, TP2=2.1×ATR
|
||||||
|
- V5.2: SL=2.1×ATR, TP1=1.4×ATR, TP2=3.15×ATR
|
||||||
|
- Signal cooldown: 10 minutes per symbol per direction.
|
||||||
|
|
||||||
|
### Risk Guard Circuit-Break Rules
|
||||||
|
| Rule | Threshold | Action |
|
||||||
|
|------|-----------|--------|
|
||||||
|
| Daily loss | -5R | Full close + shutdown |
|
||||||
|
| Consecutive losses | 5 | Pause 60 min |
|
||||||
|
| API disconnect | >30 s | Pause new orders |
|
||||||
|
| Balance too low | < risk×2 | Reject new orders |
|
||||||
|
| Data stale | >30 s | Block new orders |
|
||||||
|
| Hold timeout yellow | 45 min | Alert |
|
||||||
|
| Hold timeout auto-close | 70 min | Force close |
|
||||||
|
|
||||||
|
### Frontend Architecture
|
||||||
|
- **Next.js App Router** (`frontend/app/`): page-per-route, all pages are client components (`"use client"`).
|
||||||
|
- **Auth**: JWT stored in `localStorage`; `lib/auth.tsx` provides `useAuth()` hook + `authFetch()` helper with auto-refresh.
|
||||||
|
- **API client**: `lib/api.ts` — typed wrapper, distinguishes public (`/api/rates`, `/api/health`) from protected (all other) endpoints.
|
||||||
|
- **Polling strategy**: rates every 2 s, slow data (stats, history, signals) every 120 s; kline charts re-render every 30 s.
|
||||||
|
|
||||||
|
## Interfaces / Dependencies
|
||||||
|
- PG NOTIFY channel name: `new_signal`
|
||||||
|
- `live_config` table keys: `risk_per_trade_usd`, `max_positions`, `circuit_break` (inferred)
|
||||||
|
- `market_indicators` populated by `market_data_collector.py` with types: `long_short_ratio`, `top_trader_position`, `open_interest_hist`, `coinbase_premium`, `funding_rate`
|
||||||
|
|
||||||
|
## Unknowns & Risks
|
||||||
|
- [inference] PM2 config (`ecosystem.dev.config.js`) not read; exact restart/watch/env-file settings unknown.
|
||||||
|
- [inference] `signal_pusher.py` exact Discord webhook configuration (env var name, rate limit handling) not confirmed.
|
||||||
|
- [unknown] Cloud SQL write failure does not block signal_engine but may create data divergence between local PG and Cloud SQL.
|
||||||
|
- [risk] Hardcoded testnet credentials in source code (`arb_engine_2026`); production requires explicit env var override.
|
||||||
|
|
||||||
|
## Source Refs
|
||||||
|
- `backend/signal_engine.py:1-16` — architecture docstring
|
||||||
|
- `backend/live_executor.py:1-10` — executor architecture comment
|
||||||
|
- `backend/risk_guard.py:1-12, 55-73` — risk rules and config
|
||||||
|
- `backend/signal_engine.py:170-245` — `TradeWindow`, `ATRCalculator` classes
|
||||||
|
- `backend/signal_engine.py:44-67` — strategy config loading
|
||||||
|
- `backend/strategies/v51_baseline.json`, `backend/strategies/v52_8signals.json`
|
||||||
|
- `backend/main.py:61-83` — background snapshot loop
|
||||||
|
- `frontend/lib/api.ts:103-116` — API client methods
|
||||||
|
- `frontend/app/page.tsx:149-154` — polling intervals
|
||||||
218
docs/ai/02-module-cheatsheet.md
Normal file
218
docs/ai/02-module-cheatsheet.md
Normal file
@ -0,0 +1,218 @@
|
|||||||
|
---
|
||||||
|
generated_by: repo-insight
|
||||||
|
version: 1
|
||||||
|
created: 2026-03-03
|
||||||
|
last_updated: 2026-03-03
|
||||||
|
source_commit: 0d9dffa
|
||||||
|
coverage: standard
|
||||||
|
---
|
||||||
|
|
||||||
|
# 02 — Module Cheatsheet
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
Module-by-module index: file path, role, key public interfaces, and dependencies.
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
- Backend has 20 Python modules; signal_engine.py is the largest and most complex (~1000+ lines).
|
||||||
|
- Frontend has 2 TypeScript lib files + 9 pages + 6 components.
|
||||||
|
- `db.py` is the only shared infrastructure module; all other backend modules import from it.
|
||||||
|
- `signal_engine.py` is the core business logic module; `live_executor.py` and `risk_guard.py` are independent processes that only use `db.py` and direct PG connections.
|
||||||
|
- Strategy configs are external JSON; no code changes needed to tune weights/thresholds.
|
||||||
|
|
||||||
|
## Canonical Facts
|
||||||
|
|
||||||
|
### Backend Modules
|
||||||
|
|
||||||
|
#### `backend/main.py` — FastAPI HTTP API
|
||||||
|
- **Role**: Primary HTTP API server; rate/snapshot/history proxy; aggTrade query endpoints; signal history.
|
||||||
|
- **Key interfaces**:
|
||||||
|
- `GET /api/health` — liveness check (public)
|
||||||
|
- `GET /api/rates` — live Binance premiumIndex for 4 symbols (public, 3 s cache)
|
||||||
|
- `GET /api/snapshots` — rate snapshot history from PG (auth required)
|
||||||
|
- `GET /api/kline` — candlestick bars aggregated from `rate_snapshots` (auth required)
|
||||||
|
- `GET /api/stats` — 7-day funding rate stats per symbol (auth required, 60 s cache)
|
||||||
|
- `GET /api/stats/ytd` — YTD annualized stats (auth required, 3600 s cache)
|
||||||
|
- `GET /api/history` — 7-day raw funding rate history (auth required, 60 s cache)
|
||||||
|
- `GET /api/signals/history` — `signal_logs` table (auth required)
|
||||||
|
- `GET /api/trades/meta` — `agg_trades_meta` (auth required)
|
||||||
|
- `GET /api/trades/summary` — aggregated OHLCV from `agg_trades` (auth required)
|
||||||
|
- Many more: paper trades, signals v52, live trades, live config, position sync, etc. (full list in saved tool output)
|
||||||
|
- **Deps**: `auth.py`, `db.py`, `httpx`, `asyncio`
|
||||||
|
- **Background task**: `background_snapshot_loop()` writes `rate_snapshots` every 2 s.
|
||||||
|
|
||||||
|
#### `backend/signal_engine.py` — V5 Signal Engine (PM2 worker)
|
||||||
|
- **Role**: Core signal computation loop; 15 s interval; in-memory rolling-window indicators; scored signal output.
|
||||||
|
- **Key classes**:
|
||||||
|
- `TradeWindow(window_ms)` — rolling CVD/VWAP calculator using `deque`; props: `cvd`, `vwap`
|
||||||
|
- `ATRCalculator(period_ms, length)` — 5-min candle ATR; props: `atr`, `atr_percentile`
|
||||||
|
- `SymbolState` — per-symbol state container holding `TradeWindow` ×3, `ATRCalculator`, large-order percentile deques
|
||||||
|
- **Key functions**:
|
||||||
|
- `load_strategy_configs() -> list[dict]` — reads JSON files from `strategies/`
|
||||||
|
- `fetch_market_indicators(symbol) -> dict` — reads `market_indicators` table
|
||||||
|
- `fetch_new_trades(symbol, last_id) -> list` — reads new rows from `agg_trades`
|
||||||
|
- `save_indicator(ts, symbol, result, strategy)` — writes to `signal_indicators`
|
||||||
|
- `paper_open_trade(...)` — inserts `paper_trades` row
|
||||||
|
- `paper_check_positions(symbol, price, now_ms)` — checks TP/SL for paper positions
|
||||||
|
- `main()` — entry point; calls `load_historical()` then enters main loop
|
||||||
|
- **Deps**: `db.py`, `json`, `collections.deque`
|
||||||
|
|
||||||
|
#### `backend/live_executor.py` — Live Trade Executor (PM2 worker)
|
||||||
|
- **Role**: Listens on PG `NOTIFY new_signal`; places Binance Futures market orders; writes `live_trades`.
|
||||||
|
- **Key functions**:
|
||||||
|
- `reload_live_config(conn)` — refreshes `RISK_PER_TRADE_USD`, `MAX_POSITIONS` from `live_config` every 60 s
|
||||||
|
- `binance_request(session, method, path, params)` — HMAC-signed Binance API call
|
||||||
|
- **Config**: `TRADE_ENV` (`testnet`/`production`), `LIVE_STRATEGIES`, `RISK_PER_TRADE_USD`, `MAX_POSITIONS`
|
||||||
|
- **Deps**: `psycopg2`, `aiohttp`, HMAC signing
|
||||||
|
|
||||||
|
#### `backend/risk_guard.py` — Risk Circuit-Breaker (PM2 worker)
|
||||||
|
- **Role**: Every 5 s; monitors PnL, balance, data freshness, hold timeouts; writes circuit-break flags.
|
||||||
|
- **Key classes**: `RiskState` — holds `status` (`normal`/`warning`/`circuit_break`), loss counters
|
||||||
|
- **Key functions**:
|
||||||
|
- `check_daily_loss(conn)` — sums `pnl_r` from today's `live_trades`
|
||||||
|
- `check_unrealized_loss(session, risk_usd_dynamic)` — queries Binance positions API
|
||||||
|
- `check_balance(session)` — queries Binance account balance
|
||||||
|
- `check_data_freshness(conn)` — checks `market_indicators` recency
|
||||||
|
- `check_hold_timeout(session, conn)` — force-closes positions held >70 min
|
||||||
|
- `trigger_circuit_break(session, conn, reason, action)` — writes to `live_events`, may flat positions
|
||||||
|
- `check_auto_resume()` — re-enables trading after cooldown
|
||||||
|
- `check_emergency_commands(session, conn)` — watches for manual DB commands
|
||||||
|
- **Deps**: `trade_config.py`, `aiohttp`, `psycopg2`
|
||||||
|
|
||||||
|
#### `backend/db.py` — Database Layer
|
||||||
|
- **Role**: All PG connectivity; schema creation; partition management.
|
||||||
|
- **Key interfaces**:
|
||||||
|
- Sync (psycopg2): `get_sync_conn()`, `sync_execute()`, `sync_executemany()`
|
||||||
|
- Async (asyncpg): `async_fetch()`, `async_fetchrow()`, `async_execute()`
|
||||||
|
- Cloud SQL sync pool: `get_cloud_sync_conn()` (non-fatal on failure)
|
||||||
|
- `init_schema()` — creates all tables from `SCHEMA_SQL`
|
||||||
|
- `ensure_partitions()` — creates `agg_trades_YYYYMM` partitions for current+next 2 months
|
||||||
|
- **Deps**: `asyncpg`, `psycopg2`
|
||||||
|
|
||||||
|
#### `backend/auth.py` — JWT Auth + Registration
|
||||||
|
- **Role**: FastAPI router at `/api`; register/login/refresh/logout endpoints.
|
||||||
|
- **Key interfaces**:
|
||||||
|
- `POST /api/register` — invite-code gated registration
|
||||||
|
- `POST /api/login` — returns `access_token` + `refresh_token`
|
||||||
|
- `POST /api/refresh` — token refresh
|
||||||
|
- `POST /api/logout` — revokes refresh token
|
||||||
|
- `GET /api/me` — current user info
|
||||||
|
- `get_current_user` — FastAPI `Depends` injector; validates Bearer JWT
|
||||||
|
- **Token storage**: HMAC-SHA256 hand-rolled JWT (no PyJWT); refresh tokens stored in `refresh_tokens` table.
|
||||||
|
- **Deps**: `db.py`, `hashlib`, `hmac`, `secrets`
|
||||||
|
|
||||||
|
#### `backend/agg_trades_collector.py` — AggTrades Collector (PM2 worker)
|
||||||
|
- **Role**: Streams Binance `aggTrade` WebSocket events; batch-inserts into `agg_trades` partitioned table; maintains `agg_trades_meta`.
|
||||||
|
- **Key functions**: `ws_collect(symbol)`, `rest_catchup(symbol, from_id)`, `continuity_check()`, `flush_buffer(symbol, trades)`
|
||||||
|
- **Deps**: `db.py`, `websockets`/`httpx`
|
||||||
|
|
||||||
|
#### `backend/market_data_collector.py` — Market Data Collector (PM2 worker)
|
||||||
|
- **Role**: Collects Binance market indicators (LS ratio, OI, coinbase premium, funding rate) via REST polling; stores in `market_indicators` JSONB.
|
||||||
|
- **Key class**: `MarketDataCollector`
|
||||||
|
- **Deps**: `db.py`, `httpx`
|
||||||
|
|
||||||
|
#### `backend/liquidation_collector.py` — Liquidation Collector (PM2 worker)
|
||||||
|
- **Role**: Streams Binance liquidation WS; aggregates into `liquidation_events` and `liquidation_agg` tables.
|
||||||
|
- **Key functions**: `ensure_table()`, `save_liquidation()`, `save_aggregated()`, `run()`
|
||||||
|
- **Deps**: `db.py`, `websockets`
|
||||||
|
|
||||||
|
#### `backend/backtest.py` — Offline Backtester
|
||||||
|
- **Role**: Replays `agg_trades` from PG to simulate signal engine and measure strategy performance.
|
||||||
|
- **Key classes**: `Position`, `BacktestEngine`
|
||||||
|
- **Key functions**: `load_trades()`, `run_backtest()`, `main()`
|
||||||
|
- **Deps**: `db.py`
|
||||||
|
|
||||||
|
#### `backend/trade_config.py` — Symbol / Qty Config
|
||||||
|
- **Role**: Constants for symbols and Binance qty precision.
|
||||||
|
- **Deps**: none
|
||||||
|
|
||||||
|
#### `backend/admin_cli.py` — Admin CLI
|
||||||
|
- **Role**: CLI for invite-code and user management (gen_invite, list_invites, ban_user, set_admin).
|
||||||
|
- **Deps**: `db.py`
|
||||||
|
|
||||||
|
#### `backend/subscriptions.py` — Subscription Query Helper
|
||||||
|
- **Role**: Helpers for querying signal history (used internally).
|
||||||
|
- **Deps**: `db.py`
|
||||||
|
|
||||||
|
#### `backend/paper_monitor.py` — Paper Trade Monitor
|
||||||
|
- **Role**: Standalone script to print paper trade status.
|
||||||
|
- **Deps**: `db.py`
|
||||||
|
|
||||||
|
#### `backend/signal_pusher.py` — Discord Notifier (PM2 worker)
|
||||||
|
- **Role**: Polls `signal_indicators` for high-score events; pushes Discord webhook notifications.
|
||||||
|
- **Deps**: `db.py`, `httpx`
|
||||||
|
|
||||||
|
#### `backend/position_sync.py` — Position Sync
|
||||||
|
- **Role**: Syncs live positions between `live_trades` table and Binance account state.
|
||||||
|
- **Deps**: `db.py`, `aiohttp`
|
||||||
|
|
||||||
|
#### `backend/fix_historical_pnl.py` — PnL Fix Script
|
||||||
|
- **Role**: One-time migration to recalculate historical PnL in `paper_trades`.
|
||||||
|
- **Deps**: `db.py`
|
||||||
|
|
||||||
|
### Frontend Modules
|
||||||
|
|
||||||
|
#### `frontend/lib/api.ts` — API Client
|
||||||
|
- **Role**: Typed `api` object with all backend endpoint wrappers; distinguishes public vs. protected fetches.
|
||||||
|
- **Interfaces exported**: `RateData`, `RatesResponse`, `HistoryPoint`, `HistoryResponse`, `StatsResponse`, `SignalHistoryItem`, `SnapshotItem`, `KBar`, `KlineResponse`, `YtdStatsResponse`, `api` object
|
||||||
|
- **Deps**: `lib/auth.tsx` (`authFetch`)
|
||||||
|
|
||||||
|
#### `frontend/lib/auth.tsx` — Auth Context
|
||||||
|
- **Role**: React context for current user; `useAuth()` hook; `authFetch()` with access-token injection and auto-refresh.
|
||||||
|
- **Deps**: Next.js router, `localStorage`
|
||||||
|
|
||||||
|
#### `frontend/app/` Pages
|
||||||
|
| Page | Route | Description |
|
||||||
|
|------|-------|-------------|
|
||||||
|
| `page.tsx` | `/` | Main dashboard: rates, kline, history, signal log |
|
||||||
|
| `dashboard/page.tsx` | `/dashboard` | (inferred) extended dashboard |
|
||||||
|
| `signals/page.tsx` | `/signals` | Signal history view (V5.1) |
|
||||||
|
| `signals-v52/page.tsx` | `/signals-v52` | Signal history view (V5.2) |
|
||||||
|
| `paper/page.tsx` | `/paper` | Paper trades view (V5.1) |
|
||||||
|
| `paper-v52/page.tsx` | `/paper-v52` | Paper trades view (V5.2) |
|
||||||
|
| `live/page.tsx` | `/live` | Live trades view |
|
||||||
|
| `history/page.tsx` | `/history` | Funding rate history |
|
||||||
|
| `kline/page.tsx` | `/kline` | Kline chart page |
|
||||||
|
| `trades/page.tsx` | `/trades` | aggTrades summary |
|
||||||
|
| `server/page.tsx` | `/server` | Server status / metrics |
|
||||||
|
| `about/page.tsx` | `/about` | About page |
|
||||||
|
| `login/page.tsx` | `/login` | Login form |
|
||||||
|
| `register/page.tsx` | `/register` | Registration form |
|
||||||
|
|
||||||
|
#### `frontend/components/`
|
||||||
|
| Component | Role |
|
||||||
|
|-----------|------|
|
||||||
|
| `Navbar.tsx` | Top navigation bar |
|
||||||
|
| `Sidebar.tsx` | Sidebar navigation |
|
||||||
|
| `AuthHeader.tsx` | Auth-aware header with user info |
|
||||||
|
| `RateCard.tsx` | Displays current funding rate for one asset |
|
||||||
|
| `StatsCard.tsx` | Displays 7d mean and annualized stats |
|
||||||
|
| `FundingChart.tsx` | Funding rate chart component |
|
||||||
|
| `LiveTradesCard.tsx` | Live trades summary card |
|
||||||
|
|
||||||
|
## Interfaces / Dependencies
|
||||||
|
|
||||||
|
### Key import graph (backend)
|
||||||
|
```
|
||||||
|
main.py → auth.py, db.py
|
||||||
|
signal_engine.py → db.py
|
||||||
|
live_executor.py → psycopg2 direct (no db.py module import)
|
||||||
|
risk_guard.py → trade_config.py, psycopg2 direct
|
||||||
|
backtest.py → db.py
|
||||||
|
agg_trades_collector.py → db.py
|
||||||
|
market_data_collector.py → db.py
|
||||||
|
liquidation_collector.py → db.py
|
||||||
|
admin_cli.py → db.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Unknowns & Risks
|
||||||
|
- [inference] Content of `frontend/app/dashboard/`, `signals/`, `paper/`, `live/` pages not read; role described from filename convention.
|
||||||
|
- [unknown] `signal_pusher.py` Discord webhook env var name not confirmed.
|
||||||
|
- [inference] `position_sync.py` exact interface not read; role inferred from name and listing.
|
||||||
|
|
||||||
|
## Source Refs
|
||||||
|
- `backend/main.py` — all API route definitions
|
||||||
|
- `backend/signal_engine.py:170-285` — `TradeWindow`, `ATRCalculator`, `SymbolState`
|
||||||
|
- `backend/auth.py:23-23` — router prefix `/api`
|
||||||
|
- `backend/db.py:35-157` — all public DB functions
|
||||||
|
- `frontend/lib/api.ts:103-116` — `api` export object
|
||||||
|
- `frontend/lib/auth.tsx` — auth context (not fully read)
|
||||||
242
docs/ai/03-api-contracts.md
Normal file
242
docs/ai/03-api-contracts.md
Normal file
@ -0,0 +1,242 @@
|
|||||||
|
---
|
||||||
|
generated_by: repo-insight
|
||||||
|
version: 1
|
||||||
|
created: 2026-03-03
|
||||||
|
last_updated: 2026-03-03
|
||||||
|
source_commit: 0d9dffa
|
||||||
|
coverage: standard
|
||||||
|
---
|
||||||
|
|
||||||
|
# 03 — API Contracts
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
Documents all REST API endpoints, authentication requirements, request/response shapes, and error conventions.
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
- Base URL: `https://arb.zhouyangclaw.com` (prod) or `http://localhost:8000` (local).
|
||||||
|
- Auth: Bearer JWT in `Authorization` header. Two public endpoints (`/api/health`, `/api/rates`) need no token.
|
||||||
|
- Token lifecycle: access token 24 h, refresh token 7 d; use `POST /api/refresh` to renew.
|
||||||
|
- Registration is invite-code gated: must supply a valid `invite_code` in register body.
|
||||||
|
- All timestamps are Unix epoch (seconds or ms depending on field; see per-endpoint notes).
|
||||||
|
- Funding rates are stored as decimals (e.g. `0.0001` = 0.01%). Frontend multiplies by 10000 for "万分之" display.
|
||||||
|
- Error responses: standard FastAPI `{"detail": "..."}` with appropriate HTTP status codes.
|
||||||
|
|
||||||
|
## Canonical Facts
|
||||||
|
|
||||||
|
### Authentication
|
||||||
|
|
||||||
|
#### `POST /api/register`
|
||||||
|
```json
|
||||||
|
// Request
|
||||||
|
{
|
||||||
|
"email": "user@example.com",
|
||||||
|
"password": "...",
|
||||||
|
"invite_code": "XXXX"
|
||||||
|
}
|
||||||
|
// Response 200
|
||||||
|
{
|
||||||
|
"access_token": "<jwt>",
|
||||||
|
"refresh_token": "<token>",
|
||||||
|
"token_type": "bearer",
|
||||||
|
"user": { "id": 1, "email": "...", "role": "user" }
|
||||||
|
}
|
||||||
|
// Errors: 400 (invite invalid/expired), 409 (email taken)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `POST /api/login`
|
||||||
|
```json
|
||||||
|
// Request
|
||||||
|
{
|
||||||
|
"email": "user@example.com",
|
||||||
|
"password": "..."
|
||||||
|
}
|
||||||
|
// Response 200
|
||||||
|
{
|
||||||
|
"access_token": "<jwt>",
|
||||||
|
"refresh_token": "<token>",
|
||||||
|
"token_type": "bearer",
|
||||||
|
"user": { "id": 1, "email": "...", "role": "user" }
|
||||||
|
}
|
||||||
|
// Errors: 401 (invalid credentials), 403 (banned)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `POST /api/refresh`
|
||||||
|
```json
|
||||||
|
// Request
|
||||||
|
{ "refresh_token": "<token>" }
|
||||||
|
// Response 200
|
||||||
|
{ "access_token": "<new_jwt>", "token_type": "bearer" }
|
||||||
|
// Errors: 401 (expired/revoked)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `POST /api/logout`
|
||||||
|
```json
|
||||||
|
// Request header: Authorization: Bearer <access_token>
|
||||||
|
// Request body: { "refresh_token": "<token>" }
|
||||||
|
// Response 200: { "ok": true }
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `GET /api/me`
|
||||||
|
```json
|
||||||
|
// Auth required
|
||||||
|
// Response 200
|
||||||
|
{ "id": 1, "email": "...", "role": "user", "created_at": "..." }
|
||||||
|
```
|
||||||
|
|
||||||
|
### Public Endpoints (no auth)
|
||||||
|
|
||||||
|
#### `GET /api/health`
|
||||||
|
```json
|
||||||
|
{ "status": "ok", "timestamp": "2026-03-03T12:00:00" }
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `GET /api/rates`
|
||||||
|
Returns live Binance premiumIndex for BTCUSDT, ETHUSDT, XRPUSDT, SOLUSDT. Cached 3 s.
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"BTC": {
|
||||||
|
"symbol": "BTCUSDT",
|
||||||
|
"markPrice": 65000.0,
|
||||||
|
"indexPrice": 64990.0,
|
||||||
|
"lastFundingRate": 0.0001,
|
||||||
|
"nextFundingTime": 1234567890000,
|
||||||
|
"timestamp": 1234567890000
|
||||||
|
},
|
||||||
|
"ETH": { ... },
|
||||||
|
"XRP": { ... },
|
||||||
|
"SOL": { ... }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Protected Endpoints (Bearer JWT required)
|
||||||
|
|
||||||
|
#### `GET /api/history`
|
||||||
|
7-day funding rate history from Binance. Cached 60 s.
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"BTC": [
|
||||||
|
{ "fundingTime": 1234567890000, "fundingRate": 0.0001, "timestamp": "2026-03-01T08:00:00" }
|
||||||
|
],
|
||||||
|
"ETH": [ ... ], "XRP": [ ... ], "SOL": [ ... ]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `GET /api/stats`
|
||||||
|
7-day funding rate statistics. Cached 60 s.
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"BTC": { "mean7d": 0.01, "annualized": 10.95, "count": 21 },
|
||||||
|
"ETH": { ... },
|
||||||
|
"combo": { "mean7d": 0.009, "annualized": 9.85 }
|
||||||
|
}
|
||||||
|
// mean7d in %; annualized = mean * 3 * 365 * 100
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `GET /api/stats/ytd`
|
||||||
|
Year-to-date annualized stats. Cached 3600 s.
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"BTC": { "annualized": 12.5, "count": 150 },
|
||||||
|
"ETH": { ... }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `GET /api/snapshots?hours=24&limit=5000`
|
||||||
|
Rate snapshots from local PG.
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"count": 43200,
|
||||||
|
"hours": 24,
|
||||||
|
"data": [
|
||||||
|
{ "ts": 1709000000, "btc_rate": 0.0001, "eth_rate": 0.00008, "btc_price": 65000, "eth_price": 3200 }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `GET /api/kline?symbol=BTC&interval=1h&limit=500`
|
||||||
|
Candlestick bars derived from `rate_snapshots`. Rates scaled by ×10000.
|
||||||
|
- `interval`: `1m`, `5m`, `30m`, `1h`, `4h`, `8h`, `1d`, `1w`, `1M`
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"symbol": "BTC",
|
||||||
|
"interval": "1h",
|
||||||
|
"count": 24,
|
||||||
|
"data": [
|
||||||
|
{
|
||||||
|
"time": 1709000000,
|
||||||
|
"open": 1.0, "high": 1.2, "low": 0.8, "close": 1.1,
|
||||||
|
"price_open": 65000, "price_high": 65500, "price_low": 64800, "price_close": 65200
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `GET /api/signals/history?limit=100`
|
||||||
|
Legacy signal log from `signal_logs` table.
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"items": [
|
||||||
|
{ "id": 1, "symbol": "BTCUSDT", "rate": 0.0001, "annualized": 10.95, "sent_at": "2026-03-01T08:00:00", "message": "..." }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `GET /api/trades/meta`
|
||||||
|
aggTrades collection status.
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"BTC": { "last_agg_id": 123456789, "last_time_ms": 1709000000000, "updated_at": "2026-03-03 12:00:00" }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `GET /api/trades/summary?symbol=BTC&start_ms=0&end_ms=0&interval=1m`
|
||||||
|
Aggregated OHLCV from `agg_trades` via PG native aggregation.
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"symbol": "BTC",
|
||||||
|
"interval": "1m",
|
||||||
|
"count": 60,
|
||||||
|
"data": [
|
||||||
|
{ "bar_ms": 1709000000000, "buy_vol": 10.5, "sell_vol": 9.3, "trade_count": 45, "vwap": 65000.0, "max_qty": 2.5 }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Signal V52 Endpoints (inferred from frontend routes)
|
||||||
|
- `GET /api/signals/v52` — signals for v52_8signals strategy
|
||||||
|
- `GET /api/paper/trades` — paper trade history
|
||||||
|
- `GET /api/paper/trades/v52` — v52 paper trade history
|
||||||
|
- `GET /api/live/trades` — live trade history
|
||||||
|
- `GET /api/live/config` — current live config
|
||||||
|
- `GET /api/live/events` — live trading event log
|
||||||
|
- `GET /api/server/stats` — server process stats (psutil)
|
||||||
|
|
||||||
|
### Auth Header Format
|
||||||
|
```
|
||||||
|
Authorization: Bearer <access_token>
|
||||||
|
```
|
||||||
|
Frontend auto-injects via `authFetch()` in `lib/auth.tsx`. On 401, attempts token refresh before retry.
|
||||||
|
|
||||||
|
### Error Shape
|
||||||
|
All errors follow FastAPI default:
|
||||||
|
```json
|
||||||
|
{ "detail": "Human-readable error message" }
|
||||||
|
```
|
||||||
|
Common HTTP status codes: 400 (bad request), 401 (unauthorized), 403 (forbidden/banned), 404 (not found), 422 (validation error), 502 (Binance upstream error).
|
||||||
|
|
||||||
|
## Interfaces / Dependencies
|
||||||
|
- Binance USDC-M Futures REST: `https://fapi.binance.com/fapi/v1/premiumIndex`, `/fundingRate`
|
||||||
|
- CORS allowed origins: `https://arb.zhouyangclaw.com`, `http://localhost:3000`, `http://localhost:3001`
|
||||||
|
- `NEXT_PUBLIC_API_URL` env var controls the frontend base URL (empty = same-origin)
|
||||||
|
|
||||||
|
## Unknowns & Risks
|
||||||
|
- [inference] Full endpoint list for signals-v52, paper-v52, live, server pages not confirmed by reading main.py lines 300+. The full saved output contains more routes.
|
||||||
|
- [inference] `POST /api/register` exact field validation (password min length, etc.) not confirmed.
|
||||||
|
- [risk] No rate limiting visible on public endpoints; `/api/rates` with 3 s cache could be bypassed by direct calls.
|
||||||
|
|
||||||
|
## Source Refs
|
||||||
|
- `backend/main.py:101-298` — all confirmed REST endpoints
|
||||||
|
- `backend/auth.py:23` — auth router prefix
|
||||||
|
- `backend/main.py:16-21` — CORS config
|
||||||
|
- `frontend/lib/api.ts:90-116` — client-side API wrappers
|
||||||
|
- `frontend/lib/auth.tsx` — `authFetch` with auto-refresh (not fully read)
|
||||||
301
docs/ai/04-data-model.md
Normal file
301
docs/ai/04-data-model.md
Normal file
@ -0,0 +1,301 @@
|
|||||||
|
---
|
||||||
|
generated_by: repo-insight
|
||||||
|
version: 1
|
||||||
|
created: 2026-03-03
|
||||||
|
last_updated: 2026-03-03
|
||||||
|
source_commit: 0d9dffa
|
||||||
|
coverage: standard
|
||||||
|
---
|
||||||
|
|
||||||
|
# 04 — Data Model
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
Documents all PostgreSQL tables, columns, relations, constraints, storage design, and partitioning strategy.
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
- Single PostgreSQL database `arb_engine`; 15+ tables defined in `db.py` `SCHEMA_SQL` + `auth.py` `AUTH_SCHEMA`.
|
||||||
|
- `agg_trades` is a range-partitioned table (by `time_ms` in milliseconds); monthly partitions auto-created by `ensure_partitions()`.
|
||||||
|
- Dual-write: local PG is primary; Cloud SQL at `10.106.0.3` receives same writes via a secondary psycopg2 pool (non-fatal if down).
|
||||||
|
- All timestamps: `ts` columns are Unix seconds (integer); `time_ms` columns are Unix milliseconds (bigint); `created_at` columns are PG `TIMESTAMP`.
|
||||||
|
- JSONB used for `score_factors` in `paper_trades`/`live_trades`, `detail` in `live_events`, `value` in `market_indicators`.
|
||||||
|
- Auth tokens stored in DB: refresh tokens in `refresh_tokens` table (revocable); no session table.
|
||||||
|
|
||||||
|
## Canonical Facts
|
||||||
|
|
||||||
|
### Tables
|
||||||
|
|
||||||
|
#### `rate_snapshots` — Funding Rate Snapshots
|
||||||
|
Populated every 2 s by `background_snapshot_loop()` in `main.py`.
|
||||||
|
| Column | Type | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `id` | BIGSERIAL PK | |
|
||||||
|
| `ts` | BIGINT NOT NULL | Unix seconds |
|
||||||
|
| `btc_rate` | DOUBLE PRECISION | BTC funding rate (decimal) |
|
||||||
|
| `eth_rate` | DOUBLE PRECISION | ETH funding rate |
|
||||||
|
| `btc_price` | DOUBLE PRECISION | BTC mark price USD |
|
||||||
|
| `eth_price` | DOUBLE PRECISION | ETH mark price USD |
|
||||||
|
| `btc_index_price` | DOUBLE PRECISION | BTC index price |
|
||||||
|
| `eth_index_price` | DOUBLE PRECISION | ETH index price |
|
||||||
|
|
||||||
|
Index: `idx_rate_snapshots_ts` on `ts`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `agg_trades` — Aggregate Trades (Partitioned)
|
||||||
|
Partitioned by `RANGE(time_ms)`; monthly child tables named `agg_trades_YYYYMM`.
|
||||||
|
| Column | Type | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `agg_id` | BIGINT NOT NULL | Binance aggTrade ID |
|
||||||
|
| `symbol` | TEXT NOT NULL | e.g. `BTCUSDT` |
|
||||||
|
| `price` | DOUBLE PRECISION | Trade price |
|
||||||
|
| `qty` | DOUBLE PRECISION | Trade quantity (BTC/ETH/etc.) |
|
||||||
|
| `time_ms` | BIGINT NOT NULL | Trade timestamp ms |
|
||||||
|
| `is_buyer_maker` | SMALLINT | 0=taker buy, 1=taker sell |
|
||||||
|
|
||||||
|
PK: `(time_ms, symbol, agg_id)`.
|
||||||
|
Indexes: `idx_agg_trades_sym_time` on `(symbol, time_ms DESC)`, `idx_agg_trades_sym_agg` on `(symbol, agg_id)`.
|
||||||
|
|
||||||
|
Partitions auto-created for current + next 2 months. Named `agg_trades_YYYYMM`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `agg_trades_meta` — Collection State
|
||||||
|
| Column | Type | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `symbol` | TEXT PK | e.g. `BTCUSDT` |
|
||||||
|
| `last_agg_id` | BIGINT | Last processed aggTrade ID |
|
||||||
|
| `last_time_ms` | BIGINT | Timestamp of last trade |
|
||||||
|
| `earliest_agg_id` | BIGINT | Oldest buffered ID |
|
||||||
|
| `earliest_time_ms` | BIGINT | Oldest buffered timestamp |
|
||||||
|
| `updated_at` | TEXT | Human-readable update time |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `signal_indicators` — Signal Engine Output (15 s cadence)
|
||||||
|
| Column | Type | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `id` | BIGSERIAL PK | |
|
||||||
|
| `ts` | BIGINT | Unix seconds |
|
||||||
|
| `symbol` | TEXT | |
|
||||||
|
| `cvd_fast` | DOUBLE PRECISION | CVD 30 m window |
|
||||||
|
| `cvd_mid` | DOUBLE PRECISION | CVD 4 h window |
|
||||||
|
| `cvd_day` | DOUBLE PRECISION | CVD UTC day |
|
||||||
|
| `cvd_fast_slope` | DOUBLE PRECISION | CVD momentum |
|
||||||
|
| `atr_5m` | DOUBLE PRECISION | ATR (5 m candles, 14 periods) |
|
||||||
|
| `atr_percentile` | DOUBLE PRECISION | ATR rank in 24 h history |
|
||||||
|
| `vwap_30m` | DOUBLE PRECISION | VWAP 30 m |
|
||||||
|
| `price` | DOUBLE PRECISION | Current mark price |
|
||||||
|
| `p95_qty` | DOUBLE PRECISION | P95 large-order threshold |
|
||||||
|
| `p99_qty` | DOUBLE PRECISION | P99 large-order threshold |
|
||||||
|
| `buy_vol_1m` | DOUBLE PRECISION | 1 m buy volume |
|
||||||
|
| `sell_vol_1m` | DOUBLE PRECISION | 1 m sell volume |
|
||||||
|
| `score` | INTEGER | Signal score 0–100 |
|
||||||
|
| `signal` | TEXT | `LONG`, `SHORT`, or null |
|
||||||
|
|
||||||
|
Indexes: `idx_si_ts`, `idx_si_sym_ts`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `signal_indicators_1m` — 1-Minute Signal Snapshot
|
||||||
|
Subset of `signal_indicators` columns; written at 1 m cadence for lightweight chart queries.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `signal_trades` — Signal Engine Trade Tracking
|
||||||
|
| Column | Type | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `id` | BIGSERIAL PK | |
|
||||||
|
| `ts_open` | BIGINT | Open timestamp (Unix s) |
|
||||||
|
| `ts_close` | BIGINT | Close timestamp |
|
||||||
|
| `symbol` | TEXT | |
|
||||||
|
| `direction` | TEXT | `LONG` / `SHORT` |
|
||||||
|
| `entry_price` | DOUBLE PRECISION | |
|
||||||
|
| `exit_price` | DOUBLE PRECISION | |
|
||||||
|
| `qty` | DOUBLE PRECISION | |
|
||||||
|
| `score` | INTEGER | Signal score at entry |
|
||||||
|
| `pnl` | DOUBLE PRECISION | Realized PnL |
|
||||||
|
| `sl_price` | DOUBLE PRECISION | Stop-loss level |
|
||||||
|
| `tp1_price` | DOUBLE PRECISION | Take-profit 1 level |
|
||||||
|
| `tp2_price` | DOUBLE PRECISION | Take-profit 2 level |
|
||||||
|
| `status` | TEXT DEFAULT `open` | `open`, `closed`, `stopped` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `paper_trades` — Paper Trading Records
|
||||||
|
| Column | Type | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `id` | BIGSERIAL PK | |
|
||||||
|
| `symbol` | TEXT | |
|
||||||
|
| `direction` | TEXT | `LONG`/`SHORT` |
|
||||||
|
| `score` | INT | Signal score |
|
||||||
|
| `tier` | TEXT | `light`/`standard`/`heavy` |
|
||||||
|
| `entry_price` | DOUBLE PRECISION | |
|
||||||
|
| `entry_ts` | BIGINT | Unix ms |
|
||||||
|
| `exit_price` | DOUBLE PRECISION | |
|
||||||
|
| `exit_ts` | BIGINT | |
|
||||||
|
| `tp1_price` | DOUBLE PRECISION | |
|
||||||
|
| `tp2_price` | DOUBLE PRECISION | |
|
||||||
|
| `sl_price` | DOUBLE PRECISION | |
|
||||||
|
| `tp1_hit` | BOOLEAN DEFAULT FALSE | |
|
||||||
|
| `status` | TEXT DEFAULT `active` | `active`, `tp1`, `tp2`, `sl`, `timeout` |
|
||||||
|
| `pnl_r` | DOUBLE PRECISION | PnL in R units |
|
||||||
|
| `atr_at_entry` | DOUBLE PRECISION | ATR snapshot at entry |
|
||||||
|
| `score_factors` | JSONB | Breakdown of signal score components |
|
||||||
|
| `strategy` | VARCHAR(32) DEFAULT `v51_baseline` | Strategy name |
|
||||||
|
| `created_at` | TIMESTAMP | |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `live_trades` — Live Trading Records
|
||||||
|
| Column | Type | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `id` | BIGSERIAL PK | |
|
||||||
|
| `symbol` | TEXT | |
|
||||||
|
| `strategy` | TEXT | |
|
||||||
|
| `direction` | TEXT | `LONG`/`SHORT` |
|
||||||
|
| `status` | TEXT DEFAULT `active` | |
|
||||||
|
| `entry_price` / `exit_price` | DOUBLE PRECISION | |
|
||||||
|
| `entry_ts` / `exit_ts` | BIGINT | Unix ms |
|
||||||
|
| `sl_price`, `tp1_price`, `tp2_price` | DOUBLE PRECISION | |
|
||||||
|
| `tp1_hit` | BOOLEAN | |
|
||||||
|
| `score` | DOUBLE PRECISION | |
|
||||||
|
| `tier` | TEXT | |
|
||||||
|
| `pnl_r` | DOUBLE PRECISION | |
|
||||||
|
| `fee_usdt` | DOUBLE PRECISION | Exchange fees |
|
||||||
|
| `funding_fee_usdt` | DOUBLE PRECISION | Funding fees paid while holding |
|
||||||
|
| `risk_distance` | DOUBLE PRECISION | Entry to SL distance |
|
||||||
|
| `atr_at_entry` | DOUBLE PRECISION | |
|
||||||
|
| `score_factors` | JSONB | |
|
||||||
|
| `signal_id` | BIGINT | FK → signal_indicators.id |
|
||||||
|
| `binance_order_id` | TEXT | Binance order ID |
|
||||||
|
| `fill_price` | DOUBLE PRECISION | Actual fill price |
|
||||||
|
| `slippage_bps` | DOUBLE PRECISION | Slippage in basis points |
|
||||||
|
| `protection_gap_ms` | BIGINT | Time between SL order and fill |
|
||||||
|
| `signal_to_order_ms` | BIGINT | Latency: signal → order placed |
|
||||||
|
| `order_to_fill_ms` | BIGINT | Latency: order → fill |
|
||||||
|
| `qty` | DOUBLE PRECISION | |
|
||||||
|
| `created_at` | TIMESTAMP | |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `live_config` — Runtime Configuration KV Store
|
||||||
|
| Column | Type | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `key` | TEXT PK | Config key |
|
||||||
|
| `value` | TEXT | Config value (string) |
|
||||||
|
| `label` | TEXT | Human label |
|
||||||
|
| `updated_at` | TIMESTAMP | |
|
||||||
|
|
||||||
|
Known keys: `risk_per_trade_usd`, `max_positions`, `circuit_break` (inferred).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `live_events` — Trade Event Log
|
||||||
|
| Column | Type | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `id` | BIGSERIAL PK | |
|
||||||
|
| `ts` | BIGINT | Unix ms (default: NOW()) |
|
||||||
|
| `level` | TEXT | `info`/`warning`/`error` |
|
||||||
|
| `category` | TEXT | Event category |
|
||||||
|
| `symbol` | TEXT | |
|
||||||
|
| `message` | TEXT | |
|
||||||
|
| `detail` | JSONB | Structured event data |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `signal_logs` — Legacy Signal Log
|
||||||
|
Kept for backwards compatibility with the original funding-rate signal system.
|
||||||
|
| Column | Type |
|
||||||
|
|--------|------|
|
||||||
|
| `id` | BIGSERIAL PK |
|
||||||
|
| `symbol` | TEXT |
|
||||||
|
| `rate` | DOUBLE PRECISION |
|
||||||
|
| `annualized` | DOUBLE PRECISION |
|
||||||
|
| `sent_at` | TEXT |
|
||||||
|
| `message` | TEXT |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Auth Tables (defined in `auth.py` AUTH_SCHEMA)
|
||||||
|
|
||||||
|
**`users`**
|
||||||
|
| Column | Type |
|
||||||
|
|--------|------|
|
||||||
|
| `id` | BIGSERIAL PK |
|
||||||
|
| `email` | TEXT UNIQUE NOT NULL |
|
||||||
|
| `password_hash` | TEXT NOT NULL |
|
||||||
|
| `discord_id` | TEXT |
|
||||||
|
| `role` | TEXT DEFAULT `user` |
|
||||||
|
| `banned` | INTEGER DEFAULT 0 |
|
||||||
|
| `created_at` | TEXT |
|
||||||
|
|
||||||
|
**`subscriptions`**
|
||||||
|
| Column | Type |
|
||||||
|
|--------|------|
|
||||||
|
| `user_id` | BIGINT PK → users |
|
||||||
|
| `tier` | TEXT DEFAULT `free` |
|
||||||
|
| `expires_at` | TEXT |
|
||||||
|
|
||||||
|
**`invite_codes`**
|
||||||
|
| Column | Type |
|
||||||
|
|--------|------|
|
||||||
|
| `id` | BIGSERIAL PK |
|
||||||
|
| `code` | TEXT UNIQUE |
|
||||||
|
| `created_by` | INTEGER |
|
||||||
|
| `max_uses` | INTEGER DEFAULT 1 |
|
||||||
|
| `used_count` | INTEGER DEFAULT 0 |
|
||||||
|
| `status` | TEXT DEFAULT `active` |
|
||||||
|
| `expires_at` | TEXT |
|
||||||
|
|
||||||
|
**`invite_usage`**
|
||||||
|
| Column | Type |
|
||||||
|
|--------|------|
|
||||||
|
| `id` | BIGSERIAL PK |
|
||||||
|
| `code_id` | BIGINT → invite_codes |
|
||||||
|
| `user_id` | BIGINT → users |
|
||||||
|
| `used_at` | TEXT |
|
||||||
|
|
||||||
|
**`refresh_tokens`**
|
||||||
|
| Column | Type |
|
||||||
|
|--------|------|
|
||||||
|
| `id` | BIGSERIAL PK |
|
||||||
|
| `user_id` | BIGINT → users |
|
||||||
|
| `token` | TEXT UNIQUE |
|
||||||
|
| `expires_at` | TEXT |
|
||||||
|
| `revoked` | INTEGER DEFAULT 0 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `market_indicators` — Market Indicator JSONB Store
|
||||||
|
Populated by `market_data_collector.py`.
|
||||||
|
| Column | Type | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `symbol` | TEXT | |
|
||||||
|
| `indicator_type` | TEXT | `long_short_ratio`, `top_trader_position`, `open_interest_hist`, `coinbase_premium`, `funding_rate` |
|
||||||
|
| `timestamp_ms` | BIGINT | |
|
||||||
|
| `value` | JSONB | Raw indicator payload |
|
||||||
|
|
||||||
|
Query pattern: `WHERE symbol=? AND indicator_type=? ORDER BY timestamp_ms DESC LIMIT 1`.
|
||||||
|
|
||||||
|
### Storage Design Decisions
|
||||||
|
- **Partitioning**: `agg_trades` partitioned by month to avoid table bloat; partition maintenance is automated.
|
||||||
|
- **Dual-write**: Cloud SQL secondary is best-effort (errors logged, never fatal).
|
||||||
|
- **JSONB `score_factors`**: allows schema-free storage of per-strategy signal breakdowns without migrations.
|
||||||
|
- **Timestamps**: mix of Unix seconds (`ts`), Unix ms (`time_ms`, `timestamp_ms`, `entry_ts`), ISO strings (`created_at` TEXT in auth tables), and PG `TIMESTAMP`; be careful when querying across tables.
|
||||||
|
|
||||||
|
## Interfaces / Dependencies
|
||||||
|
- `db.py:init_schema()` — creates all tables in `SCHEMA_SQL`
|
||||||
|
- `auth.py:ensure_tables()` — creates auth tables from `AUTH_SCHEMA`
|
||||||
|
- `db.py:ensure_partitions()` — auto-creates monthly `agg_trades_YYYYMM` partitions
|
||||||
|
|
||||||
|
## Unknowns & Risks
|
||||||
|
- [unknown] `market_indicators` table schema not in `SCHEMA_SQL`; likely created by `market_data_collector.py` separately — verify before querying.
|
||||||
|
- [risk] Timestamp inconsistency: some tables use TEXT for timestamps (auth tables), others use BIGINT, others use PG TIMESTAMP — cross-table JOINs on time fields require explicit casting.
|
||||||
|
- [inference] `live_config` circuit-break key name not confirmed from source; inferred from `risk_guard.py` behavior.
|
||||||
|
- [risk] `users` table defined in both `SCHEMA_SQL` (db.py) and `AUTH_SCHEMA` (auth.py); duplicate CREATE TABLE IF NOT EXISTS; actual schema diverges between the two definitions (db.py version lacks `discord_id`, `banned`).
|
||||||
|
|
||||||
|
## Source Refs
|
||||||
|
- `backend/db.py:166-356` — `SCHEMA_SQL` with all table definitions
|
||||||
|
- `backend/auth.py:28-71` — `AUTH_SCHEMA` auth tables
|
||||||
|
- `backend/db.py:360-414` — `ensure_partitions()`, `init_schema()`
|
||||||
|
- `backend/signal_engine.py:123-158` — `market_indicators` query pattern
|
||||||
251
docs/ai/05-build-run-test.md
Normal file
251
docs/ai/05-build-run-test.md
Normal file
@ -0,0 +1,251 @@
|
|||||||
|
---
|
||||||
|
generated_by: repo-insight
|
||||||
|
version: 1
|
||||||
|
created: 2026-03-03
|
||||||
|
last_updated: 2026-03-03
|
||||||
|
source_commit: 0d9dffa
|
||||||
|
coverage: deep
|
||||||
|
---
|
||||||
|
|
||||||
|
# 05 — Build, Run & Test
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
所有构建、运行、测试、部署相关命令及环境变量配置说明。
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
- 无 CI/CD 流水线；手动部署到 GCP VM，PM2 管理进程。
|
||||||
|
- 后端无构建步骤，直接 `python3 main.py` 或 PM2 启动。
|
||||||
|
- 前端标准 Next.js：`npm run dev` / `npm run build` / `npm start`。
|
||||||
|
- 测试文件未发现；验证通过 backtest.py 回测和 paper trading 模拟盘进行。
|
||||||
|
- 本地开发：前端 `/api/*` 通过 Next.js rewrite 代理到 `http://127.0.0.1:4332`（即 uvicorn 端口）。
|
||||||
|
- 数据库 schema 自动在启动时初始化（`init_schema()`），无独立 migration 工具。
|
||||||
|
|
||||||
|
## Canonical Facts
|
||||||
|
|
||||||
|
### 环境要求
|
||||||
|
| 组件 | 要求 |
|
||||||
|
|------|------|
|
||||||
|
| Python | 3.10+（使用 `list[dict]` 等 3.10 语法） |
|
||||||
|
| Node.js | 推荐 20.x（package.json `@types/node: ^20`） |
|
||||||
|
| PostgreSQL | 本地实例 + Cloud SQL（`10.106.0.3`） |
|
||||||
|
| PM2 | 用于进程管理（需全局安装） |
|
||||||
|
|
||||||
|
### 后端依赖安装
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
pip install -r requirements.txt
|
||||||
|
# requirements.txt 内容：fastapi, uvicorn, httpx, python-dotenv, psutil
|
||||||
|
# 实际还需要（从源码 import 推断）：
|
||||||
|
# asyncpg, psycopg2-binary, aiohttp, websockets
|
||||||
|
```
|
||||||
|
|
||||||
|
> [inference] `requirements.txt` 内容不完整，仅列出 5 个包，但源码 import 了 `asyncpg`、`psycopg2`、`aiohttp` 等。运行前需确认完整依赖已安装。
|
||||||
|
|
||||||
|
### 后端启动
|
||||||
|
|
||||||
|
#### 单进程开发模式
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
|
||||||
|
# FastAPI HTTP API（默认端口 4332，从 next.config.ts 推断）
|
||||||
|
uvicorn main:app --host 0.0.0.0 --port 4332 --reload
|
||||||
|
|
||||||
|
# 信号引擎（独立进程）
|
||||||
|
python3 signal_engine.py
|
||||||
|
|
||||||
|
# aggTrades 收集器
|
||||||
|
python3 agg_trades_collector.py
|
||||||
|
|
||||||
|
# 市场数据收集器
|
||||||
|
python3 market_data_collector.py
|
||||||
|
|
||||||
|
# 清算数据收集器
|
||||||
|
python3 liquidation_collector.py
|
||||||
|
|
||||||
|
# 实盘执行器
|
||||||
|
TRADE_ENV=testnet python3 live_executor.py
|
||||||
|
|
||||||
|
# 风控模块
|
||||||
|
TRADE_ENV=testnet python3 risk_guard.py
|
||||||
|
|
||||||
|
# 信号推送（Discord）
|
||||||
|
python3 signal_pusher.py
|
||||||
|
```
|
||||||
|
|
||||||
|
#### PM2 生产模式
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
|
||||||
|
# 使用 ecosystem 配置（目前只定义了 arb-dev-signal）
|
||||||
|
pm2 start ecosystem.dev.config.js
|
||||||
|
|
||||||
|
# 查看进程状态
|
||||||
|
pm2 status
|
||||||
|
|
||||||
|
# 查看日志
|
||||||
|
pm2 logs arb-dev-signal
|
||||||
|
|
||||||
|
# 停止所有
|
||||||
|
pm2 stop all
|
||||||
|
|
||||||
|
# 重启
|
||||||
|
pm2 restart all
|
||||||
|
```
|
||||||
|
|
||||||
|
> [inference] `ecosystem.dev.config.js` 当前只配置了 `signal_engine.py`，其他进程需手动启动或添加到 PM2 配置。
|
||||||
|
|
||||||
|
### 环境变量配置
|
||||||
|
|
||||||
|
#### 数据库（所有后端进程共用）
|
||||||
|
```bash
|
||||||
|
export PG_HOST=127.0.0.1 # 本地 PG
|
||||||
|
export PG_PORT=5432
|
||||||
|
export PG_DB=arb_engine
|
||||||
|
export PG_USER=arb
|
||||||
|
export PG_PASS=arb_engine_2026 # 测试网默认，生产需覆盖
|
||||||
|
|
||||||
|
export CLOUD_PG_HOST=10.106.0.3 # Cloud SQL
|
||||||
|
export CLOUD_PG_ENABLED=true
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 认证
|
||||||
|
```bash
|
||||||
|
export JWT_SECRET=<> # 生产环境必填，长度 ≥32
|
||||||
|
# 测试网有默认值 "arb-engine-jwt-secret-v2-2026"，生产环境 TRADE_ENV != testnet 时必须设置
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 交易环境
|
||||||
|
```bash
|
||||||
|
export TRADE_ENV=testnet # 或 production
|
||||||
|
export LIVE_STRATEGIES='["v52_8signals"]'
|
||||||
|
export RISK_PER_TRADE_USD=2 # 每笔风险 USD
|
||||||
|
export MAX_POSITIONS=4 # 最大同时持仓数
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 实盘专用（live_executor + risk_guard）
|
||||||
|
```bash
|
||||||
|
export DB_HOST=10.106.0.3
|
||||||
|
export DB_PASSWORD=<生产密码>
|
||||||
|
export DB_NAME=arb_engine
|
||||||
|
export DB_USER=arb
|
||||||
|
|
||||||
|
# 币安 API Key（需在 Binance 配置）
|
||||||
|
export BINANCE_API_KEY=<key>
|
||||||
|
export BINANCE_API_SECRET=<secret>
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 前端
|
||||||
|
```bash
|
||||||
|
# .env.local 或部署环境
|
||||||
|
NEXT_PUBLIC_API_URL= # 留空=同源，生产时设为 https://arb.zhouyangclaw.com
|
||||||
|
```
|
||||||
|
|
||||||
|
### 数据库初始化
|
||||||
|
```bash
|
||||||
|
# schema 在 FastAPI 启动时自动创建（init_schema + ensure_auth_tables）
|
||||||
|
# 手动初始化：
|
||||||
|
cd backend
|
||||||
|
python3 -c "from db import init_schema; init_schema()"
|
||||||
|
|
||||||
|
# 分区维护（自动在 init_schema 内调用）：
|
||||||
|
python3 -c "from db import ensure_partitions; ensure_partitions()"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 前端构建与启动
|
||||||
|
```bash
|
||||||
|
cd frontend
|
||||||
|
npm install
|
||||||
|
|
||||||
|
# 开发模式（热重载，端口 3000）
|
||||||
|
npm run dev
|
||||||
|
|
||||||
|
# 生产构建
|
||||||
|
npm run build
|
||||||
|
npm start
|
||||||
|
|
||||||
|
# Lint
|
||||||
|
npm run lint
|
||||||
|
```
|
||||||
|
|
||||||
|
### 前端 API 代理配置
|
||||||
|
`frontend/next.config.ts` 将 `/api/*` 代理到 `http://127.0.0.1:4332`。
|
||||||
|
- 本地开发时 uvicorn 需监听 **4332 端口**。
|
||||||
|
- 生产部署时通过 `NEXT_PUBLIC_API_URL` 或 nginx 反向代理处理跨域。
|
||||||
|
|
||||||
|
### 回测（离线验证）
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
|
||||||
|
# 指定天数回测
|
||||||
|
python3 backtest.py --symbol BTCUSDT --days 20
|
||||||
|
|
||||||
|
# 指定日期范围回测
|
||||||
|
python3 backtest.py --symbol BTCUSDT --start 2026-02-08 --end 2026-02-28
|
||||||
|
|
||||||
|
# 输出：胜率、盈亏比、夏普比率、最大回撤等统计
|
||||||
|
```
|
||||||
|
|
||||||
|
### 模拟盘（Paper Trading）
|
||||||
|
通过 `paper_config.json` 控制：
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"enabled": true,
|
||||||
|
"enabled_strategies": ["v52_8signals"],
|
||||||
|
"initial_balance": 10000,
|
||||||
|
"risk_per_trade": 0.02,
|
||||||
|
"max_positions": 4
|
||||||
|
}
|
||||||
|
```
|
||||||
|
修改后 signal_engine 下次循环自动读取（无需重启）。
|
||||||
|
|
||||||
|
监控模拟盘：
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
python3 paper_monitor.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 管理员 CLI
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
python3 admin_cli.py gen_invite [count] [max_uses]
|
||||||
|
python3 admin_cli.py list_invites
|
||||||
|
python3 admin_cli.py disable_invite <code>
|
||||||
|
python3 admin_cli.py list_users
|
||||||
|
python3 admin_cli.py ban_user <user_id>
|
||||||
|
python3 admin_cli.py unban_user <user_id>
|
||||||
|
python3 admin_cli.py set_admin <user_id>
|
||||||
|
python3 admin_cli.py usage
|
||||||
|
```
|
||||||
|
|
||||||
|
### 日志位置
|
||||||
|
| 进程 | 日志文件 |
|
||||||
|
|------|---------|
|
||||||
|
| signal_engine | `signal-engine.log`（项目根目录） |
|
||||||
|
| risk_guard | `backend/logs/risk_guard.log`（RotatingFileHandler，10MB×5） |
|
||||||
|
| 其他进程 | stdout / PM2 logs |
|
||||||
|
|
||||||
|
### 无测试框架
|
||||||
|
项目中未发现 `pytest`、`unittest` 或任何测试文件。验证策略依赖：
|
||||||
|
1. **回测**：`backtest.py` 逐 tick 回放历史数据
|
||||||
|
2. **模拟盘**：paper trading 实时验证信号质量
|
||||||
|
3. **手动测试**：前端页面人工验证
|
||||||
|
|
||||||
|
## Interfaces / Dependencies
|
||||||
|
- uvicorn 端口：**4332**（从 `next.config.ts` 推断）
|
||||||
|
- 前端开发端口：**3000**（Next.js 默认）
|
||||||
|
- CORS 允许 `localhost:3000` 和 `localhost:3001`
|
||||||
|
|
||||||
|
## Unknowns & Risks
|
||||||
|
- [inference] uvicorn 端口 4332 从 `next.config.ts` 推断，未在 `main.py` 或启动脚本中显式确认。
|
||||||
|
- [inference] `requirements.txt` 不完整，实际依赖需从源码 import 语句归纳。
|
||||||
|
- [unknown] 生产部署的 nginx 配置未在仓库中。
|
||||||
|
- [risk] 无自动化测试，代码变更风险完全依赖人工回测和 paper trading 验证。
|
||||||
|
|
||||||
|
## Source Refs
|
||||||
|
- `frontend/next.config.ts` — API rewrite 代理到 `127.0.0.1:4332`
|
||||||
|
- `backend/ecosystem.dev.config.js` — PM2 配置（仅 signal_engine）
|
||||||
|
- `backend/requirements.txt` — 后端依赖（不完整）
|
||||||
|
- `backend/backtest.py:1-13` — 回测用法说明
|
||||||
|
- `backend/paper_config.json` — 模拟盘配置
|
||||||
|
- `backend/admin_cli.py:88` — CLI usage 函数
|
||||||
|
- `backend/risk_guard.py:81-82` — 日志 RotatingFileHandler 配置
|
||||||
162
docs/ai/06-decision-log.md
Normal file
162
docs/ai/06-decision-log.md
Normal file
@ -0,0 +1,162 @@
|
|||||||
|
---
|
||||||
|
generated_by: repo-insight
|
||||||
|
version: 1
|
||||||
|
created: 2026-03-03
|
||||||
|
last_updated: 2026-03-03
|
||||||
|
source_commit: 0d9dffa
|
||||||
|
coverage: deep
|
||||||
|
---
|
||||||
|
|
||||||
|
# 06 — Decision Log
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
记录项目中关键的技术决策、选型原因及权衡取舍（从代码注释和架构特征推断）。
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
- 选择 PostgreSQL 作为唯一消息总线（NOTIFY/LISTEN），避免引入 Kafka/Redis 等额外组件。
|
||||||
|
- signal_engine 改为 15 秒循环（原 5 秒），CPU 降 60%，信号质量无影响。
|
||||||
|
- 双写 Cloud SQL 作为灾备，失败不阻断主流程。
|
||||||
|
- `agg_trades` 按月分区，避免单表过大影响查询性能。
|
||||||
|
- 认证采用自研 HMAC-SHA256 JWT，不依赖第三方库。
|
||||||
|
- 前端使用 Next.js App Router + 纯客户端轮询，不使用 WebSocket 推送。
|
||||||
|
- 策略参数外置为 JSON 文件，支持热修改无需重启进程。
|
||||||
|
- 信号评分采用多层加权体系（5层），每层独立可调，支持多策略并行。
|
||||||
|
|
||||||
|
## Canonical Facts
|
||||||
|
|
||||||
|
### 决策 1：PostgreSQL 作为进程间消息总线
|
||||||
|
**决策**：使用 PostgreSQL `NOTIFY/LISTEN` 在 signal_engine 和 live_executor 之间传递信号，而非 Redis pub/sub 或消息队列。
|
||||||
|
|
||||||
|
**原因**（从代码推断）：
|
||||||
|
- 系统已强依赖 PG；避免引入新的基础设施依赖。
|
||||||
|
- 信号触发频率低（每 15 秒最多一次），PG NOTIFY 完全满足延迟要求。
|
||||||
|
- 信号 payload 直接写入 `signal_indicators` 表，NOTIFY 仅做触发通知，消费者可直接查表。
|
||||||
|
|
||||||
|
**取舍**：单点依赖 PG；PG 宕机时信号传递和持久化同时失败（可接受，因为两者本就强耦合）。
|
||||||
|
|
||||||
|
**来源**：`live_executor.py:1-10` 架构注释，`signal_engine.py:save_indicator` 函数。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 决策 2：信号引擎循环间隔从 5 秒改为 15 秒
|
||||||
|
**决策**：`LOOP_INTERVAL = 15`（原注释说明原值为 5）。
|
||||||
|
|
||||||
|
**原因**：代码注释明确写道 "CPU降60%，信号质量无影响"。
|
||||||
|
|
||||||
|
**取舍**：信号触发延迟最坏增加 10 秒；对于短线但非高频的策略（TP/SL 以 ATR 倍数计，通常 >1% 波动），10 秒的额外延迟影响可忽略不计。
|
||||||
|
|
||||||
|
**来源**：`signal_engine.py:39` `LOOP_INTERVAL = 15 # 秒（从5改15，CPU降60%，信号质量无影响）`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 决策 3：agg_trades 表按月范围分区
|
||||||
|
**决策**：`agg_trades` 使用 `PARTITION BY RANGE(time_ms)`，按月创建子表（如 `agg_trades_202603`）。
|
||||||
|
|
||||||
|
**原因**：
|
||||||
|
- aggTrades 是最大的写入表（每秒数百条），无分区会导致单表膨胀。
|
||||||
|
- 按月分区支持高效的时间范围查询（PG 分区裁剪）。
|
||||||
|
- 旧分区可独立归档或删除，不影响主表。
|
||||||
|
|
||||||
|
**取舍**：分区管理需要维护（`ensure_partitions()` 自动创建当月+未来2个月分区，需定期执行）；跨分区查询性能取决于分区裁剪是否生效（`time_ms` 条件必须是常量）。
|
||||||
|
|
||||||
|
**来源**：`db.py:191-201, 360-393`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 决策 4：Cloud SQL 双写（非阻塞）
|
||||||
|
**决策**：所有写入操作在本地 PG 成功后，尝试相同写入到 Cloud SQL（`10.106.0.3`），Cloud SQL 失败不影响主流程。
|
||||||
|
|
||||||
|
**原因**：提供数据异地备份；Cloud SQL 作为只读副本或灾备使用。
|
||||||
|
|
||||||
|
**取舍**：
|
||||||
|
- 本地 PG 和 Cloud SQL 可能出现数据不一致（local 成功 + cloud 失败）。
|
||||||
|
- 双写增加每次写操作的延迟（两个网络 RTT），但因为是 best-effort 且使用独立连接池，实际阻塞极少。
|
||||||
|
- live_executor 直接连 Cloud SQL（`DB_HOST=10.106.0.3`），绕过本地 PG。
|
||||||
|
|
||||||
|
**来源**：`db.py:23-29, 80-118`，`live_executor.py:50-55`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 决策 5：自研 JWT（不用 PyJWT 等第三方库）
|
||||||
|
**决策**：使用 Python 标准库 `hmac`、`hashlib`、`base64` 手动实现 JWT 签发和验证。
|
||||||
|
|
||||||
|
**原因**（推断）：减少依赖；JWT 结构相对简单，HMAC-SHA256 签名几十行代码即可实现。
|
||||||
|
|
||||||
|
**取舍**：
|
||||||
|
- 需要自行处理过期、revoke、refresh token 等逻辑（代码中已有 `refresh_tokens` 表）。
|
||||||
|
- 非标准实现可能在边界情况（时钟偏差、特殊字符等）上与标准库行为不同。
|
||||||
|
- 无 JWT 生态工具支持（调试工具、密钥轮转库等）。
|
||||||
|
|
||||||
|
**来源**：`auth.py:1-6`（import hashlib, secrets, hmac, base64, json），`auth.py:16-19`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 决策 6：策略配置外置为 JSON 文件
|
||||||
|
**决策**：V5.x 策略的权重、阈值、TP/SL 倍数等参数存放在 `backend/strategies/*.json`，signal_engine 每次 `load_strategy_configs()` 读取。
|
||||||
|
|
||||||
|
**原因**：
|
||||||
|
- 策略调优频繁（v51→v52 权重变化显著），外置避免每次改参数都要修改代码。
|
||||||
|
- 多策略并行：signal_engine 同时运行 v51_baseline 和 v52_8signals，对每个 symbol 分别评分。
|
||||||
|
- [inference] 支持未来通过前端或 API 修改策略参数而不重启进程（目前 signal_engine 每次循环重读文件 —— 需确认）。
|
||||||
|
|
||||||
|
**取舍**：JSON 文件无类型检查，配置错误在运行时才发现；缺少配置 schema 校验。
|
||||||
|
|
||||||
|
**来源**：`signal_engine.py:41-67`，`backend/strategies/v51_baseline.json`，`backend/strategies/v52_8signals.json`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 决策 7：信号评分采用五层加权体系
|
||||||
|
**决策**：信号评分分为 5 个独立层次（方向层、拥挤层、资金费率层、环境层、确认层、清算层、辅助层），每层有独立权重，总分 0~100，阈值 75 触发信号。
|
||||||
|
|
||||||
|
**设计特点**：
|
||||||
|
- 方向层（CVD）权重最高（V5.1: 45分，V5.2: 40分），是核心指标。
|
||||||
|
- "standard" 档位：score ≥ threshold（75）；"heavy" 档位：score ≥ max(threshold+10, 85)。
|
||||||
|
- 信号冷却：同一 symbol 同一策略触发后 10 分钟内不再触发。
|
||||||
|
- CVD 快慢线需同向才产生完整方向信号；否则标记 `no_direction=True` 不触发。
|
||||||
|
|
||||||
|
**取舍**：权重缩放逻辑较复杂（各层原始满分不统一，需先归一化再乘权重）；`market_indicators` 缺失时给默认中间分，保证系统在数据不完整时仍能运行。
|
||||||
|
|
||||||
|
**来源**：`signal_engine.py:410-651`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 决策 8：前端使用轮询而非 WebSocket
|
||||||
|
**决策**：React 前端对 `/api/rates` 每 2 秒轮询，慢速数据（stats/history/signals）每 120 秒轮询，K 线图每 30 秒刷新。
|
||||||
|
|
||||||
|
**原因**（推断）：
|
||||||
|
- 实现简单，无需维护 WebSocket 连接状态和断线重连逻辑。
|
||||||
|
- 数据更新频率（2 秒/30 秒）对轮询友好；WebSocket 的优势在于毫秒级推送。
|
||||||
|
- FastAPI 已支持 WebSocket，但实现 SSE/WS 推送需要额外的后端状态管理。
|
||||||
|
|
||||||
|
**取舍**：每 2 秒轮询 `/api/rates` 会产生持续的服务器负载；当用户量增加时需要加缓存或换 WebSocket。
|
||||||
|
|
||||||
|
**来源**：`frontend/app/page.tsx:149-154`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 决策 9：live_executor 和 risk_guard 直连 Cloud SQL
|
||||||
|
**决策**：`live_executor.py` 和 `risk_guard.py` 默认 `DB_HOST=10.106.0.3`（Cloud SQL），而不是本地 PG。
|
||||||
|
|
||||||
|
**原因**（推断）：这两个进程运行在与 signal_engine 不同的环境（可能是另一台 GCP VM 或容器），直连 Cloud SQL 避免通过本地 PG 中转。
|
||||||
|
|
||||||
|
**取舍**：live_executor 和 signal_engine 使用不同的 PG 实例，理论上存在数据读取延迟（双写同步延迟）。
|
||||||
|
|
||||||
|
**来源**：`live_executor.py:50-55`，`risk_guard.py:47-53`
|
||||||
|
|
||||||
|
## Interfaces / Dependencies
|
||||||
|
无额外接口依赖，均为内部架构决策。
|
||||||
|
|
||||||
|
## Unknowns & Risks
|
||||||
|
- [inference] 所有决策均从代码推断，无明确的 ADR（Architecture Decision Record）文档。
|
||||||
|
- [unknown] 策略配置是否支持热重载（signal_engine 是否每次循环都重读 JSON）未确认。
|
||||||
|
- [risk] 决策 4（双写）+ 决策 9（live executor 直连 Cloud SQL）组合下，若本地 PG 和 Cloud SQL 数据不一致，live_executor 可能读到滞后的信号或重复执行。
|
||||||
|
|
||||||
|
## Source Refs
|
||||||
|
- `backend/signal_engine.py:39` — LOOP_INTERVAL 注释
|
||||||
|
- `backend/signal_engine.py:44-67` — load_strategy_configs
|
||||||
|
- `backend/signal_engine.py:410-651` — evaluate_signal 完整评分逻辑
|
||||||
|
- `backend/db.py:23-29, 80-118` — Cloud SQL 双写连接池
|
||||||
|
- `backend/live_executor.py:50-55` — DB_HOST 配置
|
||||||
|
- `backend/auth.py:1-6` — 自研 JWT import
|
||||||
|
- `frontend/app/page.tsx:149-154` — 轮询间隔
|
||||||
|
- `backend/strategies/v51_baseline.json`, `v52_8signals.json`
|
||||||
100
docs/ai/07-glossary.md
Normal file
100
docs/ai/07-glossary.md
Normal file
@ -0,0 +1,100 @@
|
|||||||
|
---
|
||||||
|
generated_by: repo-insight
|
||||||
|
version: 1
|
||||||
|
created: 2026-03-03
|
||||||
|
last_updated: 2026-03-03
|
||||||
|
source_commit: 0d9dffa
|
||||||
|
coverage: deep
|
||||||
|
---
|
||||||
|
|
||||||
|
# 07 — Glossary
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
项目中使用的专业术语、领域术语和项目自定义术语的定义。
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
- 项目混合使用量化交易、加密货币和工程术语，中英文混用。
|
||||||
|
- "R" 是风险单位，1R = 单笔风险金额（PAPER_RISK_PER_TRADE × 余额）。
|
||||||
|
- CVD 是核心指标：累计 delta = 主动买量 - 主动卖量，衡量买卖压力。
|
||||||
|
- ATR 用于动态计算止盈止损距离（TP/SL 均以 ATR 倍数表示）。
|
||||||
|
- "tier" 指仓位档位：light/standard/heavy，对应不同仓位大小倍数。
|
||||||
|
|
||||||
|
## Canonical Facts
|
||||||
|
|
||||||
|
### 交易与量化术语
|
||||||
|
|
||||||
|
| 术语 | 定义 |
|
||||||
|
|------|------|
|
||||||
|
| **资金费率（Funding Rate）** | 永续合约中多空双方每8小时相互支付的费率。正费率：多头付给空头；负费率：空头付给多头。以小数表示（如 `0.0001` = 0.01%）。 |
|
||||||
|
| **永续合约（Perpetual / Perp）** | 无到期日的期货合约，通过资金费率机制锚定现货价格。本项目操作 Binance USDC-M 永续合约。 |
|
||||||
|
| **套利（Arbitrage）** | 持有现货多头 + 永续空头，资金费率为正时空头每8小时收取费率收益，实现无方向性风险的稳定收益。 |
|
||||||
|
| **年化（Annualized）** | `平均费率 × 3次/天 × 365天 × 100%`，将单次资金费率换算为年化百分比。 |
|
||||||
|
| **CVD（Cumulative Volume Delta）** | 累计成交量差值 = 主动买量 - 主动卖量。正值表示买方主导，负值表示卖方主导。本项目计算三个窗口：CVD_fast（30分钟）、CVD_mid（4小时）、CVD_day（UTC日内）。 |
|
||||||
|
| **aggTrade** | Binance 聚合成交数据：同一方向、同一价格、同一时刻的多笔成交合并为一条记录，包含 `is_buyer_maker` 字段（0=主动买，1=主动卖）。 |
|
||||||
|
| **is_buyer_maker** | `0`：买方是 taker（主动买入），`1`：买方是 maker（被动成交，即主动卖）。CVD 计算：0→买量，1→卖量。 |
|
||||||
|
| **VWAP（Volume Weighted Average Price）** | 成交量加权平均价格。用于判断当前价格相对于短期平均成本的位置。 |
|
||||||
|
| **ATR（Average True Range）** | 平均真实波动幅度，衡量市场波动性。本项目使用 5 分钟 K 线、14 周期 EMA 计算。 |
|
||||||
|
| **ATR Percentile** | 当前 ATR 在过去 24 小时内的历史分位数（0~100），衡量当前波动性是高还是低。 |
|
||||||
|
| **P95 / P99** | 过去 24 小时内成交量的第 95/99 百分位数，作为"大单阈值"。超过 P99 的成交视为大单，对信号评分有影响。 |
|
||||||
|
| **Long/Short Ratio（多空比）** | 全市场多头账户数 / 空头账户数。反映市场情绪拥挤程度。 |
|
||||||
|
| **Top Trader Position（顶级交易者持仓比）** | 大户多头持仓占比，范围 0~1。高于 0.55 视为多头拥挤，低于 0.45 视为空头拥挤。 |
|
||||||
|
| **Open Interest（OI，持仓量）** | 市场上所有未平仓合约的总名义价值（USD）。OI 增加 ≥3% 视为环境强势信号。 |
|
||||||
|
| **Coinbase Premium** | Coinbase Pro BTC/USD 现货价格相对 Binance BTC/USDT 的溢价比例。正溢价（>0.05%）被视为看涨信号（美国机构买入）。以比例存储（如 `0.0005` = 0.05%）。 |
|
||||||
|
| **清算（Liquidation）** | 爆仓事件。空头清算多于多头清算（短时间内）视为看涨信号（逼空）。本项目使用 5 分钟窗口内多空清算 USD 之比进行评分。 |
|
||||||
|
| **R（风险单位）** | 单笔风险金额。1R = `初始余额 × 风险比例`（默认 2%，即 200U）。盈亏以 R 倍数表示：1R=保本，2R=盈利1倍风险，-1R=全亏。 |
|
||||||
|
| **PnL_R** | 以 R 为单位的盈亏：`pnl_r = (exit_price - entry_price) / risk_distance × direction_sign`。 |
|
||||||
|
| **TP1 / TP2（Take Profit）** | 止盈目标价。TP1 为第一目标（触发后平一半仓位），TP2 为第二目标（平剩余）。 |
|
||||||
|
| **SL（Stop Loss）** | 止损价。SL 触发后视 TP1 是否已命中：未命中→亏损 1R；已命中→保本（sl_be 状态）。 |
|
||||||
|
| **Tier（档位）** | 仓位大小分级。`light`=0.5×R，`standard`=1.0×R，`heavy`=1.5×R。信号分数越高触发越重的档位：score ≥ max(threshold+10, 85) → heavy；score ≥ threshold → standard。 |
|
||||||
|
| **Warmup（冷启动）** | signal_engine 启动时读取历史 `agg_trades` 填充滚动窗口的过程，完成前不产生信号（`state.warmup=True`）。 |
|
||||||
|
| **Signal Cooldown（信号冷却）** | 同一 symbol 同一策略触发信号后，10 分钟内不再触发新信号，防止过度交易。 |
|
||||||
|
|
||||||
|
### 策略术语
|
||||||
|
|
||||||
|
| 术语 | 定义 |
|
||||||
|
|------|------|
|
||||||
|
| **v51_baseline** | V5.1 基准策略。6 个信号：cvd, p99, accel, ls_ratio, oi, coinbase_premium。SL=1.4×ATR，TP1=1.05×ATR，TP2=2.1×ATR。 |
|
||||||
|
| **v52_8signals** | V5.2 扩展策略。8 个信号（v51 + funding_rate + liquidation）。SL=2.1×ATR，TP1=1.4×ATR，TP2=3.15×ATR（更宽止损，更高盈亏比目标）。 |
|
||||||
|
| **Score / 信号分数** | 0~100 的综合评分，由多层加权指标累加得出，阈值 75 触发信号。 |
|
||||||
|
| **Direction Layer（方向层）** | 评分第一层，最高 45 分（v51）或 40 分（v52）。基于 CVD_fast、CVD_mid 同向性和 P99 大单方向。 |
|
||||||
|
| **Crowding Layer（拥挤层）** | 基于多空比和顶级交易者持仓的市场拥挤度评分。 |
|
||||||
|
| **Environment Layer（环境层）** | 基于持仓量变化（OI change）的市场环境评分。 |
|
||||||
|
| **Confirmation Layer（确认层）** | CVD 快慢线同向确认，15 分（满足）或 0 分。 |
|
||||||
|
| **Auxiliary Layer（辅助层）** | Coinbase Premium 辅助确认，0~5 分。 |
|
||||||
|
| **Accel Bonus（加速奖励）** | CVD 快线斜率正在加速时额外加分（v51: +5分，v52: +0分）。 |
|
||||||
|
| **Score Factors** | 各层得分详情，以 JSONB 格式存储在 `paper_trades.score_factors` 和 `live_trades.score_factors`。 |
|
||||||
|
|
||||||
|
### 工程术语
|
||||||
|
|
||||||
|
| 术语 | 定义 |
|
||||||
|
|------|------|
|
||||||
|
| **Paper Trading / 模拟盘** | 不真实下单、仅模拟记录的交易，用于验证策略。数据存储在 `paper_trades` 表。 |
|
||||||
|
| **Live Trading / 实盘** | 通过 Binance API 真实下单执行的交易。数据存储在 `live_trades` 表。 |
|
||||||
|
| **Testnet** | Binance 测试网（`https://testnet.binancefuture.com`），使用虚拟资金。`TRADE_ENV=testnet`。 |
|
||||||
|
| **Production** | Binance 生产环境（`https://fapi.binance.com`），使用真实资金。`TRADE_ENV=production`。 |
|
||||||
|
| **Circuit Break（熔断）** | risk_guard 触发的保护机制，阻止新开仓甚至强制平仓。通过 `live_config` 表的 flag 通知 live_executor。 |
|
||||||
|
| **Dual Write（双写）** | 同一数据同时写入本地 PG 和 Cloud SQL，Cloud SQL 写失败不阻断主流程。 |
|
||||||
|
| **Partition / 分区** | `agg_trades` 表的月度子表（如 `agg_trades_202603`），用于管理大表性能。 |
|
||||||
|
| **NOTIFY/LISTEN** | PostgreSQL 原生异步通知机制。signal_engine 用 `NOTIFY new_signal` 触发，live_executor 用 `LISTEN new_signal` 接收。 |
|
||||||
|
| **TradeWindow** | signal_engine 中的滚动时间窗口类，维护 CVD 和 VWAP 的实时滚动计算。 |
|
||||||
|
| **SymbolState** | 每个交易对的完整状态容器，包含三个 TradeWindow、ATRCalculator、market_indicators 缓存和信号冷却记录。 |
|
||||||
|
| **Invite Code（邀请码）** | 注册时必须提供的一次性（或限次）代码，由管理员通过 `admin_cli.py` 生成。 |
|
||||||
|
| **Subscription Tier** | 用户订阅等级（`free` 等），存储在 `subscriptions` 表，当前代码中使用有限。 |
|
||||||
|
| **万分之** | 前端显示资金费率时的单位表述，实际值 × 10000 展示。例如 `0.0001` 显示为 `1.0000 万分之`。 |
|
||||||
|
|
||||||
|
## Interfaces / Dependencies
|
||||||
|
无。
|
||||||
|
|
||||||
|
## Unknowns & Risks
|
||||||
|
- [inference] `Subscription Tier` 功能在 schema 中有定义但实际业务逻辑中使用程度不确定（可能是预留字段）。
|
||||||
|
- [inference] "no_direction" 状态（CVD_fast 和 CVD_mid 不一致时）的处理逻辑：方向取 CVD_fast，但标记为不触发信号，可用于反向平仓判断。
|
||||||
|
|
||||||
|
## Source Refs
|
||||||
|
- `backend/signal_engine.py:1-16` — CVD/ATR/VWAP/P95/P99 架构注释
|
||||||
|
- `backend/signal_engine.py:69-81` — Paper trading 参数定义（R、tier 倍数）
|
||||||
|
- `backend/signal_engine.py:170-207` — TradeWindow 类（CVD/VWAP 定义）
|
||||||
|
- `backend/signal_engine.py:209-257` — ATRCalculator 类
|
||||||
|
- `backend/signal_engine.py:410-651` — evaluate_signal（各层评分逻辑）
|
||||||
|
- `backend/strategies/v51_baseline.json`, `v52_8signals.json` — 策略参数
|
||||||
|
- `backend/trade_config.py` — 交易对精度配置
|
||||||
|
- `frontend/app/page.tsx:186` — "万分之" 显示注释
|
||||||
141
docs/ai/99-open-questions.md
Normal file
141
docs/ai/99-open-questions.md
Normal file
@ -0,0 +1,141 @@
|
|||||||
|
---
|
||||||
|
generated_by: repo-insight
|
||||||
|
version: 1
|
||||||
|
created: 2026-03-03
|
||||||
|
last_updated: 2026-03-03
|
||||||
|
source_commit: 0d9dffa
|
||||||
|
coverage: deep
|
||||||
|
---
|
||||||
|
|
||||||
|
# 99 — Open Questions
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
记录文档生成过程中发现的未解决问题、不确定点和潜在风险。
|
||||||
|
|
||||||
|
## TL;DR
|
||||||
|
- `requirements.txt` 不完整，实际依赖需手动补齐。
|
||||||
|
- `users` 表在两个地方定义且 schema 不一致（db.py vs auth.py）。
|
||||||
|
- live_executor / risk_guard 直连 Cloud SQL 但 signal_engine 写本地 PG，存在数据同步延迟风险。
|
||||||
|
- 策略是否支持热重载（每次循环重读 JSON）未确认。
|
||||||
|
- uvicorn 监听端口 4332 未在启动脚本中显式确认。
|
||||||
|
- 无 CI/CD，无自动化测试。
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
### 高优先级（影响正确性）
|
||||||
|
|
||||||
|
#### Q1：users 表 schema 双定义不一致
|
||||||
|
**问题**：`db.py` 的 `SCHEMA_SQL` 和 `auth.py` 的 `AUTH_SCHEMA` 均定义了 `users` 表，但字段不同：
|
||||||
|
- `db.py` 版本：`id, email, password_hash, role, created_at`（无 `discord_id`、无 `banned`）
|
||||||
|
- `auth.py` 版本：`id, email, password_hash, discord_id, role, banned, created_at`
|
||||||
|
|
||||||
|
`init_schema()` 和 `ensure_auth_tables()` 都在 FastAPI startup 中调用，两次 `CREATE TABLE IF NOT EXISTS` 第一次成功后第二次静默跳过。**实际创建的是哪个版本？** 取决于调用顺序（先 `init_schema` 后 `ensure_auth_tables`），如果本地 PG 已有旧版表则字段可能缺失。
|
||||||
|
|
||||||
|
**影响**：auth 相关功能（discord_id 关联、banned 状态检查）可能在 schema 未更新的环境下失效。
|
||||||
|
|
||||||
|
**建议行动**：统一到 auth.py 版本，或添加 `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` 迁移。
|
||||||
|
|
||||||
|
**来源**：`db.py:269-276`，`auth.py:28-37`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Q2：live_executor 读 Cloud SQL，signal_engine 写本地 PG，双写延迟是否可接受
|
||||||
|
**问题**：signal_engine 写入本地 PG（`signal_indicators`），同时双写 Cloud SQL；live_executor 直连 Cloud SQL 读取信号。若某次双写失败或延迟，live_executor 可能错过信号或读到不一致数据。
|
||||||
|
|
||||||
|
**影响**：实盘信号丢失或执行延迟。
|
||||||
|
|
||||||
|
**建议行动**：确认 NOTIFY 是否也发送到 Cloud SQL（即 live_executor 通过 LISTEN 接收信号，不依赖轮询读表）；或将 live_executor 改为连接本地 PG。
|
||||||
|
|
||||||
|
**来源**：`live_executor.py:50-55`，`db.py:76-95`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Q3：requirements.txt 不完整
|
||||||
|
**问题**：`requirements.txt` 只列出 `fastapi, uvicorn, httpx, python-dotenv, psutil`，但源码还 import 了 `asyncpg`、`psycopg2`（用于 psycopg2-binary）、`aiohttp`、`websockets`（推断）等。
|
||||||
|
|
||||||
|
**影响**：新环境安装后进程无法启动。
|
||||||
|
|
||||||
|
**建议行动**：执行 `pip freeze > requirements.txt` 或手动补全所有依赖。
|
||||||
|
|
||||||
|
**来源**：`backend/requirements.txt:1-5`，`backend/db.py:9-11`，`backend/live_executor.py:28-29`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 中优先级（影响维护性）
|
||||||
|
|
||||||
|
#### Q4：策略 JSON 是否支持热重载
|
||||||
|
**问题**：`load_strategy_configs()` 在 `main()` 函数开头调用一次。不清楚 signal_engine 的主循环是否每次迭代都重新调用此函数。
|
||||||
|
|
||||||
|
**影响**：如果不支持热重载，修改策略 JSON 后需要重启 signal_engine 进程。
|
||||||
|
|
||||||
|
**来源**：`signal_engine.py:44-67, 964`（需查看 main 函数结构）
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Q5：uvicorn 端口确认
|
||||||
|
**问题**：从 `frontend/next.config.ts` 推断 uvicorn 运行在 `127.0.0.1:4332`，但没有找到后端启动脚本明确指定此端口。
|
||||||
|
|
||||||
|
**建议行动**：在 `ecosystem.dev.config.js` 或启动脚本中显式记录端口。
|
||||||
|
|
||||||
|
**来源**：`frontend/next.config.ts:8`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Q6：market_indicators 表 schema 未在 SCHEMA_SQL 中定义
|
||||||
|
**问题**：signal_engine 从 `market_indicators` 表读取数据（指标类型：`long_short_ratio`, `top_trader_position`, `open_interest_hist`, `coinbase_premium`, `funding_rate`），但该表的 CREATE TABLE 语句不在 `db.py` 的 `SCHEMA_SQL` 中。
|
||||||
|
|
||||||
|
**影响**：表由 `market_data_collector.py` 单独创建；如果该进程未运行过，表不存在，signal_engine 会报错或返回空数据。
|
||||||
|
|
||||||
|
**建议行动**：将 `market_indicators` 表定义加入 `SCHEMA_SQL`，确保 `init_schema()` 能覆盖全量 schema。
|
||||||
|
|
||||||
|
**来源**：`signal_engine.py:123-158`，`db.py:166-357`（未见 market_indicators 定义）
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Q7：liquidations 表 schema 未确认
|
||||||
|
**问题**：signal_engine 查询 `liquidations` 表（`SELECT FROM liquidations WHERE symbol=%s AND trade_time >= %s`），但该表定义在 `SCHEMA_SQL` 中同样未找到。可能由 `liquidation_collector.py` 自行创建。
|
||||||
|
|
||||||
|
**来源**：`signal_engine.py:395-407`，`liquidation_collector.py:28`（`ensure_table()` 函数）
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 低优先级（长期健康度）
|
||||||
|
|
||||||
|
#### Q8：无 CI/CD 流水线
|
||||||
|
**问题**：仓库中没有 `.github/workflows/`、Dockerfile、docker-compose.yml 等部署自动化文件。所有部署为手动操作（ssh + git pull + pm2 restart）。
|
||||||
|
|
||||||
|
**建议行动**：添加 GitHub Actions 用于基本 lint 检查和依赖安全扫描。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Q9：无自动化测试
|
||||||
|
**问题**：未发现任何测试文件（`test_*.py`、`*.test.ts` 等）。策略验证完全依赖人工回测和模拟盘。
|
||||||
|
|
||||||
|
**建议行动**：至少为 `evaluate_signal()`、`TradeWindow`、`ATRCalculator` 添加单元测试，防止重构回归。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Q10：生产环境硬编码密码风险
|
||||||
|
**问题**：`db.py`、`live_executor.py`、`risk_guard.py` 中均有 testnet 默认密码 `arb_engine_2026` 硬编码在源代码里（通过 `os.getenv(..., "arb_engine_2026")` 方式）。
|
||||||
|
|
||||||
|
**影响**：代码一旦泄露，testnet 数据库可被访问；生产环境如果环境变量设置失败，会静默使用错误密码（失败时的错误信息较明确，但仍有风险）。
|
||||||
|
|
||||||
|
**建议行动**：testnet 默认密码移除或通过单独的 `.env.testnet` 文件管理，不内嵌到源代码。
|
||||||
|
|
||||||
|
**来源**：`db.py:19`，`live_executor.py:44`，`risk_guard.py:42`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Q11：`signal_indicators` 表含 `strategy` 字段但 schema 未声明
|
||||||
|
**问题**：`save_indicator()` 函数的 INSERT 语句包含 `strategy` 字段，但 `SCHEMA_SQL` 中的 `signal_indicators` 表定义不包含该字段。可能通过 `ALTER TABLE ADD COLUMN IF NOT EXISTS` 在运行时补充，或是后续版本添加但忘记更新 schema。
|
||||||
|
|
||||||
|
**来源**：`signal_engine.py:690-699`，`db.py:205-224`（signal_indicators 定义）
|
||||||
|
|
||||||
|
## Source Refs
|
||||||
|
- `backend/db.py:269-276` — db.py 版 users 表
|
||||||
|
- `backend/auth.py:28-37` — auth.py 版 users 表（含 discord_id, banned）
|
||||||
|
- `backend/requirements.txt` — 不完整的依赖列表
|
||||||
|
- `backend/live_executor.py:44, 50-55` — DB_HOST 和默认密码
|
||||||
|
- `backend/risk_guard.py:42, 47-53` — DB_HOST 和默认密码
|
||||||
|
- `backend/signal_engine.py:395-407` — liquidations 表查询
|
||||||
|
- `backend/signal_engine.py:690-699` — strategy 字段 INSERT
|
||||||
52
docs/ai/INDEX.md
Normal file
52
docs/ai/INDEX.md
Normal file
@ -0,0 +1,52 @@
|
|||||||
|
---
|
||||||
|
generated_by: repo-insight
|
||||||
|
version: 1
|
||||||
|
created: 2026-03-03
|
||||||
|
last_updated: 2026-03-03
|
||||||
|
source_commit: 0d9dffa
|
||||||
|
coverage: deep
|
||||||
|
---
|
||||||
|
|
||||||
|
# Arbitrage Engine — AI Documentation Index
|
||||||
|
|
||||||
|
**Project**: `arbitrage-engine`
|
||||||
|
**Summary**: Full-stack crypto perpetual futures funding-rate arbitrage monitoring and V5.x CVD/ATR-based short-term trading signal engine. Python/FastAPI backend + Next.js frontend + PostgreSQL + Binance USDC-M Futures.
|
||||||
|
|
||||||
|
## Generated Documents
|
||||||
|
|
||||||
|
| File | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| [00-system-overview.md](./00-system-overview.md) | Project purpose, tech stack, repo layout, entry points, environment variables |
|
||||||
|
| [01-architecture-map.md](./01-architecture-map.md) | Multi-process architecture, component diagram, signal pipeline data flow, risk guard rules, frontend polling |
|
||||||
|
| [02-module-cheatsheet.md](./02-module-cheatsheet.md) | Module-by-module index: role, public interfaces, dependencies for all 20 backend + 15 frontend files |
|
||||||
|
| [03-api-contracts.md](./03-api-contracts.md) | All REST endpoints, auth flows, request/response shapes, error conventions |
|
||||||
|
| [04-data-model.md](./04-data-model.md) | All PostgreSQL tables, columns, partitioning strategy, storage design decisions |
|
||||||
|
| [05-build-run-test.md](./05-build-run-test.md) | 构建/运行/部署命令，环境变量，PM2 配置，回测和模拟盘操作 |
|
||||||
|
| [06-decision-log.md](./06-decision-log.md) | 9 项关键技术决策：PG 消息总线、循环间隔、双写、分区、自研 JWT 等 |
|
||||||
|
| [07-glossary.md](./07-glossary.md) | 交易术语（CVD/ATR/R/tier）+ 工程术语（paper trading/warmup/circuit break） |
|
||||||
|
| [99-open-questions.md](./99-open-questions.md) | 11 个未解决问题：users 表双定义冲突、依赖不完整、硬编码密码、无测试等 |
|
||||||
|
|
||||||
|
## Recommended Reading Order
|
||||||
|
|
||||||
|
1. **Start here**: `00-system-overview.md` — 了解项目定位和结构。
|
||||||
|
2. **Architecture**: `01-architecture-map.md` — 理解 7+ 进程的交互方式。
|
||||||
|
3. **Data**: `04-data-model.md` — 任何 DB 相关工作的必读；注意时间戳格式不统一问题。
|
||||||
|
4. **API**: `03-api-contracts.md` — 前端开发或 API 对接时参考。
|
||||||
|
5. **Module detail**: `02-module-cheatsheet.md` — 修改特定文件前的参考。
|
||||||
|
6. **Ops**: `05-build-run-test.md` — 部署和运维操作。
|
||||||
|
7. **Concepts**: `07-glossary.md` — 不熟悉量化术语时查阅。
|
||||||
|
8. **Risks**: `99-open-questions.md` — 开始开发前必读，了解已知风险点。
|
||||||
|
|
||||||
|
## Coverage Tier
|
||||||
|
**Deep** — 包含完整的模块签名读取、核心业务模块深度阅读（signal_engine 全文、evaluate_signal 评分逻辑、backtest.py）、构建运行指南、决策日志、术语表和开放问题。
|
||||||
|
|
||||||
|
## Key Facts for AI Agents
|
||||||
|
- **Signal engine is the core**: `backend/signal_engine.py` — change with care; affects all trading modes.
|
||||||
|
- **Strategy tuning via JSON**: modify `backend/strategies/v51_baseline.json` or `v52_8signals.json` to change signal weights/thresholds without code changes.
|
||||||
|
- **No ORM**: raw SQL via `asyncpg`/`psycopg2`; schema in `db.py:SCHEMA_SQL`.
|
||||||
|
- **Auth is custom JWT**: no third-party auth library; hand-rolled HMAC-SHA256 in `auth.py`.
|
||||||
|
- **`TRADE_ENV=testnet` default**: production use requires explicit env override + strong JWT_SECRET.
|
||||||
|
- **Dual timestamp formats**: `ts` = Unix seconds, `time_ms`/`entry_ts`/`timestamp_ms` = Unix milliseconds — do not confuse.
|
||||||
|
|
||||||
|
## Generation Timestamp
|
||||||
|
2026-03-03T00:00:00 (UTC)
|
||||||
Loading…
Reference in New Issue
Block a user