Add core infrastructure: config, Binance adapter, docs, and auto-setup
- Add Pydantic settings with trading mode validation (paper/testnet/live) - Implement Binance USDⓈ-M Futures adapter with hedge mode, isolated margin - Add type definitions for orders, positions, and market data - Create documentation (PLAN.md, ARCHITECTURE.md, SECURITY.md) - Add setup.sh with uv/pip auto-detection for consistent dev environments - Configure Docker multi-stage build and docker-compose services - Add pyproject.toml with all dependencies and tool configs
This commit is contained in:
bnair123
370
docs/ARCHITECTURE.md
Normal file
370
docs/ARCHITECTURE.md
Normal file
@@ -0,0 +1,370 @@
|
||||
# TradeFinder Architecture
|
||||
|
||||
## System Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ TradeFinder │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌───────────┐ │
|
||||
│ │ Streamlit │ │ Engine │ │ Optimizer │ │ Backtester│ │
|
||||
│ │ UI │◄───►│ (Core) │◄───►│ (Optuna) │ │ │ │
|
||||
│ └─────────────┘ └──────┬──────┘ └─────────────┘ └───────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────────────────┼───────────────────┐ │
|
||||
│ │ │ │ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Regime │ │ Risk │ │ Order │ │
|
||||
│ │ Classifier │ │ Engine │ │ Manager │ │
|
||||
│ └─────────────┘ └─────────────┘ └──────┬──────┘ │
|
||||
│ │ │ │ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Strategies │ │ Allocator │ │ Exchange │ │
|
||||
│ │ Suite │ │ │ │ Adapter │ │
|
||||
│ └─────────────┘ └─────────────┘ └──────┬──────┘ │
|
||||
│ │ │
|
||||
├──────────────────────────────────────────────────┼──────────────────────────┤
|
||||
│ Data Layer │ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ │ │
|
||||
│ │ DuckDB │ │ Redis │ │ │
|
||||
│ │ (Analytics) │ │ (State) │ │ │
|
||||
│ └─────────────┘ └─────────────┘ │ │
|
||||
└──────────────────────────────────────────────────┼──────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────┐
|
||||
│ Binance │
|
||||
│ Futures API │
|
||||
│ (Testnet) │
|
||||
└───────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
### 1. Core Engine (`src/tradefinder/core/`)
|
||||
|
||||
The main orchestrator that coordinates all components.
|
||||
|
||||
```
|
||||
core/
|
||||
├── __init__.py
|
||||
├── config.py # Pydantic settings
|
||||
├── main.py # Entry point, event loop
|
||||
├── engine.py # Trading engine orchestrator
|
||||
├── backtest.py # Backtesting entry point
|
||||
└── optimize.py # Optimization entry point
|
||||
```
|
||||
|
||||
**Responsibilities**:
|
||||
- Load configuration from environment
|
||||
- Initialize all components
|
||||
- Run main trading loop (signal → risk → order)
|
||||
- Handle graceful shutdown
|
||||
|
||||
### 2. Exchange Adapters (`src/tradefinder/adapters/`)
|
||||
|
||||
Abstract interface for exchange connectivity with Binance implementation.
|
||||
|
||||
```
|
||||
adapters/
|
||||
├── __init__.py
|
||||
├── base.py # Abstract ExchangeAdapter interface
|
||||
├── binance_usdm.py # Binance USDⓈ-M Futures implementation
|
||||
└── types.py # Shared types (Order, Position, etc.)
|
||||
```
|
||||
|
||||
**Key Features**:
|
||||
- Hedge mode support (dual position side)
|
||||
- Isolated margin per symbol
|
||||
- Order types: limit, stop-market, market
|
||||
- WebSocket streams for real-time data
|
||||
|
||||
**Binance USDⓈ-M Specifics**:
|
||||
```python
|
||||
# Required API calls on initialization
|
||||
POST /fapi/v1/positionSide/dual # Enable hedge mode
|
||||
POST /fapi/v1/marginType # Set ISOLATED margin
|
||||
POST /fapi/v1/leverage # Set leverage per symbol
|
||||
```
|
||||
|
||||
### 3. Data Layer (`src/tradefinder/data/`)
|
||||
|
||||
Market data ingestion, storage, and retrieval.
|
||||
|
||||
```
|
||||
data/
|
||||
├── __init__.py
|
||||
├── fetcher.py # REST historical data fetcher
|
||||
├── streamer.py # WebSocket real-time streams
|
||||
├── storage.py # DuckDB operations
|
||||
└── schemas.py # Database schemas
|
||||
```
|
||||
|
||||
**Data Sources**:
|
||||
| Data Type | Source | Frequency |
|
||||
|-----------|--------|-----------|
|
||||
| OHLCV | REST (backfill) + WS (live) | 1m, 5m, 4h |
|
||||
| Mark Price | WebSocket | Real-time |
|
||||
| Funding Rate | REST + WS | 8h intervals |
|
||||
| Order Book | WebSocket (optional) | Real-time |
|
||||
|
||||
**Storage Schema** (DuckDB):
|
||||
```sql
|
||||
CREATE TABLE candles (
|
||||
symbol VARCHAR,
|
||||
timeframe VARCHAR,
|
||||
timestamp TIMESTAMP,
|
||||
open DOUBLE,
|
||||
high DOUBLE,
|
||||
low DOUBLE,
|
||||
close DOUBLE,
|
||||
volume DOUBLE,
|
||||
PRIMARY KEY (symbol, timeframe, timestamp)
|
||||
);
|
||||
|
||||
CREATE TABLE trades (
|
||||
id VARCHAR PRIMARY KEY,
|
||||
symbol VARCHAR,
|
||||
side VARCHAR,
|
||||
entry_price DOUBLE,
|
||||
exit_price DOUBLE,
|
||||
quantity DOUBLE,
|
||||
pnl_usdt DOUBLE,
|
||||
pnl_pct DOUBLE,
|
||||
strategy VARCHAR,
|
||||
entry_time TIMESTAMP,
|
||||
exit_time TIMESTAMP
|
||||
);
|
||||
```
|
||||
|
||||
### 4. Strategies (`src/tradefinder/strategies/`)
|
||||
|
||||
Trading strategy implementations with common interface.
|
||||
|
||||
```
|
||||
strategies/
|
||||
├── __init__.py
|
||||
├── base.py # Abstract Strategy interface
|
||||
├── supertrend.py # Trend-following
|
||||
├── squeeze.py # Volatility breakout
|
||||
├── mean_reversion.py # Range-bound
|
||||
└── signals.py # Signal types
|
||||
```
|
||||
|
||||
**Strategy Interface**:
|
||||
```python
|
||||
class Strategy(ABC):
|
||||
@abstractmethod
|
||||
def generate_signal(self, data: pd.DataFrame) -> Signal | None:
|
||||
"""Analyze data and return entry/exit signal."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_stop_loss(self, entry_price: float, side: Side) -> float:
|
||||
"""Calculate stop-loss price for risk sizing."""
|
||||
pass
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def parameters(self) -> dict[str, Any]:
|
||||
"""Current strategy parameters (for UI display)."""
|
||||
pass
|
||||
```
|
||||
|
||||
**Regime-Strategy Mapping**:
|
||||
| Regime | Primary Strategy | Secondary |
|
||||
|--------|------------------|-----------|
|
||||
| Trending | Supertrend | - |
|
||||
| Ranging | Mean Reversion | - |
|
||||
| High Volatility | Squeeze Breakout | - |
|
||||
| Uncertain | Reduce exposure | - |
|
||||
|
||||
### 5. Risk Engine (`src/tradefinder/core/risk.py`)
|
||||
|
||||
Position sizing and risk management.
|
||||
|
||||
**Position Sizing Formula**:
|
||||
```
|
||||
Position Size = (Account Equity × Risk %) / |Entry Price - Stop Loss|
|
||||
|
||||
Example:
|
||||
- Equity: $3000
|
||||
- Risk: 2% = $60
|
||||
- Entry: $100,000 (BTC)
|
||||
- Stop: $98,000
|
||||
- Stop Distance: $2,000
|
||||
|
||||
Position = $60 / $2,000 = 0.03 BTC
|
||||
Notional = 0.03 × $100,000 = $3,000
|
||||
```
|
||||
|
||||
**Risk Limits**:
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| Risk per trade | 1-3% |
|
||||
| Max per strategy | 25% equity |
|
||||
| Max total exposure | 100% equity |
|
||||
| Max leverage | 2x (initial) |
|
||||
|
||||
### 6. Allocator (`src/tradefinder/core/allocator.py`)
|
||||
|
||||
Multi-strategy capital allocation.
|
||||
|
||||
**Allocation Rules**:
|
||||
1. Each strategy gets max 25% of equity
|
||||
2. Hedge mode allows simultaneous long/short
|
||||
3. Reduce allocation during uncertain regimes
|
||||
4. Track "portfolio heat" (total risk exposure)
|
||||
|
||||
### 7. Order Manager (`src/tradefinder/core/orders.py`)
|
||||
|
||||
Order lifecycle management.
|
||||
|
||||
**Order Flow**:
|
||||
```
|
||||
Signal → Risk Check → Size Calculation → Order Submission
|
||||
│
|
||||
┌─────────────────────────┼─────────────────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
[PENDING] [FILLED] [REJECTED]
|
||||
│ │
|
||||
▼ ▼
|
||||
[CANCELLED] [STOP PLACED]
|
||||
│
|
||||
▼
|
||||
[STOP TRIGGERED]
|
||||
```
|
||||
|
||||
**Order Types**:
|
||||
| Purpose | Order Type | Rationale |
|
||||
|---------|------------|-----------|
|
||||
| Entry | Limit | Better fills, avoid slippage |
|
||||
| Stop-Loss | Stop-Market | Guaranteed execution |
|
||||
| Take-Profit | Limit | Optional, better exit price |
|
||||
|
||||
### 8. Regime Classifier (`src/tradefinder/core/regime.py`)
|
||||
|
||||
Market regime detection.
|
||||
|
||||
**Indicators Used**:
|
||||
| Indicator | Purpose | Threshold |
|
||||
|-----------|---------|-----------|
|
||||
| ADX | Trend strength | >25 = trending |
|
||||
| ATR% | Volatility | Relative to historical |
|
||||
| BB Width | Squeeze detection | Narrow = breakout pending |
|
||||
|
||||
**Regime States**:
|
||||
```python
|
||||
class Regime(Enum):
|
||||
TRENDING_UP = "trending_up"
|
||||
TRENDING_DOWN = "trending_down"
|
||||
RANGING = "ranging"
|
||||
HIGH_VOLATILITY = "high_volatility"
|
||||
UNCERTAIN = "uncertain"
|
||||
```
|
||||
|
||||
### 9. UI Layer (`src/tradefinder/ui/`)
|
||||
|
||||
Streamlit dashboard for monitoring.
|
||||
|
||||
```
|
||||
ui/
|
||||
├── __init__.py
|
||||
├── app.py # Main Streamlit app
|
||||
├── pages/
|
||||
│ ├── dashboard.py
|
||||
│ ├── trades.py
|
||||
│ ├── strategies.py
|
||||
│ └── optimizer.py
|
||||
└── components/
|
||||
├── charts.py
|
||||
└── tables.py
|
||||
```
|
||||
|
||||
**Dashboard Sections**:
|
||||
1. **Overview**: Equity, PnL (USDT/CHF/EUR), positions
|
||||
2. **Regime**: Current market state, indicator values
|
||||
3. **Strategies**: Active strategies, parameters, allocation
|
||||
4. **Trades**: Recent trades, performance metrics
|
||||
5. **Optimizer**: Last run results, parameter history
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Trading Loop (Every Signal Interval)
|
||||
```
|
||||
1. Fetch latest candles (4h)
|
||||
2. Update regime classification
|
||||
3. Select active strategy based on regime
|
||||
4. Generate signal from strategy
|
||||
5. If signal:
|
||||
a. Calculate position size (risk engine)
|
||||
b. Check allocation limits
|
||||
c. Submit order (limit entry + stop-market)
|
||||
6. Log state to DuckDB
|
||||
7. Update Redis with current positions
|
||||
```
|
||||
|
||||
### Order Execution (1m/5m Monitoring)
|
||||
```
|
||||
1. Monitor open orders
|
||||
2. If limit not filled within timeout:
|
||||
a. Cancel and retry at new price, OR
|
||||
b. Abandon if price moved too far
|
||||
3. Track fills and update position
|
||||
4. Ensure stop-loss is active
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deployment Architecture
|
||||
|
||||
```yaml
|
||||
# docker-compose services
|
||||
services:
|
||||
engine: # Main trading loop
|
||||
ui: # Streamlit dashboard
|
||||
redis: # Real-time state
|
||||
optimizer: # Weekly Optuna runs
|
||||
backtester: # On-demand backtesting
|
||||
```
|
||||
|
||||
**Volume Mounts**:
|
||||
```
|
||||
/opt/trading/crypto/
|
||||
├── engine/ # Engine logs, state
|
||||
├── optimizer/ # Optimization results
|
||||
├── redis/ # Redis persistence
|
||||
└── shared/ # DuckDB, shared data
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Error Type | Response |
|
||||
|------------|----------|
|
||||
| Exchange API error | Retry with backoff, log, alert if persistent |
|
||||
| Order rejected | Log reason, notify, do not retry blindly |
|
||||
| WebSocket disconnect | Auto-reconnect with backoff |
|
||||
| Invalid signal | Log and skip, continue loop |
|
||||
| Risk limit exceeded | Block order, log warning |
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
See [SECURITY.md](./SECURITY.md) for detailed security guidelines.
|
||||
|
||||
**Key Points**:
|
||||
- API keys in environment variables only
|
||||
- No withdrawal permissions on API keys
|
||||
- Testnet keys separate from production
|
||||
- Secrets never logged
|
||||
Reference in New Issue
Block a user