rob thijssen e6d464948f docs+script: document backfill strategy and chunk by quarter
- docs/api.md: add "Backfill strategy" subsection explaining nginx timeout
  risk by interval, how to detect a truncated backfill (non-JSON response),
  the quarterly-chunking approach with a self-contained shell example, and
  a coverage_pct interpretation table for post-backfill verification
- script/backfill.sh: rewrite to iterate in quarterly chunks per
  (instrument, interval), accumulate inserted counts, and print ERR on
  non-JSON responses so truncated requests are immediately visible

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 12:58:05 +02:00
2026-03-09 07:59:59 +02:00

swym-rs

Automated paper trading system built on barter-rs. Connects to live exchange market feeds, runs paper trading sessions, and persists results to PostgreSQL.

Architecture

                  ┌─────────────┐
                  │  paper-cli  │  CLI paper trading tool
                  └─────────────┘

┌──────────────┐  ┌─────────────┐  ┌─────────────────┐
│ market-worker│  │     api     │  │ paper-executor   │
│ (ingestion)  │  │   (axum)    │  │ (run daemon)     │
└──────┬───────┘  └──────┬──────┘  └────────┬─────────┘
       │                 │                   │
       └─────────────────┴───────────────────┘
                         │
                  ┌──────┴──────┐
                  │  swym-dal   │  shared data access layer
                  └──────┬──────┘
                         │
                  ┌──────┴──────┐
                  │ PostgreSQL  │
                  └─────────────┘

Crates & Services

Package Path Description
swym-dal crates/swym-dal Shared data access layer — models, repositories, migrations
paper-cli services/paper-cli CLI tool for running a live paper trading session
market-worker services/market-worker Daemon that ingests live exchange WebSocket feeds into PostgreSQL
swym-api services/api HTTP API for queuing and managing paper trade runs
paper-executor services/paper-executor Background daemon that polls for queued runs and executes them

Prerequisites

  • Rust (edition 2024)
  • PostgreSQL
  • sqlx-cli (optional, for manual migration management)

Quick Start

1. Database Setup

Create a PostgreSQL database:

createdb swym

All services auto-run migrations on startup via swym-dal, so no manual migration step is needed.

2. Paper CLI (standalone, no database required)

Run a 60-second paper trading session against live Binance feeds:

cargo run -p paper-cli

With options:

cargo run -p paper-cli -- \
  --duration 30 \
  --pairs BTCUSDT,ETHUSDT \
  --output json \
  --risk-free-return 0.05

See all options with cargo run -p paper-cli -- --help.

3. Market Worker

Ingest live market data into PostgreSQL:

# Edit config/dev/market_worker.json with your database URL
cargo run -p market-worker -- config/dev/market_worker.json

4. API Server

Start the HTTP API for managing paper trade runs:

# Edit config/dev/api.json with your database URL
cargo run -p swym-api -- config/dev/api.json

Endpoints:

GET  /health                        — health check
POST /api/v1/paper-runs             — queue a new run
GET  /api/v1/paper-runs             — list runs (?status=queued&limit=50&offset=0)
GET  /api/v1/paper-runs/{id}        — get run details and results
POST /api/v1/paper-runs/{id}/cancel — cancel a queued run

Example — queue a paper trade run:

curl -X POST http://localhost:3000/api/v1/paper-runs \
  -H 'Content-Type: application/json' \
  -d '{
    "config": {
      "instrument": {"exchange":"binance_spot","name_exchange":"BTCUSDT","underlying":{"base":"btc","quote":"usdt"},"quote":"underlying_quote","kind":"spot"},
      "execution": {"mocked_exchange":"binance_spot","latency_ms":100,"fees_percent":0.05,"initial_state":{"exchange":"binance_spot","balances":[{"asset":"usdt","balance":{"total":10000,"free":10000},"time_exchange":"2025-03-24T21:30:00Z"}],"instrument":{"instrument_name":"BTCUSDT","orders":[]}}}
    },
    "starts_at": "2025-03-24T21:30:00Z",
    "finishes_at": "2025-03-24T21:31:00Z",
    "risk_free_return": 0.05
  }'

5. Paper Executor

Start the background executor to process queued runs:

# Edit config/dev/paper_executor.json with your database URL
cargo run -p paper-executor -- config/dev/paper_executor.json

The executor polls for queued runs, claims them with FOR UPDATE SKIP LOCKED, executes the paper trading session, and writes the TradingSummary back to the database.

Configuration

Config files are organised under config/<env>/ (e.g. config/dev/). Each service takes a JSON config file as its first argument. All configs that require database access share the same database block:

{
  "database": {
    "url": "postgres://user:pass@localhost/swym",
    "max_connections": 5
  }
}

The DATABASE_URL environment variable can also be used for swym-dal pool configuration.

Deploy

script/deploy.sh builds and installs all services for a target environment.

# Build and deploy to dev
script/deploy.sh dev

# Drop all database objects (wipes tables, types, indexes, migrations) then exit
script/deploy.sh dev drop

The drop subcommand reads the database URL from config/<env>/api.json (requires jq and psql), runs DROP SCHEMA public CASCADE; CREATE SCHEMA public;, and exits without building or deploying. On the next deploy, the services will auto-run migrations against the fresh schema.

Database Schema

Managed via sqlx migrations in crates/swym-dal/migrations/:

  • exchanges — reference table for exchanges
  • instruments — reference table for trading instruments (exchange + symbol)
  • market_trades — one row per public trade (DOUBLE PRECISION price/amount)
  • market_order_book_l1 — one row per L1 order book snapshot (NUMERIC price/amount)
  • paper_runs — paper trade run queue with lifecycle status (queued / running / complete / failed / cancelled)
  • strategies — deduplicated strategy definitions keyed by content hash; linked to paper_runs via strategy_id

Strategies

Paper runs are configured via the strategy field in the run config. The paper-executor deserialises this to select a strategy implementation.

Available Strategies

Strategy "type" Description
default "default" No-op — places no orders
simple_spread "simple_spread" Market-making: places a buy and a sell at fixed spread around mid price
ema_crossover "ema_crossover" Trend-following: buys on fast/slow EMA golden cross, sells on death cross
rsi_reversal "rsi_reversal" Mean-reversion: buys when RSI is oversold, sells when overbought
bollinger_breakout "bollinger_breakout" Momentum: buys on upper-band breakout, closes when price falls below lower band
rule_based "rule_based" Declarative DSL: define arbitrary entry/exit logic as JSON rules evaluated on each candle close

Rule-Based Strategies

The rule_based type lets you define trading logic entirely in JSON without writing Rust. Rules are evaluated on every candle close. All rules whose when condition is true simultaneously fire their then action.

Condition reference

Conditions are tagged with a "kind" field:

kind Parameters True when
ema_crossover fast_period, slow_period, direction ("above"/"below") Fast EMA crossed above/below slow EMA on this candle
ema_trend period, direction ("above"/"below") Close price is above/below the EMA
rsi period (default 14), threshold, comparison ("above"/"below") RSI is above/below threshold
bollinger period (default 20), num_std_dev (default 2), band ("above_upper"/"below_lower") Price broke above upper or below lower Bollinger Band
price_level price, direction ("above"/"below") Close price is above/below a fixed level
position state ("flat"/"long"/"short") Current position matches the required state
all_of conditions: [...] All sub-conditions are true (logical AND)
any_of conditions: [...] Any sub-condition is true (logical OR)
not condition: {...} Sub-condition is false

Example — EMA crossover via rule DSL

This is equivalent to the compiled ema_crossover strategy but expressed in the rule DSL, with independent buy and sell rules:

{
  "type": "rule_based",
  "candle_interval": "1h",
  "rules": [
    {
      "when": { "kind": "all_of", "conditions": [
        { "kind": "ema_crossover", "fast_period": 12, "slow_period": 26, "direction": "above" },
        { "kind": "position", "state": "flat" }
      ]},
      "then": { "side": "buy", "quantity": "0.001" }
    },
    {
      "when": { "kind": "all_of", "conditions": [
        { "kind": "ema_crossover", "fast_period": 12, "slow_period": 26, "direction": "below" },
        { "kind": "position", "state": "long" }
      ]},
      "then": { "side": "sell", "quantity": "0.001" }
    }
  ]
}

Example — Liquidity Sweep Strategy

A liquidity sweep strategy detects when price sweeps through a recent swing high or low and immediately reverses — a common institutional order flow pattern.

The original PineScript implementation looks like:

lookback = 20
highestHigh = ta.highest(high, lookback)
lowestLow   = ta.lowest(low, lookback)

longCondition  = low < lowestLow[1]  and close > high[1]
shortCondition = high > highestHigh[1] and close < low[1]

In the swym rule DSL this becomes two rules — one long, one short. The func expression computes a rolling-window function over a candle OHLCV field; the field expression references a raw OHLCV value, with an optional "offset" (bars ago):

{
  "type": "rule_based",
  "candle_interval": "1h",
  "rules": [
    {
      "comment": "Enter long: price sweeps below the 20-period low then closes above the prior candle's high",
      "when": { "kind": "all_of", "conditions": [
        { "kind": "position", "state": "flat" },
        {
          "kind": "compare",
          "left":  { "kind": "field", "field": "low" },
          "op":    "<",
          "right": { "kind": "func", "name": "lowest", "field": "low", "period": 20, "offset": 1 }
        },
        {
          "kind": "compare",
          "left":  { "kind": "field", "field": "close" },
          "op":    ">",
          "right": { "kind": "field", "field": "high", "offset": 1 }
        }
      ]},
      "then": { "side": "buy", "quantity": "0.001" }
    },
    {
      "comment": "Exit long: opposite sweep signal fires while long",
      "when": { "kind": "all_of", "conditions": [
        { "kind": "position", "state": "long" },
        {
          "kind": "compare",
          "left":  { "kind": "field", "field": "high" },
          "op":    ">",
          "right": { "kind": "func", "name": "highest", "field": "high", "period": 20, "offset": 1 }
        },
        {
          "kind": "compare",
          "left":  { "kind": "field", "field": "close" },
          "op":    "<",
          "right": { "kind": "field", "field": "low", "offset": 1 }
        }
      ]},
      "then": { "side": "sell", "quantity": "0.001" }
    },
    {
      "comment": "Enter short: price sweeps above the 20-period high then closes below the prior candle's low",
      "when": { "kind": "all_of", "conditions": [
        { "kind": "position", "state": "flat" },
        {
          "kind": "compare",
          "left":  { "kind": "field", "field": "high" },
          "op":    ">",
          "right": { "kind": "func", "name": "highest", "field": "high", "period": 20, "offset": 1 }
        },
        {
          "kind": "compare",
          "left":  { "kind": "field", "field": "close" },
          "op":    "<",
          "right": { "kind": "field", "field": "low", "offset": 1 }
        }
      ]},
      "then": { "side": "sell", "quantity": "0.001" }
    },
    {
      "comment": "Exit short: opposite sweep signal fires while short",
      "when": { "kind": "all_of", "conditions": [
        { "kind": "position", "state": "short" },
        {
          "kind": "compare",
          "left":  { "kind": "field", "field": "low" },
          "op":    "<",
          "right": { "kind": "func", "name": "lowest", "field": "low", "period": 20, "offset": 1 }
        },
        {
          "kind": "compare",
          "left":  { "kind": "field", "field": "close" },
          "op":    ">",
          "right": { "kind": "field", "field": "high", "offset": 1 }
        }
      ]},
      "then": { "side": "buy", "quantity": "0.001" }
    }
  ]
}

Expression kinds:

kind Fields Evaluates to
literal value The constant numeric value
field field (open/high/low/close/volume), offset (default 0) That OHLCV field from N bars ago
func name, field, period, offset (default 0) Rolling-window function result
bin_op op (add/sub/mul/div), left, right Arithmetic on two sub-expressions

Comparison kinds (use as conditions directly):

kind Fields True when
compare left, op (>/</>=/<=/==), right Numeric comparison holds
cross_over left, right left crossed above right on this candle
cross_under left, right left crossed below right on this candle

Available functions ("name" values for func):

Name Description
highest Maximum of field over the last period candles
lowest Minimum of field over the last period candles
sma Simple moving average of field over period candles
ema Exponential moving average of field over period candles
rsi RSI of field over period candles (Wilder's smoothing)
std_dev Population standard deviation of field over period candles

Submitting the Liquidity Sweep as a Backtest

Ensure candle data is available for the target instrument and interval first:

# Check available 1h candle data range for BTCUSDT
curl http://localhost:3000/api/v1/market-candles/range/binance_spot/BTCUSDT?interval=1h

Then submit the backtest:

curl -X POST http://localhost:3000/api/v1/paper-runs \
  -H 'Content-Type: application/json' \
  -d '{
    "mode": "backtest",
    "starts_at": "2025-01-01T00:00:00Z",
    "finishes_at": "2025-03-01T00:00:00Z",
    "config": {
      "instrument": {
        "exchange": "binance_spot",
        "name_exchange": "BTCUSDT",
        "underlying": { "base": "btc", "quote": "usdt" },
        "quote": "underlying_quote",
        "kind": "spot"
      },
      "execution": {
        "mocked_exchange": "binance_spot",
        "latency_ms": 50,
        "fees_percent": 0.001,
        "initial_state": {
          "exchange": "binance_spot",
          "balances": [
            { "asset": "usdt", "balance": { "total": 10000, "free": 10000 }, "time_exchange": "2025-01-01T00:00:00Z" }
          ],
          "instrument": { "instrument_name": "BTCUSDT", "orders": [] }
        }
      },
      "strategy": {
        "type": "rule_based",
        "candle_interval": "1h",
        "rules": [
          {
            "comment": "Enter long",
            "when": { "kind": "all_of", "conditions": [
              { "kind": "position", "state": "flat" },
              { "kind": "compare", "left": { "kind": "field", "field": "low" }, "op": "<", "right": { "kind": "func", "name": "lowest", "field": "low", "period": 20, "offset": 1 } },
              { "kind": "compare", "left": { "kind": "field", "field": "close" }, "op": ">", "right": { "kind": "field", "field": "high", "offset": 1 } }
            ]},
            "then": { "side": "buy", "quantity": "0.001" }
          },
          {
            "comment": "Exit long",
            "when": { "kind": "all_of", "conditions": [
              { "kind": "position", "state": "long" },
              { "kind": "compare", "left": { "kind": "field", "field": "high" }, "op": ">", "right": { "kind": "func", "name": "highest", "field": "high", "period": 20, "offset": 1 } },
              { "kind": "compare", "left": { "kind": "field", "field": "close" }, "op": "<", "right": { "kind": "field", "field": "low", "offset": 1 } }
            ]},
            "then": { "side": "sell", "quantity": "0.001" }
          },
          {
            "comment": "Enter short",
            "when": { "kind": "all_of", "conditions": [
              { "kind": "position", "state": "flat" },
              { "kind": "compare", "left": { "kind": "field", "field": "high" }, "op": ">", "right": { "kind": "func", "name": "highest", "field": "high", "period": 20, "offset": 1 } },
              { "kind": "compare", "left": { "kind": "field", "field": "close" }, "op": "<", "right": { "kind": "field", "field": "low", "offset": 1 } }
            ]},
            "then": { "side": "sell", "quantity": "0.001" }
          },
          {
            "comment": "Exit short",
            "when": { "kind": "all_of", "conditions": [
              { "kind": "position", "state": "short" },
              { "kind": "compare", "left": { "kind": "field", "field": "low" }, "op": "<", "right": { "kind": "func", "name": "lowest", "field": "low", "period": 20, "offset": 1 } },
              { "kind": "compare", "left": { "kind": "field", "field": "close" }, "op": ">", "right": { "kind": "field", "field": "high", "offset": 1 } }
            ]},
            "then": { "side": "buy", "quantity": "0.001" }
          }
        ]
      }
    }
  }'

The response includes a strategy_id — a stable UUID derived from the strategy's signal logic. Resubmitting the same rules against a different instrument, candle interval, or account balance will return the same strategy_id, linking all runs for cross-instrument performance comparison.

To check the result:

# Poll until status is "complete" or "failed"
curl http://localhost:3000/api/v1/paper-runs/<id>

# Fetch the equity curve (sampled to 500 points)
curl 'http://localhost:3000/api/v1/paper-runs/<id>/positions?samples=500'

Strategy Identity and Hashing

Every run is linked to a strategies table row via strategy_id. The identity hash is computed from the strategy's signal logic — candle_interval, order_quantity, and sizing parameters are excluded, so the same rules at different timeframes or position sizes share the same identity. This enables future performance scoring across instruments and timeframes.

To compare all runs for a strategy:

SELECT pr.id, pr.starts_at, pr.finishes_at, pr.candle_interval,
       pr.config->'instrument'->>'name_exchange' AS instrument,
       pr.result_summary->'sharpe_ratio' AS sharpe
FROM paper_runs pr
WHERE pr.strategy_id = '<strategy-uuid>'
ORDER BY pr.created_at DESC;

Comparing Results to Reference Platforms

Each indicator-based strategy has a textbook equivalent on backtrader, freqtrade, and vectorbt. The implementations in services/paper-executor/src/strategy/indicators.rs match these canonical formulas:

Indicator Swym implementation Reference equivalent
EMA Seeded with SMA of first period prices, then k = 2 / (period + 1) multiplier pd.Series.ewm(span=N, adjust=False).mean() / bt.indicators.EMA(period=N)
RSI Wilder's smoothing: simple average seed for first period changes, then (prev × (N-1) + curr) / N bt.indicators.RSI(period=N) with cutler=False (default)
Bollinger Bands Population stddev (divide by N, not N-1) over trailing period prices bt.indicators.BollingerBands(period=N, devfactor=K)

Tick-vs-Candle Divergence

Reference platforms (backtrader, freqtrade, vectorbt) compute indicators on candle closes. Swym computes on every trade tick, so indicator values update more frequently and will diverge numerically from candle-based results even with identical parameters.

For exact numerical parity you would need to aggregate ticks into synthetic candles (e.g. 1-minute OHLCV) before computing. The strategy logic (entry/exit conditions) is structurally equivalent; only the timing and magnitude of individual signals will differ.

Default Strategy Parameters

Strategy Parameter Default Notes
ema_crossover fast_period 12 Matches MACD short EMA
slow_period 26 Matches MACD long EMA
interval_secs 5 Minimum seconds between orders
rsi_reversal rsi_period 14 Wilder's standard period
oversold 30 Buy threshold
overbought 70 Sell threshold
interval_secs 5
bollinger_breakout period 20 Standard Bollinger period
num_std_dev 2.0 Band width
interval_secs 5

Tech Stack

  • barter-rs — trading engine, market data streams, execution
  • sqlx — async PostgreSQL with compile-time query checking
  • axum — HTTP API framework
  • clap — CLI argument parsing
  • tokio — async runtime
Description
No description provided
Readme 3.5 MiB
Languages
Rust 57.1%
TypeScript 39.2%
Shell 3.3%
CSS 0.2%
HTML 0.1%