# Binance Futures Support for S3 Kline Ingestion

## Context

With short position support now implemented in the DSL engine, the next step is enabling
futures market data. The paper-executor already handles shorts correctly at the strategy level,
but market data ingestion is spot-only: the S3 fetcher hardcodes `data/spot/` paths, and the
worker has no concept of exchange type or instrument kind.

Binance provides futures data on S3 at `data.binance.vision` under:
- `data/futures/um/` — USD-M perpetuals (e.g., BTCUSDT)
- `data/futures/cm/` — COIN-M perpetuals (e.g., BTCUSD_PERP)

The API layer already supports futures exchanges (`BinanceFuturesUsd`, `BinanceFuturesCoin`)
for instrument lookup and REST kline backfill. The DB `instruments.kind` column is free-form
TEXT and can already store `"perpetual"`. The changes needed are confined to the ingestion
pipeline and configuration.

**Non-goal for this phase:** margin/leverage simulation in MockExchange. The barter-rs
MockExchange uses simple balance checks with no margin model. Adding leverage is a large,
independent effort. For now, futures backtests will work identically to spot — the user
provides a base quantity and the engine trades it 1:1. This is sufficient for directional
strategy validation.

## 1. Add `source_type` to `ingest_configs` table

**File:** `crates/swym-dal/migrations/` — new migration

Add a column to distinguish spot from futures ingestion:

```sql
ALTER TABLE ingest_configs
ADD COLUMN source_type TEXT NOT NULL DEFAULT 'spot'
CHECK (source_type IN ('spot', 'futures_um', 'futures_cm'));
```

Default `'spot'` ensures backward compatibility — existing rows continue working unchanged.

**File:** `crates/swym-dal/src/models/ingest_config.rs`

Add `source_type: String` to `IngestConfigRow` / `ActiveIngestConfigRow`.

**File:** `crates/swym-dal/src/repo/ingest_config.rs`

Add `source_type` to SELECT column lists and the `create` function parameters.

## 2. Parameterize S3 paths in fetcher

**File:** `services/ingest-binance/src/fetcher.rs`

The fetcher currently hardcodes:
```
data/spot/monthly/trades/{SYMBOL}/
data/spot/daily/trades/{SYMBOL}/
```

Refactor to accept a `source_type` parameter and build paths accordingly:

```rust
fn s3_base_path(source_type: &str) -> &'static str {
    match source_type {
        "futures_um" => "data/futures/um",
        "futures_cm" => "data/futures/cm",
        _ => "data/spot",
    }
}
```

Then use `{base}/monthly/trades/{symbol}/` and `{base}/daily/trades/{symbol}/` patterns.

Also parameterize the epoch constant:
- Spot: `2017-07-01`
- Futures UM: `2019-09-08` (USD-M launch)
- Futures CM: `2020-08-10` (COIN-M launch)

## 3. Thread `source_type` through the worker

**File:** `services/ingest-binance/src/worker.rs`

`ActiveIngestConfigRow` needs the `source_type` field. The worker passes it through to
the fetcher when building S3 paths. The poll query in the DAL already returns all columns
from `ingest_configs`, so adding the column to the model struct is sufficient.

## 4. API endpoint for creating futures ingest configs

**File:** `services/api/src/handlers/` (ingest config handler)

The existing create-ingest-config endpoint needs to accept an optional `source_type` field
(defaulting to `"spot"`) so users can create futures ingestion jobs via the API.

## 5. Documentation

**File:** `docs/api.md`

- Document `source_type` field on ingest config endpoints
- Add a note explaining that futures backtests use the same execution model as spot
  (no margin/leverage simulation) — the difference is only in the market data source
- Add example of creating a futures UM ingest config

## Files to modify

1. `crates/swym-dal/migrations/` — new migration adding `source_type` column
2. `crates/swym-dal/src/models/ingest_config.rs` — add field to row structs
3. `crates/swym-dal/src/repo/ingest_config.rs` — add field to queries and `create` fn
4. `services/ingest-binance/src/fetcher.rs` — parameterize S3 paths and epoch
5. `services/ingest-binance/src/worker.rs` — thread `source_type` to fetcher
6. `services/api/src/handlers/` — accept `source_type` in create endpoint
7. `docs/api.md` — document new field and futures notes

No changes to paper-executor, barter-rs, or the strategy DSL.

## Verification

```bash
cargo check -p swym-ingest-binance -p swym-api -p swym-dal
cargo clippy -p swym-ingest-binance -p swym-api -p swym-dal
```

After deploying:
1. Create a futures UM ingest config via API: `POST /api/v1/ingest-configs` with
   `"source_type": "futures_um"` and instrument `BTCUSDT`
2. Verify `ingest-binance` picks it up and fetches from `data/futures/um/daily/trades/BTCUSDT/`
3. Verify ingested trades appear in `market_trades` and can be used for candle backtests
4. Run an existing spot ingest config and confirm no regression
