Compare commits

..

11 Commits

Author SHA1 Message Date
87d31f8d7e Use flat result_summary fields from swym patch 8fb410311
BacktestResult::from_response now reads total_positions, winning_positions,
losing_positions, win_rate, profit_factor, net_pnl, total_pnl, sharpe_ratio,
and total_fees directly from the top-level result_summary object instead of
deriving them from backtest_metadata + balance delta.

Removes the quote/initial_balance parameters that were only needed for the
workaround. Restores the full summary_line format with all metrics.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 11:41:53 +02:00
3892ab37c1 fix: parse actual result_summary structure (backtest_metadata + assets)
The API doc described a flat result_summary that doesn't exist yet in the
deployed backend. The actual shape is:
  { backtest_metadata: { position_count }, assets: [...], condition_audit_summary }

- total_positions from backtest_metadata.position_count
- net_pnl from assets[quote].tear_sheet.balance_end.total - initial_balance
- win_rate, profit_factor, sharpe_ratio, total_fees, avg_bars_in_trade
  remain None until the API adds them

from_response() takes quote and initial_balance again to locate the
right asset and compute PnL. summary_line() only prints metrics that
are actually present. is_promising() falls back to net_pnl>0 + trades
when sharpe is unavailable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 10:32:13 +02:00
85896752f2 fix: ValidationError.path optional, correct position_quantity usage in prompts
- ValidationError.path is Option<String> — the API omits it for top-level
  structural errors. The required String was causing every validate call to
  fail to deserialize, falling through to submission instead of catching errors.

- Log path as "(top-level)" when absent

- Prompts: add explicit CRITICAL note that {"method":"position_quantity"} is
  wrong — position_quantity is an Expr (uses "kind") not a SizingMethod (uses
  "method"). The new SizingMethod examples caused the model to over-apply
  "method" to exits universally across the entire run.

- Prompts: note that fixed_sum has no multiplier field (additionalProperties)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 09:45:17 +02:00
ee260ea4d5 fix: parse flat result_summary structure per updated API doc
The API result_summary is a flat object with top-level fields
(total_positions, win_rate, profit_factor, net_pnl, sharpe_ratio, etc.)
not a nested backtest_metadata/instruments map. This was causing all
metrics to parse as None/zero for every completed run.

- Rewrite BacktestResult::from_response() to read flat fields directly
- Replace parse_ratio_value/parse_decimal_str with a single parse_number()
  that accepts both JSON numbers and decimal strings
- Populate winning_positions, losing_positions, total_fees, avg_bars_in_trade
  (previously always None)
- Simplify from_response signature — exchange/base/quote no longer needed
- Add expected_count and coverage_pct to CandleCoverage struct
- Update all example sell rules to use position_quantity instead of "0.01"
- Note that "9999" is a valid sell-all alias (auto-capped by the API)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 09:37:55 +02:00
3f8d4de7fb feat: add declarative SizingMethod types from upstream schema
Upstream added three new quantity sizing objects alongside DecimalString and Expr:
- fixed_sum: buy N quote-currency worth at current price
- percent_of_balance: buy N% of named asset's free balance
- fixed_units: buy exactly N base units (semantic alias for decimal string)

Update dsl-schema.json to include the three definitions and expand
Action.quantity.oneOf to reference all five valid forms.

Update prompts.rs Quantity section to present the declarative methods
as the preferred approach — they're cleaner, more readable, and
instrument-agnostic compared to raw Expr composition.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 09:33:43 +02:00
7e1ff51ae0 feat: validate endpoint integration, Expr quantity sizing, apply_func input field fix
- Add /api/v1/strategies/validate client to SwymClient; wire into agent loop
  before submission so all DSL errors are surfaced in one round-trip
- Update dsl-schema.json to upstream: quantity is now oneOf[DecimalString, Expr],
  ExprApplyFunc uses "input" field (renamed from "expr")
- Update prompts: document expression-based quantity sizing (fixed-fraction and
  ATR-based examples), fix apply_func to use "input" not "expr" throughout
- Remove unused ValidationError import

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 09:12:12 +02:00
5146b3f764 fix: replace negligible 0.001 quantity with meaningful sizing guidance
The previous example quantity "0.001" represented <1% of the $10k
initial balance for BTC and near-zero exposure for ETH/SOL, making
P&L and Sharpe results statistically meaningless.

- Update Quantity section with instrument-appropriate reference values
  (BTC: 0.01 ≈ $800, ETH: 3.0 ≈ $600, SOL: 50.0 ≈ $700)
- Replace "0.001" with "0.01" in all four working examples
- Explain that 5–10% of $10k initial balance is the sizing target
- Explicitly warn against "0.001" as it produces negligible exposure

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 07:41:28 +02:00
759439313e fix: two Bollinger Band DSL errors from 50-iteration run
- bollinger_upper/lower func Exprs must NOT include a "field" parameter;
  they compute from close internally. Setting "field":"bollinger_upper"
  causes API rejection: expected one of open/high/low/close/volume.
- bollinger Condition "band" only accepts "above_upper" or "below_lower";
  "above_lower" and "below_upper" are invalid variants.

Both errors appeared repeatedly across the 50-iteration run, causing
failed backtest submissions on every Bollinger crossover strategy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 07:39:09 +02:00
9a7761b452 fix: add hma/ma to unsupported list, clarify quantity exit semantics
- Add `hma` (Hull MA) and generic `ma` to unsupported func names —
  both were used by R1 and rejected by the API
- Note that Hull MA can be approximated via apply_func with wma
- Add `"all"` to the quantity placeholder blacklist; explain that exit
  rules must repeat the entry decimal — there is no "close all" concept

Observed in run 2026-03-09T20:10:55: 2 iterations failed on hma/ma,
3 iterations skipped by client-side validation on quantity="all".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 20:23:30 +02:00
8d53d6383d fix: correct DSL mistakes from observed R1 failures
- ADX: clarify it is a FuncName inside {"kind":"func","name":"adx",...},
  not a Condition kind — with inline usage example (ADX > 25 filter)
- Expr "kind" field: add explicit note that every Expr object requires
  "kind"; {"field":"close"} without "kind" is rejected by the API
- MACD: add Example 4 showing full crossover strategy composed from
  bin_op(sub, ema12, ema26) and apply_func(ema,9) as signal line

All three mistakes were observed across consecutive R1-32B runs and
caused repeated API submission failures. Each prompt addition follows
the same pattern as the successful bollinger_upper fix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 20:11:05 +02:00
55e41b6795 fix: log R1 thinking, catch repeated DSL errors, add unsupported indicators
Three improvements from the 2026-03-09T18:45:04 run analysis:

**R1 thinking visibility (claude.rs, agent.rs)**
extract_think_content() returns the raw <think> block content before it
is stripped. agent.rs logs it at DEBUG level so 'RUST_LOG=debug' lets
you see why the model keeps repeating a mistake — currently the think
block is silently discarded after stripping.

**Prompt: unsupported indicators and bollinger_upper Expr mistake (prompts.rs)**
- bollinger_upper / bollinger_lower used as {"kind":"bollinger_upper",...}
  was the dominant failure in iters 9-15. Added explicit correction:
  use {"kind":"func","name":"bollinger_upper","period":20} in Expr context,
  never as a standalone kind.
- roc, hma, vwap, macd, cci, stoch are NOT in the swym schema. Added a
  clear "NOT supported" list alongside the supported func names.

**Repeated API error detection in diagnose_history (agent.rs)**
If the same "unknown variant `X`" error appears 2+ times in the last 4
iterations, a targeted diagnosis note is emitted naming the bad variant
and pointing to the DSL reference. This surfaces in the next iteration
prompt so the model gets actionable feedback before it wastes another
backtest budget on the same mistake.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 18:58:50 +02:00
5 changed files with 422 additions and 64 deletions

View File

@@ -265,6 +265,12 @@ pub async fn run(cli: &Cli) -> Result<()> {
content: response_text.clone(), content: response_text.clone(),
}); });
// Log R1 reasoning chain at debug level so it can be inspected when
// the model makes repeated DSL mistakes (run with RUST_LOG=debug).
if let Some(thinking) = claude::extract_think_content(&response_text) {
debug!("R1 thinking ({} chars):\n{}", thinking.len(), thinking);
}
// Extract strategy JSON // Extract strategy JSON
let strategy = match claude::extract_json(&response_text) { let strategy = match claude::extract_json(&response_text) {
Ok(s) => s, Ok(s) => s,
@@ -319,7 +325,7 @@ pub async fn run(cli: &Cli) -> Result<()> {
let strat_path = cli.output_dir.join(format!("strategy_{iteration:03}.json")); let strat_path = cli.output_dir.join(format!("strategy_{iteration:03}.json"));
std::fs::write(&strat_path, serde_json::to_string_pretty(&strategy)?)?; std::fs::write(&strat_path, serde_json::to_string_pretty(&strategy)?)?;
// Hard validation errors: skip the expensive backtest and give immediate feedback. // Hard client-side validation errors: skip without hitting the API.
if !hard_errors.is_empty() { if !hard_errors.is_empty() {
let record = IterationRecord { let record = IterationRecord {
iteration, iteration,
@@ -332,6 +338,40 @@ pub async fn run(cli: &Cli) -> Result<()> {
continue; continue;
} }
// Server-side validation: call /strategies/validate to get ALL DSL errors
// at once before submitting a backtest. This avoids burning a full backtest
// round-trip on a structurally invalid strategy and gives the model a
// complete list of errors to fix in one shot.
match swym.validate_strategy(&strategy).await {
Ok(api_errors) if !api_errors.is_empty() => {
for e in &api_errors {
warn!(" DSL error at {}: {}", e.path.as_deref().unwrap_or("(top-level)"), e.message);
}
let error_notes: Vec<String> = api_errors
.iter()
.map(|e| format!("DSL error at {}: {}", e.path.as_deref().unwrap_or("(top-level)"), e.message))
.collect();
validation_notes.extend(error_notes);
let record = IterationRecord {
iteration,
strategy: strategy.clone(),
results: vec![],
validation_notes,
};
info!("{}", record.summary());
history.push(record);
continue;
}
Ok(_) => {
// Valid — proceed to backtest
}
Err(e) => {
// Network/parse failure from the validate endpoint — log and proceed
// anyway so a transient API issue doesn't stall the run.
warn!(" strategy validation request failed (proceeding): {e:#}");
}
}
// Run backtests against all instruments (in-sample) // Run backtests against all instruments (in-sample)
let mut results: Vec<BacktestResult> = Vec::new(); let mut results: Vec<BacktestResult> = Vec::new();
@@ -530,13 +570,7 @@ async fn run_single_backtest(
.await .await
.context("poll")?; .context("poll")?;
Ok(BacktestResult::from_response( Ok(BacktestResult::from_response(&final_resp, &inst.symbol))
&final_resp,
&inst.symbol,
&inst.exchange,
&inst.base(),
&inst.quote(),
))
} }
fn save_validated_strategy( fn save_validated_strategy(
@@ -665,6 +699,48 @@ pub fn diagnose_history(history: &[IterationRecord]) -> (String, bool) {
} }
} }
// --- Repeated API error detection ---
// If the same DSL error variant has appeared in 2+ consecutive iterations,
// call it out explicitly so the model knows exactly what to fix.
{
let recent_errors: Vec<String> = history
.iter()
.rev()
.take(4)
.flat_map(|rec| rec.results.iter())
.filter_map(|r| r.error_message.as_deref())
.filter(|e| e.contains("unknown variant"))
.map(|e| {
// Extract the variant name: "unknown variant `foo`, expected ..."
e.split('`')
.nth(1)
.unwrap_or(e)
.to_string()
})
.collect();
if recent_errors.len() >= 2 {
// Find the most frequent bad variant
let mut counts: std::collections::HashMap<&str, usize> = std::collections::HashMap::new();
for v in &recent_errors {
*counts.entry(v.as_str()).or_default() += 1;
}
if let Some((bad_variant, count)) = counts.into_iter().max_by_key(|(_, c)| *c) {
if count >= 2 {
notes.push(format!(
"⚠ DSL ERROR (repeated {count}×): the swym API rejected \
`{bad_variant}` as an unknown variant. \
Check the 'Critical: expression kinds' section — \
`{bad_variant}` may be a FuncName (use inside \
{{\"kind\":\"func\",\"name\":\"{bad_variant}\",...}}) \
or it may not be supported at all. \
Use ONLY the documented kinds and func names."
));
}
}
}
}
// --- Zero-trade check --- // --- Zero-trade check ---
let zero_trade_iters = history let zero_trade_iters = history
.iter() .iter()

View File

@@ -213,6 +213,14 @@ fn lmstudio_context_length(json: &Value, model_id: &str) -> Option<u32> {
None None
} }
/// Return the content of the first `<think>` block, if any.
/// Used for debug logging of R1 reasoning chains.
pub fn extract_think_content(text: &str) -> Option<String> {
let start = text.find("<think>")? + "<think>".len();
let end = text[start..].find("</think>").map(|i| start + i)?;
Some(text[start..end].trim().to_string())
}
/// Extract a JSON object from a model response text. /// Extract a JSON object from a model response text.
/// Handles markdown code fences and R1-style `<think>...</think>` blocks. /// Handles markdown code fences and R1-style `<think>...</think>` blocks.
pub fn extract_json(text: &str) -> Result<Value> { pub fn extract_json(text: &str) -> Result<Value> {

View File

@@ -66,11 +66,48 @@
"properties": { "properties": {
"side": { "type": "string", "enum": ["buy", "sell"] }, "side": { "type": "string", "enum": ["buy", "sell"] },
"quantity": { "quantity": {
"$ref": "#/definitions/DecimalString", "description": "Per-order size in base asset units. Fixed decimal string (e.g. \"0.001\"), a declarative SizingMethod object, or a dynamic Expr object. When a method or Expr returns None the order is skipped; negative values are clamped to zero.",
"description": "Per-order size in base asset units, e.g. \"0.001\" for BTC." "oneOf": [
{ "$ref": "#/definitions/DecimalString" },
{ "$ref": "#/definitions/SizingFixedSum" },
{ "$ref": "#/definitions/SizingPercentOfBalance" },
{ "$ref": "#/definitions/SizingFixedUnits" },
{ "$ref": "#/definitions/Expr" }
]
} }
} }
}, },
"SizingFixedSum": {
"description": "Buy `amount` worth of quote currency at the current price. qty = amount / current_price.",
"type": "object",
"required": ["method", "amount"],
"additionalProperties": false,
"properties": {
"method": { "const": "fixed_sum" },
"amount": { "$ref": "#/definitions/DecimalString", "description": "Quote-currency amount, e.g. \"500\" means buy $500 worth." }
}
},
"SizingPercentOfBalance": {
"description": "Buy percent% of the named asset's free balance worth of base asset. qty = balance(asset) * percent/100 / current_price.",
"type": "object",
"required": ["method", "percent", "asset"],
"additionalProperties": false,
"properties": {
"method": { "const": "percent_of_balance" },
"percent": { "$ref": "#/definitions/DecimalString", "description": "Percentage, e.g. \"2\" means 2% of the free balance." },
"asset": { "type": "string", "description": "Asset name to look up, e.g. \"usdc\". Matched case-insensitively." }
}
},
"SizingFixedUnits": {
"description": "Buy exactly `units` of base asset. Semantic alias for a fixed decimal quantity.",
"type": "object",
"required": ["method", "units"],
"additionalProperties": false,
"properties": {
"method": { "const": "fixed_units" },
"units": { "$ref": "#/definitions/DecimalString", "description": "Base asset quantity, e.g. \"0.01\" means 0.01 BTC." }
}
},
"Rule": { "Rule": {
"type": "object", "type": "object",
"required": ["when", "then"], "required": ["when", "then"],
@@ -280,7 +317,12 @@
{ "$ref": "#/definitions/ExprBinOp" }, { "$ref": "#/definitions/ExprBinOp" },
{ "$ref": "#/definitions/ExprApplyFunc" }, { "$ref": "#/definitions/ExprApplyFunc" },
{ "$ref": "#/definitions/ExprUnaryOp" }, { "$ref": "#/definitions/ExprUnaryOp" },
{ "$ref": "#/definitions/ExprBarsSince" } { "$ref": "#/definitions/ExprBarsSince" },
{ "$ref": "#/definitions/ExprEntryPrice" },
{ "$ref": "#/definitions/ExprPositionQuantity" },
{ "$ref": "#/definitions/ExprUnrealisedPnl" },
{ "$ref": "#/definitions/ExprBarsSinceEntry" },
{ "$ref": "#/definitions/ExprBalance" }
] ]
}, },
"ExprLiteral": { "ExprLiteral": {
@@ -417,6 +459,55 @@
"description": "Maximum bars to look back." "description": "Maximum bars to look back."
} }
} }
},
"ExprEntryPrice": {
"description": "Volume-weighted average entry price of the current open position. Returns None when flat.",
"type": "object",
"required": ["kind"],
"additionalProperties": false,
"properties": {
"kind": { "const": "entry_price" }
}
},
"ExprPositionQuantity": {
"description": "Absolute quantity of the current open position in base asset units. Returns None when flat.",
"type": "object",
"required": ["kind"],
"additionalProperties": false,
"properties": {
"kind": { "const": "position_quantity" }
}
},
"ExprUnrealisedPnl": {
"description": "Estimated unrealised PnL of the current open position in quote asset. Returns None when flat.",
"type": "object",
"required": ["kind"],
"additionalProperties": false,
"properties": {
"kind": { "const": "unrealised_pnl" }
}
},
"ExprBarsSinceEntry": {
"description": "Number of complete primary-interval bars elapsed since the current position was opened. Computed as floor((now - time_enter) / primary_interval_secs). Returns None when flat.",
"type": "object",
"required": ["kind"],
"additionalProperties": false,
"properties": {
"kind": { "const": "bars_since_entry" }
}
},
"ExprBalance": {
"description": "Free balance of the named asset (matched case-insensitively). Returns None when the asset is not found or balance data is unavailable.",
"type": "object",
"required": ["kind", "asset"],
"additionalProperties": false,
"properties": {
"kind": { "const": "balance" },
"asset": {
"type": "string",
"description": "Internal asset name, e.g. \"usdt\", \"btc\". Case-insensitive."
}
}
} }
} }
} }

View File

@@ -52,6 +52,10 @@ sma, ema, wma, rsi, std_dev, sum, highest, lowest, atr, supertrend, adx,
bollinger_upper, bollinger_lower — applied to any candle field (open/high/low/close/volume) bollinger_upper, bollinger_lower — applied to any candle field (open/high/low/close/volume)
with configurable period and optional offset. with configurable period and optional offset.
These are FuncNames used INSIDE `{{"kind":"func","name":"...","period":N}}` expressions.
`atr`, `adx`, and `supertrend` use OHLC internally and ignore the `field` parameter.
To use ADX as a trend-strength filter: `{{"kind":"compare","left":{{"kind":"func","name":"adx","period":14}},"op":">","right":{{"kind":"literal","value":"25"}}}}`
### Composed indicators (apply_func) ### Composed indicators (apply_func)
Apply rolling functions to arbitrary expressions: EMA of EMA, Hull MA (WMA of expression), Apply rolling functions to arbitrary expressions: EMA of EMA, Hull MA (WMA of expression),
VWAP (sum of close*volume / sum of volume), standard deviation of returns, etc. VWAP (sum of close*volume / sum of volume), standard deviation of returns, etc.
@@ -70,11 +74,49 @@ bars_since_entry — complete bars elapsed since position was opened
balance — free balance of a named asset (e.g. "usdt", "usdc") balance — free balance of a named asset (e.g. "usdt", "usdc")
### Quantity ### Quantity
Action quantity MUST be a fixed decimal string that parses as a floating-point number, Action quantity accepts four forms — pick the simplest one for your intent:
e.g. `"quantity": "0.001"`.
NEVER use an expression object for quantity — only plain decimal strings are accepted. **1. Declarative sizing methods (preferred — instrument-agnostic, readable):**
NEVER use placeholder strings like `"ATR_SIZED"`, `"FULL_BALANCE"`, `"percent_of_balance"`,
`"dynamic"`, or any non-numeric string — these will be rejected immediately. Spend a fixed quote amount (e.g. $500 worth of base at current price):
```json
{{"method":"fixed_sum","amount":"500"}}
```
Spend a percentage of free quote balance (e.g. 5% of USDC):
```json
{{"method":"percent_of_balance","percent":"5","asset":"usdc"}}
```
Buy a fixed number of base units (semantic alias for a decimal string):
```json
{{"method":"fixed_units","units":"0.01"}}
```
**2. Plain decimal string** — use only when you have a specific reason:
`"0.01"` (0.01 BTC, 3.0 ETH, 50.0 SOL — instrument-specific, not portable)
**3. Expr** — for dynamic sizing not covered by the methods above, e.g. ATR-based:
```json
{{"kind":"bin_op","op":"div",
"left":{{"kind":"literal","value":"200"}},
"right":{{"kind":"func","name":"atr","period":14}}}}
```
**4. Exit rules** — use `position_quantity` to close the exact open size:
```json
{{"kind":"position_quantity"}}
```
Alternatively, `"9999"` works for exits: sell quantities are automatically capped to the open
position size, so a large fixed number is equivalent to `position_quantity`.
CRITICAL mistakes to never make:
- `{{"method":"position_quantity"}}` is WRONG — `position_quantity` is an Expr, not a SizingMethod.
CORRECT: `{{"kind":"position_quantity"}}`. The `"method"` field belongs ONLY to the three
declarative sizing objects (`fixed_sum`, `percent_of_balance`, `fixed_units`).
- `{{"method":"fixed_sum","amount":"100","multiplier":"2.0"}}` is WRONG — `fixed_sum` has no
`multiplier` field. Only `amount` is accepted alongside `method`.
- NEVER add extra fields to SizingMethod objects — they use `additionalProperties: false`.
### Multi-timeframe ### Multi-timeframe
Any expression can reference a different timeframe via "timeframe" field. Any expression can reference a different timeframe via "timeframe" field.
@@ -146,11 +188,28 @@ Common mistakes to NEVER make:
- `"kind": "bars_since_entry"` is a valid standalone Expr (no extra fields needed). - `"kind": "bars_since_entry"` is a valid standalone Expr (no extra fields needed).
Do NOT put `"bars_since_entry"` as a `"name"` inside `{{"kind":"func",...}}` — that is WRONG. Do NOT put `"bars_since_entry"` as a `"name"` inside `{{"kind":"func",...}}` — that is WRONG.
- `"kind": "expr_field"` does NOT exist. Use `{{"kind":"field","field":"close"}}`. - `"kind": "expr_field"` does NOT exist. Use `{{"kind":"field","field":"close"}}`.
- Every Expr object MUST have a `"kind"` field. `{{"field":"close"}}` is WRONG — missing `"kind"`.
CORRECT: `{{"kind":"field","field":"close"}}`. The `"kind"` is never optional.
- `rsi`, `adx`, `supertrend` are NOT valid inside `apply_func`. Use only `apply_func` - `rsi`, `adx`, `supertrend` are NOT valid inside `apply_func`. Use only `apply_func`
with `ApplyFuncName` values: `highest`, `lowest`, `sma`, `ema`, `wma`, `std_dev`, `sum`, with `ApplyFuncName` values: `highest`, `lowest`, `sma`, `ema`, `wma`, `std_dev`, `sum`,
`bollinger_upper`, `bollinger_lower`. `bollinger_upper`, `bollinger_lower`.
- `volume` is a candle FIELD, not a func name. Access it as `{{"kind":"field","field":"volume"}}`. - `volume` is a candle FIELD, not a func name. Access it as `{{"kind":"field","field":"volume"}}`.
To compute EMA of volume: `{{"kind":"apply_func","name":"ema","period":20,"expr":{{"kind":"field","field":"volume"}}}}`. To compute EMA of volume: `{{"kind":"apply_func","name":"ema","period":20,"input":{{"kind":"field","field":"volume"}}}}`.
- `bollinger_upper` and `bollinger_lower` are FUNC NAMES, not Expr kinds. To compare close to the upper band:
`{{"kind":"compare","left":{{"kind":"field","field":"close"}},"op":">","right":{{"kind":"func","name":"bollinger_upper","period":20}}}}`
NEVER write `{{"kind":"bollinger_upper",...}}` — `bollinger_upper` is not an Expr kind.
NEVER set `"field":"bollinger_upper"` on a func Expr — `bollinger_upper`/`bollinger_lower` have no `field`
parameter; they compute from close internally. Just `{{"kind":"func","name":"bollinger_upper","period":20}}`.
- The `{{"kind":"bollinger",...}}` Condition (shorthand) only accepts `"band": "above_upper"` or
`"band": "below_lower"`. There is NO `above_lower` or `below_upper` — those are invalid and will be
rejected. Use `above_upper` (price above the upper band) or `below_lower` (price below the lower band).
- `adx` is a FUNC NAME, not a Condition kind. To filter for strong trends (ADX > 25):
`{{"kind":"compare","left":{{"kind":"func","name":"adx","period":14}},"op":">","right":{{"kind":"literal","value":"25"}}}}`
NEVER write `{{"kind":"adx",...}}` — `adx` is not a Condition kind, it is a FuncName used inside `{{"kind":"func",...}}`.
- `roc` (rate of change), `hma` (Hull MA), `ma` (generic), `vwap`, `macd`, `cci`, `stoch` are NOT supported.
Use `sma`, `ema`, `wma`, `rsi`, `atr`, `adx`, `supertrend`, `std_dev`, `sum`, `highest`, `lowest`,
`bollinger_upper`, `bollinger_lower` only. There is no generic `ma` — use `sma` or `ema` explicitly.
Hull MA can be approximated as: WMA(2*WMA(n/2) - WMA(n)) using `apply_func`.
## Working examples ## Working examples
@@ -171,7 +230,7 @@ Common mistakes to NEVER make:
{{"kind": "ema_trend", "period": 50, "direction": "above"}} {{"kind": "ema_trend", "period": 50, "direction": "above"}}
] ]
}}, }},
"then": {{"side": "buy", "quantity": "0.001"}} "then": {{"side": "buy", "quantity": "0.01"}}
}}, }},
{{ {{
"comment": "Sell: EMA9 crosses below EMA21, OR 2% stop-loss, OR 72-bar time exit", "comment": "Sell: EMA9 crosses below EMA21, OR 2% stop-loss, OR 72-bar time exit",
@@ -199,7 +258,7 @@ Common mistakes to NEVER make:
}} }}
] ]
}}, }},
"then": {{"side": "sell", "quantity": "0.001"}} "then": {{"side": "sell", "quantity": {{"kind": "position_quantity"}}}}
}} }}
] ]
}} }}
@@ -222,7 +281,7 @@ Common mistakes to NEVER make:
{{"kind": "bollinger", "period": 20, "band": "below_lower"}} {{"kind": "bollinger", "period": 20, "band": "below_lower"}}
] ]
}}, }},
"then": {{"side": "buy", "quantity": "0.001"}} "then": {{"side": "buy", "quantity": "0.01"}}
}}, }},
{{ {{
"comment": "Sell: RSI recovers above 55, OR 3% stop-loss, OR 48-bar time exit", "comment": "Sell: RSI recovers above 55, OR 3% stop-loss, OR 48-bar time exit",
@@ -250,7 +309,7 @@ Common mistakes to NEVER make:
}} }}
] ]
}}, }},
"then": {{"side": "sell", "quantity": "0.001"}} "then": {{"side": "sell", "quantity": {{"kind": "position_quantity"}}}}
}} }}
] ]
}} }}
@@ -277,7 +336,7 @@ Common mistakes to NEVER make:
}} }}
] ]
}}, }},
"then": {{"side": "buy", "quantity": "0.001"}} "then": {{"side": "buy", "quantity": "0.01"}}
}}, }},
{{ {{
"comment": "Sell: 2-ATR stop-loss below entry price, OR 48-bar time exit", "comment": "Sell: 2-ATR stop-loss below entry price, OR 48-bar time exit",
@@ -312,12 +371,102 @@ Common mistakes to NEVER make:
}} }}
] ]
}}, }},
"then": {{"side": "sell", "quantity": "0.001"}} "then": {{"side": "sell", "quantity": {{"kind": "position_quantity"}}}}
}} }}
] ]
}} }}
``` ```
### Example 4 — MACD crossover (composed from primitives)
MACD has no native support, but can be composed from `func` and `apply_func`.
The MACD line is `EMA(12) - EMA(26)`; the signal line is `EMA(9)` of the MACD line.
```json
{{
"type": "rule_based",
"candle_interval": "4h",
"rules": [
{{
"comment": "Buy: MACD line crosses above signal line",
"when": {{
"kind": "all_of",
"conditions": [
{{"kind": "position", "state": "flat"}},
{{
"kind": "cross_over",
"left": {{
"kind": "bin_op", "op": "sub",
"left": {{"kind": "func", "name": "ema", "period": 12}},
"right": {{"kind": "func", "name": "ema", "period": 26}}
}},
"right": {{
"kind": "apply_func", "name": "ema", "period": 9,
"input": {{
"kind": "bin_op", "op": "sub",
"left": {{"kind": "func", "name": "ema", "period": 12}},
"right": {{"kind": "func", "name": "ema", "period": 26}}
}}
}}
}}
]
}},
"then": {{"side": "buy", "quantity": "0.01"}}
}},
{{
"comment": "Sell: MACD crosses below signal, OR 2% stop-loss, OR 72-bar time exit",
"when": {{
"kind": "all_of",
"conditions": [
{{"kind": "position", "state": "long"}},
{{
"kind": "any_of",
"conditions": [
{{
"kind": "cross_under",
"left": {{
"kind": "bin_op", "op": "sub",
"left": {{"kind": "func", "name": "ema", "period": 12}},
"right": {{"kind": "func", "name": "ema", "period": 26}}
}},
"right": {{
"kind": "apply_func", "name": "ema", "period": 9,
"input": {{
"kind": "bin_op", "op": "sub",
"left": {{"kind": "func", "name": "ema", "period": 12}},
"right": {{"kind": "func", "name": "ema", "period": 26}}
}}
}}
}},
{{
"kind": "compare",
"left": {{"kind": "field", "field": "close"}},
"op": "<",
"right": {{"kind": "bin_op", "op": "mul",
"left": {{"kind": "entry_price"}},
"right": {{"kind": "literal", "value": "0.98"}}}}
}},
{{
"kind": "compare",
"left": {{"kind": "bars_since_entry"}},
"op": ">=",
"right": {{"kind": "literal", "value": "72"}}
}}
]
}}
]
}},
"then": {{"side": "sell", "quantity": {{"kind": "position_quantity"}}}}
}}
]
}}
```
Key pattern: `apply_func` wraps any `Expr` tree using the `"input"` field (NOT `"expr"`).
This enables EMA-of-expression (signal line), WMA-of-expression (Hull MA), or std_dev-of-returns.
There is NO native `macd` func name — always compose it as `bin_op(sub, func(ema,12), func(ema,26))` as shown above.
CRITICAL: `apply_func` uses `"input"`, not `"expr"`. Writing `"expr":` will be rejected by the API.
## Anti-patterns to avoid ## Anti-patterns to avoid
- Don't use the same indicator for both entry and exit (circular logic) - Don't use the same indicator for both entry and exit (circular logic)
@@ -328,8 +477,8 @@ Common mistakes to NEVER make:
- Don't ignore fees — a strategy needs to overcome 0.1% per round trip - Don't ignore fees — a strategy needs to overcome 0.1% per round trip
- Always gate buy rules with position state "flat" and sell rules with "long" - Always gate buy rules with position state "flat" and sell rules with "long"
- Never add a short-entry (sell when flat) rule — spot markets are long-only - Never add a short-entry (sell when flat) rule — spot markets are long-only
- Never use an expression object for `quantity` — it must always be a plain decimal string like `"0.001"` - Never use an expression object for `quantity` — it must always be a plain decimal string like `"0.01"`
- Never use a placeholder string for `quantity` — `"ATR_SIZED"`, `"FULL_BALANCE"`, `"dynamic"`, etc. are all invalid and will be rejected. Use `"0.001"` or similar. - Never use a placeholder string for `quantity` — `"ATR_SIZED"`, `"FULL_BALANCE"`, `"dynamic"`, etc. are all invalid and will be rejected. Use `"0.01"` or similar.
"## "##
) )
} }

View File

@@ -4,6 +4,21 @@ use serde::{Deserialize, Serialize};
use serde_json::Value; use serde_json::Value;
use uuid::Uuid; use uuid::Uuid;
/// Response from `POST /api/v1/strategies/validate`.
#[derive(Debug, Deserialize)]
pub struct ValidationResponse {
pub valid: bool,
#[serde(default)]
pub errors: Vec<ValidationError>,
}
#[derive(Debug, Deserialize, Clone)]
pub struct ValidationError {
/// Dotted JSON path to the offending field. Absent for top-level structural errors.
pub path: Option<String>,
pub message: String,
}
/// Client for the swym backtesting API. /// Client for the swym backtesting API.
pub struct SwymClient { pub struct SwymClient {
client: Client, client: Client,
@@ -30,6 +45,8 @@ pub struct CandleCoverage {
pub first_open: String, pub first_open: String,
pub last_close: String, pub last_close: String,
pub count: u64, pub count: u64,
pub expected_count: Option<u64>,
pub coverage_pct: Option<f64>,
} }
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
@@ -52,16 +69,10 @@ pub struct BacktestResult {
} }
impl BacktestResult { impl BacktestResult {
/// Parse a backtest response. /// Parse a backtest response using the flat summary fields added in swym patch 8fb410311.
///
/// `exchange`, `base`, `quote` are needed to derive the instrument key used
/// in the `result_summary.instruments` map (e.g. `binancespot-eth_usdc`).
pub fn from_response( pub fn from_response(
resp: &PaperRunResponse, resp: &PaperRunResponse,
instrument: &str, instrument: &str,
exchange: &str,
base: &str,
quote: &str,
) -> Self { ) -> Self {
let summary = resp.result_summary.as_ref(); let summary = resp.result_summary.as_ref();
if let Some(s) = summary { if let Some(s) = summary {
@@ -70,28 +81,29 @@ impl BacktestResult {
tracing::debug!("[{instrument}] result_summary: null"); tracing::debug!("[{instrument}] result_summary: null");
} }
// The API key for per-instrument stats: "binance_spot" + "eth" + "usdc" → "binancespot-eth_usdc" let total_positions = summary.and_then(|s| s["total_positions"].as_u64().map(|v| v as u32));
let inst_key = format!("{}-{}_{}", exchange.replace('_', ""), base, quote); let winning_positions = summary.and_then(|s| s["winning_positions"].as_u64().map(|v| v as u32));
let losing_positions = summary.and_then(|s| s["losing_positions"].as_u64().map(|v| v as u32));
let total_positions = summary.and_then(|s| { let win_rate = summary.and_then(|s| parse_number(&s["win_rate"]));
s["backtest_metadata"]["position_count"].as_u64().map(|v| v as u32) let profit_factor = summary.and_then(|s| parse_number(&s["profit_factor"]));
}); let net_pnl = summary.and_then(|s| parse_number(&s["net_pnl"]));
let total_pnl = summary.and_then(|s| parse_number(&s["total_pnl"]));
let inst_stats = summary.and_then(|s| s["instruments"].get(&inst_key)); let sharpe_ratio = summary.and_then(|s| parse_number(&s["sharpe_ratio"]));
let total_fees = summary.and_then(|s| parse_number(&s["total_fees"]));
Self { Self {
run_id: resp.id, run_id: resp.id,
instrument: instrument.to_string(), instrument: instrument.to_string(),
status: resp.status.clone(), status: resp.status.clone(),
total_positions, total_positions,
winning_positions: None, winning_positions,
losing_positions: None, losing_positions,
win_rate: inst_stats.and_then(|s| parse_ratio_value(&s["win_rate"])), win_rate,
profit_factor: inst_stats.and_then(|s| parse_ratio_value(&s["profit_factor"])), profit_factor,
total_pnl: inst_stats.and_then(|s| parse_decimal_str(&s["pnl"])), total_pnl,
net_pnl: inst_stats.and_then(|s| parse_decimal_str(&s["pnl"])), net_pnl,
sharpe_ratio: inst_stats.and_then(|s| parse_ratio_value(&s["sharpe_ratio"])), sharpe_ratio,
total_fees: None, total_fees,
avg_bars_in_trade: None, avg_bars_in_trade: None,
error_message: resp.error_message.clone(), error_message: resp.error_message.clone(),
condition_audit_summary: summary.and_then(|s| s.get("condition_audit_summary").cloned()), condition_audit_summary: summary.and_then(|s| s.get("condition_audit_summary").cloned()),
@@ -129,26 +141,22 @@ impl BacktestResult {
} }
/// Is this result promising enough to warrant out-of-sample validation? /// Is this result promising enough to warrant out-of-sample validation?
/// Uses sharpe if available, otherwise falls back to net_pnl > 0.
pub fn is_promising(&self, min_sharpe: f64, min_trades: u32) -> bool { pub fn is_promising(&self, min_sharpe: f64, min_trades: u32) -> bool {
self.status == "complete" if self.status != "complete" { return false; }
&& self.sharpe_ratio.unwrap_or(0.0) > min_sharpe if self.total_positions.unwrap_or(0) < min_trades { return false; }
&& self.total_positions.unwrap_or(0) >= min_trades if self.net_pnl.unwrap_or(0.0) <= 0.0 { return false; }
&& self.net_pnl.unwrap_or(0.0) > 0.0 match self.sharpe_ratio {
Some(sr) => sr > min_sharpe,
None => true, // sharpe absent (e.g. 0 trades); net_pnl + trades is sufficient signal
}
} }
} }
/// Parse a `{"interval": null, "value": "123.45"}` ratio wrapper. /// Parse a numeric JSON value — accepts either a plain JSON number or a decimal string.
/// Returns `None` for null, missing, or sentinel values (Decimal::MAX ≈ 7.9e28). /// Returns `None` for null, missing, or sentinel values (>1e20 in magnitude).
fn parse_ratio_value(v: &Value) -> Option<f64> { fn parse_number(v: &Value) -> Option<f64> {
let s = v.get("value")?.as_str()?; let f = v.as_f64().or_else(|| v.as_str()?.parse().ok())?;
let f: f64 = s.parse().ok()?;
if f.abs() > 1e20 { None } else { Some(f) }
}
/// Parse a plain decimal string JSON value.
/// Returns `None` for null, missing, or sentinel values.
fn parse_decimal_str(v: &Value) -> Option<f64> {
let f: f64 = v.as_str()?.parse().ok()?;
if f.abs() > 1e20 { None } else { Some(f) } if f.abs() > 1e20 { None } else { Some(f) }
} }
@@ -254,6 +262,32 @@ impl SwymClient {
resp.json().await.context("parse candle coverage") resp.json().await.context("parse candle coverage")
} }
/// Validate a strategy against the swym DSL schema.
///
/// Calls `POST /api/v1/strategies/validate` and returns a structured list
/// of all validation errors. Returns `Ok(vec![])` when the strategy is valid.
/// Returns `Err` only on network or parse failures, not on DSL errors.
pub async fn validate_strategy(&self, strategy: &Value) -> Result<Vec<ValidationError>> {
let url = format!("{}/strategies/validate", self.base_url);
let resp = self
.client
.post(&url)
.json(strategy)
.send()
.await
.context("validate strategy request")?;
if !resp.status().is_success() {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
anyhow::bail!("validate strategy {status}: {body}");
}
let parsed: ValidationResponse =
resp.json().await.context("parse validation response")?;
Ok(parsed.errors)
}
/// Submit a backtest run. /// Submit a backtest run.
pub async fn submit_backtest( pub async fn submit_backtest(
&self, &self,