rob thijssen e27aabae34 feat(agent): improve LLM feedback loop and convergence detection
Three related improvements to help the model learn and explore effectively:

Strategy JSON in history: include the compact strategy JSON in each
IterationRecord::summary() so the LLM knows exactly what was tested in
every past iteration, not just the outcome metrics. Without this the model
had no record of what it tried once conversation history was trimmed.

Rule comment in audit: include rule_comment from the condition audit in
the formatted audit string so the LLM can correlate hit-rate data with
the rule's stated purpose.

Convergence detection and anti-anchoring: diagnose_history() now returns
(String, bool) where the bool signals that the last 3 iterations had
avg_sharpe spread < 0.03 (model stuck in local optimum). When converged:
- Emit a ⚠ CONVERGENCE DETECTED note listing untried candle intervals
- Suppress best_so_far JSON to break the anchoring effect that was
  causing the model to produce near-identical strategies for 13+ iterations
- Targeted "try a different approach" instruction

Also add volume-as-field clarification to the DSL mistakes section in
the system prompt, fixing the "unknown variant `volume`" submit error.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 14:38:07 +02:00
2026-03-09 10:15:33 +02:00
2026-03-09 10:15:33 +02:00
2026-03-09 10:15:33 +02:00
2026-03-09 12:24:30 +02:00

scout

Autonomous strategy search agent for the swym backtesting platform.

Runs a loop: asks Claude to generate trading strategies → submits backtests to swym → evaluates results → feeds learnings back → repeats. Promising strategies are automatically validated on out-of-sample data to filter overfitting.

Quick start

export ANTHROPIC_API_KEY="sk-ant-..."

cargo run -- \
  --swym-url https://dev.swym.hanzalova.internal/api/v1 \
  --max-iterations 50 \
  --instruments binance_spot:BTCUSDC,binance_spot:ETHUSDC,binance_spot:SOLUSDC \
  --backtest-from 2025-01-01T00:00:00Z \
  --backtest-to 2025-10-01T00:00:00Z \
  --oos-from 2025-10-01T00:00:00Z \
  --oos-to 2026-03-01T00:00:00Z

How it works

  1. Coverage check — verifies candle data exists for all instruments and finds common available intervals.

  2. Strategy generation — sends the DSL schema + prior results to Claude, which produces a new strategy JSON each iteration.

  3. In-sample backtest — submits the strategy against all instruments for the training period. Evaluates Sharpe ratio, profit factor, win rate, net PnL.

  4. Out-of-sample validation — if any instrument shows Sharpe > threshold with enough trades, the strategy is re-tested on held-out data. Only strategies that pass both phases are saved as "validated".

  5. Learning loop — all results (including failures) are fed back to Claude so it can learn from what works and what doesn't. The conversation is trimmed to avoid context exhaustion while the full results history is passed as structured text.

Configuration

All options are available as CLI flags and environment variables:

Flag Env Default Description
--swym-url SWYM_API_URL https://dev.swym.hanzalova.internal/api/v1 Swym API base URL
--anthropic-key ANTHROPIC_API_KEY required Anthropic API key
--model CLAUDE_MODEL claude-sonnet-4-20250514 Claude model
--max-iterations 50 Maximum search iterations
--min-sharpe 1.0 Minimum Sharpe for "promising"
--min-trades 10 Minimum trades for significance
--instruments BTC,ETH,SOL vs USDC Comma-separated exchange:SYMBOL
--backtest-from 2025-01-01 In-sample start
--backtest-to 2025-10-01 In-sample end
--oos-from 2025-10-01 Out-of-sample start
--oos-to 2026-03-01 Out-of-sample end
--initial-balance 10000 Starting USDC balance
--fees-percent 0.001 Fee per trade (0.1%)
--output-dir ./scout-results Where to save strategies and reports

Output

scout-results/
├── strategy_001.json      # Every strategy attempted
├── strategy_002.json
├── ...
├── validated_017.json     # Strategies that passed OOS validation
├── validated_031.json     # (includes in-sample + OOS metrics)
└── best_strategy.json     # Highest avg Sharpe across instruments

Tips

  • Start with Sonnet (claude-sonnet-4-20250514) for cost efficiency during exploration. Switch to Opus for refinement of promising strategies.

  • 50 iterations is a reasonable starting point. The agent typically finds interesting patterns within 20-30 iterations if they exist.

  • Watch the logs — the per-iteration summaries show you what the agent is learning in real time.

  • Adjust dates to match your actual candle coverage. The agent checks coverage at startup and will fail fast if data is missing.

  • The OOS validation threshold is intentionally relaxed (70% of in-sample Sharpe, half the trade count) because out-of-sample degradation is expected. Strategies that maintain edge through this filter are genuinely interesting.

Description
No description provided
Readme 509 KiB
Languages
Rust 100%