LLM strategy generation: idea → validated backtest

An LLM can propose a trading strategy in seconds. "Momentum top-30 by 30-day return, vol-targeted at 10% annualized, rebalanced weekly." It will produce a thesis paragraph that reads like a CIO memo and a Python file that looks like a backtest. The numbers it prints — Sharpe 2.3, +94% return, max DD -11% — feel real. The strategy feels real.

The trap is that almost none of those numbers were actually computed. The Python often uses a Sharpe formula that drops the risk-free term or annualizes wrong. It assumes midpoint fills. It silently uses fees the LLM made up. It ignores funding payments — on a perpetual-futures strategy where funding is first-order P&L. It loads a CSV the LLM hasn’t actually seen and computes returns on a window the LLM peeked at while writing the script. The strategy reads as alpha; it is a vibe.

This is not a knock on LLMs — it is a knock on using them as the engine. LLMs are language models. They produce plausible-looking text, including plausible-looking code and plausible-looking numbers. They are not deterministic mathematical engines, and asking them to be one is asking the wrong question.

Keel is a quantitative trading platform for Hyperliquid. The architecture splits the work between agent and engine — agents handle composition, summarization, and code; the engine handles the deterministic math, funding accrual, and execution.

LLM does: ideation, hypothesis framing, library search, graph composition, metric interpretation, iteration proposals, summary writing. Anything that benefits from broad knowledge, natural-language reasoning, and judgment about what to try next. The LLM proposes a structure and reads back results.

Engine does: deterministic math, real Hyperliquid price history, hourly funding accrual on every open position, exchange-accurate fee schedules, modeled slippage, point-in-time universe, bit-for-bit parity between the compiled backtest artifact and the artifact that runs live. Anything where a number has to be exact and reproducible.

The interface between the two halves is MCP — Anthropic’s Model Context Protocol. The LLM calls typed tools (keel_components_search, keel_strategy_compose, keel_backtest_run) the way it would call any other function. The tools execute against the real engine and return structured results the LLM reads back. Composition stays in the LLM; computation stays in the engine.

If you ask an LLM to backtest a strategy with no engine behind it, here are the failures you should expect — in roughly the order of frequency.

• Wrong Sharpe formula. The LLM picks an annualization factor at random (252, 365, 8760 for 15-minute bars). It often drops the risk-free term. It computes on returns it did not actually compute correctly. The number is plausible and wrong.
• Ignored funding payments. On a perpetuals strategy, funding is paid every hour. A backtest that ignores it can show positive Sharpe on a strategy that loses money live to carry. LLMs default to ignoring it because the prompt didn’t mention it.
• Peeked dates. The LLM read about HL’s 2025 volatility regime in its training data. It "tested" a strategy that conveniently switches risk-off during that period — because it knew the answer before writing the test.
• Gut-feel position sizing. "We’ll size at 2% per name" — without computing what the resulting portfolio vol or DD will look like. Vol-targeting requires a feedback loop the LLM doesn’t maintain across the script.
• Fabricated fees and slippage. The LLM picks "5 bps" because it sounds reasonable. Actual HL maker/taker fees and the spread profile across the 200+ listed perps are not what the LLM remembers, if it remembers at all.
• No live parity. The Python script that "passed" is not the artifact that would deploy. There is no executable bridge between the notebook and a real exchange — even if the strategy worked in the notebook, the deployment is from scratch and will diverge.

The mitigation is structural, not behavioral: route every quantitative claim through the engine. The LLM never produces a Sharpe number directly; it calls keel_backtest_run and reads back the engine’s Sharpe. The LLM never decides slippage; the engine’s slippage model decides. The LLM never assumes parity; the same compiled artifact runs in backtest and live.

Concrete loop, with the actual tool calls. The user says: "Build a funding-carry strategy on Hyperliquid and backtest it from 2024 to now."

Claude proposes the thesis (funding-receivers persist, funding-payers mean-revert, gate by funding-volatility regime). It calls keel_components_search with query "funding carry" and finds the funding-data loader, the funding-level regime detector, and the volatility-target weight converter in the typed library. It composes a graph: load price + funding → carry signal → cross-sectional normalization → regime gate → vol-target sizing → portfolio rules → execution. It calls keel_strategy_compose to author the DSL graph and validate types.

Then it calls keel_backtest_run --wait with start 2024-08-15, end 2026-04-30. The deterministic engine runs the strategy against real Hyperliquid history. It returns: Sharpe 2.17, total return +79.6%, max DD -9.7%, sample 2024-08-15 → 2026-04-30. The share URL is app.usekeel.io/share/gDXjURKqWPs8CZ4eXdqAI— a live, inspectable tearsheet anyone can pull up.

The LLM did not compute any of those numbers. It proposed the thesis, composed the graph, called the engine, and read the result back. The Sharpe 2.17, the +79.6%, the -9.7% max DD — all engine numbers, computed against real Hyperliquid data with real fees, real funding accrual, and modeled slippage. The artifact that produced those numbers is the same one that would deploy live (bit-for-bit parity is the engine’s job, not the LLM’s).

A single backtest is not a robustness proof. Before live capital, run the standard rigor checks any quant uses to separate real edge from in-sample fitting.

• Walk-forward optimization: splits the sample into rolling IS/OOS chunks to catch regime dependence. Catches strategies that worked in one window and not the next.
• Monte Carlo: resamples the strategy-return series to give confidence bands around max DD. Catches path-dependence — your "max DD -10%" might have a 25% tail at -20%.
• PBO: when an LLM iterates 20 times on a strategy, the champion is selected from 20 trials. PBO quantifies how much of the IS Sharpe is selection bias.
• DSR: adjusts the champion’s Sharpe for trial count, skew, kurtosis, sample length. Tells you whether the champion is "really 2.0" or "really 0.6 after the multiplicity correction."

Each diagnostic catches a different failure mode. Each link above runs you through the math and ships a calculator that takes a pasted tearsheet or return series and produces the diagnostic.

Two commands. The first installs the Keel MCP server as a Python package. The second registers it with Claude Code.

pipx install keel-trade
claude mcp add keel -- keel mcp serve

Restart Claude. The Keel tools appear in the tool list. First prompt to run:

Build a funding-carry strategy on Hyperliquid
and backtest it from 2024 to 2026.

Claude will call keel_status first, see you are not authenticated, and call keel_auth_login — which opens a browser for OAuth 2.1 + PKCE and captures tokens. Then it will search the component library, compose a graph, run a backtest, and read back the tearsheet. Total time from install to first backtest result: a few minutes.