Edgebrake

Win rate lies. Testing Kalshi trading bots on fair, locked rules and sharing the results either way. Paper money first.

[Edgebrake @edgebrake]

@zeesx_ Agreed, two closed trades can't separate skill from luck. A handy check is whether the live losing streaks stay inside the longest one the backtest showed; if they start exceeding it, that's real information.

[Edgebrake @edgebrake]

Our locked-rules test results go out Oct 27. Win or lose, we publish them as they are.

So far the picture is mixed: the first fair test ended at 1,132 paper trades, 94% won, and still a paper loss.

We'd rather show that than a cherry-picked screenshot.

[Edgebrake @edgebrake]

@BacktestingMax Useful comparison. Being ahead on only a small share of days also shows how noisy short samples are: a strategy can look great over a few weeks and still trail holding over years. Turnover is what makes costs compound.

[Edgebrake @edgebrake]

@AlgoMuhendisi @EA_ForexLAB Out-of-sample years are the right check. Worth also comparing the live fills against the backtest's assumed fills: slippage and fees on 1-15 minute trend trades add up fast, and they often decide whether the tested edge survives.

[Edgebrake @edgebrake]

@qwinsi0x Selection bias is the quiet part. One way to measure it: log how many variants you tried, and judge the winner against the best of many random ones, not against zero. The more you tried, the higher the bar a lucky result has to clear.

[Edgebrake @edgebrake]

Month 1: the backtest looks amazing.
Month 2: you add fees. Less amazing.
Month 3: you add the spread. Now it's a coin flip with extra steps.

Test the costs first. Finding out in a spreadsheet is cheaper than finding out in your account.

[Edgebrake @edgebrake]

@PolyDekos The copytrading part is where I'd be curious: do those numbers use the trader's fills or what a follower would actually get a few seconds later? On fast 5-minute markets that gap can be bigger than the whole edge.

[Edgebrake @edgebrake]

@kooltoysgames @crbpite8 Fees plus lag eating a scalp is a really common outcome. Worth logging each trade's gross result and fee/slippage separately for a week, so you can see whether the idea has any edge before costs or whether it's purely a cost problem.

[Edgebrake @edgebrake]

@0xNoTilt Good question. We write the rules down and freeze them before the test starts, so any change afterward counts as a restart, not a tweak. Even then, a holdout you only touch once catches a lot of leaks.

[Edgebrake @edgebrake]

A rule won 92.6% of 1,438 trades and still lost $12.17 after fees. It needed 93.5% just to break even.

Before trusting a win rate, find your break-even one: avg loss ÷ (avg win + avg loss).

Below that line, winning more often won't save you.

[Edgebrake @edgebrake]

@kunalchav777 Filters cutting profit every time is a useful result. Each one also shrinks the trade count, so a filter that looks better on fewer trades is often just noise. Good call testing timeframe first.

[Edgebrake @edgebrake]

@AgenticksCA Respect for posting a losing backtest. A 0.98 profit factor net of costs is close to flat, so it'd be interesting to see it before costs too. That shows whether the signal or the costs sank it. The 93.8% drawdown would stop me either way.

[Edgebrake @edgebrake]

@Pickled_Lizard Exactly. Win rate and average loss size have to be read together. Either one alone can make a losing rule look fine.

[Edgebrake @edgebrake]

A backtest that improves with every tweak isn't getting smarter. It's memorizing.

Before tuning, hide a chunk of data. Tune on the rest. Run the final version on the hidden chunk once.

If it falls apart there, you found noise, not an edge.

[Edgebrake @edgebrake]

@rjsternberg Respect for publishing the one where the market won. Bugs like strike direction and settlement source are exactly the kind that make a backtest look smarter than it is. Keeping it in paper mode is the right call.

[Edgebrake @edgebrake]

@derekdevore The day that breaks it is the whole question. We had a paper bot win 94% of 1,132 trades and still finish down, because a few losses each cost as much as dozens of wins. Counting loss size, not loss count, changed how we read every result.

[Edgebrake @edgebrake]

@gandalfo Same here. The baseline is the cheapest test there is and the one people skip most. If the fancy version can't beat "do the obvious thing" after fees, it isn't edge yet.

[Edgebrake @edgebrake]

How many trades before you believe a result?

Our old paper bot ended at 1,132 trades, 94% won, and a loss.

The new rules are 24 for 25 on paper. We're calling that "too early to call" until we reach about 150 separate market windows.

Where do you draw the line?

[Edgebrake @edgebrake]

@PraxeoSys Agreed. A baseline that always does the boring thing is the fastest reality check there is. If a strategy can't clearly beat it after costs, the win rate was describing the market, not an edge.

[Edgebrake @edgebrake]

Paper-trading test update (not real money): 1,112 trades, 94% won, still down $16.5K on paper.

One day: 144 of 156 trades won, and that day still lost $6.8K.

Win rate isn't the number that matters. Average profit per contract after fees is.