Honest question. I built AsOf to catch the future leaking into a backtest. But a check has to live somewhere in the pipeline. Would you put it at data ingest, at training, or right before the backtest runs, and why there?
Published trading edges shrink. A study of 97 return predictors found returns 26% lower out of sample and 58% lower after publication.
Now let an AI agent try thousands of variations on the same data.
The question that matters: what was the model allowed to know, and when?
Open source check: https://t.co/ILegchP6ep
Most backtests don't fail because the model is bad.
They fail because of what the model was allowed to know.
Here are the three things almost nobody checks:
1. The timestamp trap
A daily number describes a day but is published after it ends. Stamp it with the day it describes and your backtest thinks you knew it that morning. Nothing crashes. The results just look great.
2. The revision trap
Economic series get revised. Use today's value as if it existed then and you are trading on the future.
3. The count trap
Try enough variants and one will look brilliant. One study found that with 5 years of data, as few as 45 configurations can produce a backtest that means nothing (Bailey, Borwein, Lopez de Prado and Zhu, 2014). New factors should clear a t ratio of 3.0, not the usual 2.0 (Harvey, Liu and Zhu, 2016).
The cost is real. Across 97 published return predictors, returns were 26% lower out of sample and 58% lower after publication (McLean and Pontiff, 2016).
AI agents make all three easier. They pull their own data, and they can try thousands of variants before anyone counts.
The fix is simple to say and hard to do: record when something happened, when you could have known, and when you recorded it. Serve only what was available at decision time. Log what was blocked. Count every try.
That is what I'm building at Strathora, in the open (strathora/asof on GitHub).
I'm not selling anything yet. By early December I want 10 to 15 conversations with people who build, buy or review strategies:
• Manager selection and due diligence
• Prop firms and emerging manager platforms
• Quant teams and crypto shops
• Teams using AI agents in trading or research
If that's you, or you know someone who is, comment or message me. 15 minutes, or a few questions by message.
What is the worst leak you have found in your own backtest?
AI agents that pull their own data can drift into later information unnoticed. A time lock and a receipt show they did not. Before you buy a strategy or an agent, ask for the three timestamps, what was blocked, and how many variants were tried. strathora/asof on GitHub.
The nastiest bug in quant research is not a bad model. It is data stamped a day early. A daily number describes a day but is published after it ends. Stamp it with that day and your backtest thinks you knew it that morning. Nothing crashes. It just looks great.
Hypothetical!!!
Tuesday’s bar only exists after Tuesday closes, so a strategy trading Tuesday’s open that uses it has one day of look ahead. Revisions are the same trap. Use today’s revised value as if it was known then and you are using the future.
The fix?
record when it happened, when you could have known, and when you recorded it. Then make it mechanical. At decision time T, serve only what was available by T, block the rest, and log every block as a receipt. A leak becomes a number you can count.
Most backtests are lying to you, and the research says so.
Three studies worth knowing:
→ McLean and Pontiff looked at 97 published return predictors. Out of sample, returns were 26% lower. After publication, 58% lower.
→ Bailey, Borwein, Lopez de Prado and Zhu showed that with 5 years of data, as few as 45 strategy variations can be enough to produce a backtest that looks great and means nothing.
→ Harvey, Liu and Zhu argued that with hundreds of factors tested, a new discovery should clear a t ratio of 3.0, not the usual 2.0.
Now give an AI agent the same job. It can try thousands of variations a day, and its training data may already contain the answer.
So the question is no longer “does it backtest well?” It is “can you prove what the model knew, and when?”
That is what we are building at Strathora. Every record carries three clocks: when it happened, when it could be known, and when it was recorded. The clock only moves forward, and the ledger is hash chained, so changing one byte breaks verification.
Our open source time lock, AsOf, is live: https://t.co/ILegchOyoR
Prove your AI didn’t see the future.
#QuantFinance #AIinFinance #Backtesting #OpenSource #Strathora
Be honest, have you ever checked what your prompts are telling your model about the future?
Your data can be clean and the leak can still be sitting right there in the prompt.
where would you look first?
Harvey, Liu and Zhu (2014) looked at hundreds of published trading factors and concluded most of them are probably false.
Not unlucky. False. Hundreds of strategies, all mined from the same history, all judged by a bar built for a single test. A t-ratio of 2.0 passes — until you count how many tried. Then the bar is 3.0, and most of them don't clear it.
The backtest doesn't know it's lying. It just reports the winner.
Strathora's AsOf is built for exactly this — check every input, prompt, and file against the clock before you trust the number. Check the data. Lock the clock. Seal the run. Prove it.
Your Sharpe is lying to you.
Every point of it could be borrowed from the future — a revised number here, a dead ticker there, a timestamp off by a day. Strip the leaks out and the genius disappears.
IT WAS NEVER SKILL.
I built AsOf to do the reading, so you know what your Sharpe is actually worth.
Your best strategy never beat the market. It beat the calendar. It traded on numbers that didn't exist yet — and called it skill?
The backtest said genius. Real money said otherwise. The difference was the future hiding in the data.
Somewhere in your pipeline, your model saw tomorrow. You just never asked it what it saw.
The numbers were revised. The losers were deleted. The timestamps were wrong. And your Sharpe kept climbing. That's not a strategy. That's a crime scene with good lighting.
You didn't find alpha, but found a leak and named it alpha.
Every backtest is innocent until you check the clock. Most of them aren't.
The model knew the answers before the test started. You just didn't know the model knew.
One leaked data point. That's all it takes. A worthless strategy looks like a genius, a team burns a quarter chasing it, and nobody ever checks the clock.
Let us help you.
I’m 21. I don’t want to test one strategy at a time.
Run them all at once. Nothing sees past its T.
The bigger vision is as-of compute: being able to ask any data, any model, any question as of any moment in history, fast, at scale, with proof.
Finance can’t run on machines that cheat by accident.
In 1900, Louis Bachelier modeled the stock market with math. Nobody believed him for 50 years.
In the 1980s, quants turned math into the most profitable trades on earth.
In 2026, AI is making trading decisions, and almost nobody can prove what it knew at the time it decided.
That’s the next frontier. We’re building it.
Your AI trading backtest isn't predicting the market.
It's remembering it.
LLMs were trained on data from after your backtest dates. Lock every tool you want: the model already read the ending. 🧵
What's the sneakiest look-ahead leak you've ever caught in your own pipeline? The one hiding in a timestamp, a revised number, or a model that remembered too much.