Antoine Didisheim

Assistant Professor @unimelb | AI & Finance (ML, DL, LLMs) | Papers:

[Antoine Didisheim @AntoineDidiML]

We describe these results in more detail in our paper (now accepted at Economics Letters): https://t.co/ym6KLwEU5X

[Antoine Didisheim @AntoineDidiML]

Do LLMs remember stock returns, and does it matter for finance?
It does, and how much they remember is predictable: it depends on model size, frequency, and the type of asset. 👇

[Antoine Didisheim @AntoineDidiML]

AI’s Predictable Memory: What It Means for Finance?

Suppose you ask an LLM to read yesterday’s news and forecast tomorrow’s return. Can you trust the forecast? How reliable would a backtest of such a strategy be?

Here’s the catch: if the news predates the LLM’s knowledge cutoff, you don’t know whether the model is actually analyzing the news, or just “remembering” the return from training. This look-ahead problem is built into all standard LLMs and poses a challenge for applications using historical data. But how severe is it across different settings?

To investigate, we ran a simple experiment: without any news context, we asked OpenAI’s LLMs to recall asset returns directly. A few fascinating patterns emerged:
1.   Frequency matters. Memory is weak for daily returns (correlation ~5%), but rises to ~20% at monthly frequency and ~43% at yearly frequency for GPT-4.1.
2.   Model size matters. Smaller models recall far less. At monthly frequency, GPT-4.1 hits ~20%, but GPT-4.1-mini manages only ~4%, while 4.1-Nano is nearly flat at 1%.
3.   Indexes vs. stocks. LLMs virtually perfectly recall U.S. index returns (99% correlation monthly) and show strong memory of other indexes (50–90%), but are weaker on individual stocks.
4.   Recency matters. Returns from recent years are remembered much more accurately than those from the early 2000s — consistent with the training sample’s composition.

This “pure look-ahead bias” doesn’t capture the full risk: once you add context, recall can improve further, potentially worsening contamination. But it provides an apples-to-apples way of seeing where leakage risk is high or low.