5/ This is a known problem with a name: adaptive data analysis (Dwork et al., Science 2015, "The reusable holdout"). The fix is mostly discipline, not math.
If your platform lets you rerun the out-of-sample window freely, it doesn't have one.
1/ The research mistake that fooled us longest wasn't a bug. It was a habit: going back to the holdout "just one more time."
Every peek turns your out-of-sample data into a backtest. Here's the math, and the rule we use now. 🧵
4/ Our rule now: the backtest can nominate a strategy. Only forward data that didn't exist when it was built can promote it.
Every look at held-out data counts as a trial and goes in the log. The holdout gets opened once, and the answer stands.
@LuxAlgo Missing piece for AI ideas. A backtest can nominate, but only data that didn't exist yet can promote. Log every idea's paper results against its backtest and users get a real out-of-sample check.
@NinjaTrader Stops cap the trade, not always the system. Kaminski & Lo (2014) found they cost you in random-walk markets and only help in trends. Test the stop on your own data before you trust it.
@Quantradin@alphaarchitect Fair. One year is one draw. Build the stress yourself with synthetic stretches where stocks and bonds sell off together, and see if it survives more than one 2022
AI is making trading strategies easier to build, and much easier to fool yourself with.
Here's the problem. If you test one idea and it looks good, that means something. If you test 10,000 variations and keep the best one, it almost always looks good, just by luck.
AI strategy builders can test thousands of versions in an afternoon. But they report the winner as if it were the only thing you tried. Research on hundreds of published factors (Harvey, Liu & Zhu) found that the more ideas get tested, the higher the bar a "discovery" has to clear.
The most important number on any backtest platform is one they never show you: how many tries it took to get there.
@KrisAbdelmessih The test we'd run: do they still offset each other on the worst 5% of days, not just on average? Two premiums can look uncorrelated for years and then move together in a liquidation. Size off the stressed correlation.
@AlpacaHQ@Money20 One thing 24/7 equities will break is most stock backtests. They're built on session closes and overnight gaps. When markets never close, every rule tuned on daily closes needs re-testing.
@quantian1 The whisper number is the interesting part. When expectations get built inside the same small group of traders, a "miss" measures how crowded the positioning was, not how good the quarter was. Positioning problems usually clear in days, not quarters.
"Third time in its history" charts have a hidden filter: we only look at Axon because it survived. Run the same rule on every stock that broke its 200-week, including the ones that got delisted, and the bounce rate looks very different. Two cases is a story. The full set is the signal.
@TrendSpider Fun pattern, but 4 out of 4 is where research starts, not where it ends. The real test is the same rule across hundreds of stocks and timeframes. If it holds then its an edge. If it only works on NVDA...
The "Metaplanet heist" article, checked against the filings
An anonymous article calling Simon Gerovich (@gerovich) and Dylan LeClair (@DylanLeClair) "grifters" is circulating. The article is sourced almost entirely to press summaries, CoinDesk, Cointelegraph, COINOTAG, TipRanks, Cryptopolitan, Investing, and not one primary filing. That shows, and it matters, because several of its central claims are wrong in ways the original documents settle in a paragraph
Point by point
"the roughly $600M in option value that shareholders paid management along the way" and "Simon is charging shareholders 20% every time he issues new equity"
The 20% arithmetic is right, and I will not pretend otherwise. The adjustment clause held the pool at 20% of fully diluted capital, which means the pool grew by one share for every four issued to investors, or 20% of all new shares created counting both
You can watch it happen in the company's own share-count table. Over the third quarter of 2025 issued shares rose by 486,260,000, and the gap between issued and diluted shares widened by 121,565,000 [1]. That is exactly 25% of the new issuance, which is the clause working precisely as written. The international offering of 385,000,000 shares at ¥553, ¥212.9 billion raised, was the bulk of that quarter [2]; its share of the ratchet is 96,250,000 shares. The mechanism was indefensible and the company deleted it for that reason
But here is what the article never checks, and it reverses the story
Bitcoin per share went up 63% over the same period the stock fell 86%
This is not my estimate. It is in the same 18 August notice the article cites for the pool size. BTC per 1,000 fully diluted shares was 0.016151 at the end of June 2025 and 0.026355 at the end of June 2026 [1], a rise of 63.2%. The company puts the whole series in one table, every quarter since the strategy began, and notes that on this basis bitcoin per share has risen roughly 43-fold since June 2024 [1]. Shareholders ended the period owning substantially more bitcoin per share than they started with, not less
The share price fell anyway because two other things moved: bitcoin itself dropped roughly 40%, to the ¥9,523,104 the company marked at 30 June [3], and the premium to net asset value collapsed from roughly 3x to below 1x. Dilution was the one factor working in shareholders' favour. Reading an 86% price decline as evidence of what the option pool did is reading the wrong variable
The ratchet's real cost is measurable and much smaller: the 96,250,000 shares attributable to the September offering were 6.7% of the 1,434,392,925 diluted shares outstanding at that quarter end [1]. Everyone's bitcoin per share is 6.7% lower than it would otherwise have been. That is a genuine grievance. It is not an 86% one
"Dylan LeClair used his influence and affinity with bitcoiners to help sell the Metaplanet dream... He was paid millions in compensation for his efforts"
The article asserts this. It does not evidence it, the author concedes the figure is his own estimate, and he says he tweeted at LeClair and got no reply. Here is what the filings establish and what they genuinely leave open, because the distinction matters when you are naming someone
Start with what is open, since I got this wrong myself at first. The 18 August notice says the rights were originally allotted to seven people, two directors and five employees; that those five employees later departed; and that their units were reallocated with board approval "in their entirety to the other three SAR holders (the current one executive officer and two employees)" [1]. Seven minus five is two, but the notice then lists five current holders. The company's own account does not close, which means at least three of the five present holders came into the plan after December 2022. So the roster is not a closed set, and nobody can point to that notice and prove LeClair is absent from it. The names of the four holders who are not Gerovich have never been published. I cannot exclude him and I am not going to pretend I can
What I can do is check the scale being claimed, and that is where it fails. LeClair is not a director or an auditor, so the officer remuneration table does not cover him. The right one is the employee disclosure in the same annual report: at 31 December 2025 Metaplanet the parent employed 16 people at an average annual salary of ¥16 million [4]. That puts the entire parent-company payroll, every employee including him, at roughly ¥256 million, about $1.7 million. Across the whole consolidated group there were 35 employees [4]
So "paid millions of dollars" is arithmetically impossible if he is on the parent payroll, unless he was taking most of it. Subsidiary salaries are not broken out, so there is a door left open there, and I will say so rather than overstate. What there is no version of is the picture the article paints, of a marketing hire quietly extracting a fortune, sitting inside a company whose entire disclosed officer remuneration for FY2025 was ¥67 million across a board of nine directors and three auditors, with the stock-option column blank in every row [4]
"Eric Trump... was granted stock acquisition rights covering a reported 3.3 million shares"
He did not receive them. He bought them
The 19th Series was resolved on 9 May 2025 and allotted on 26 May to Eric Trump (33,000 units) and David Bailey (3,000 units), at an issue price of ¥255 per right set by an independent valuation from Tokyo Financial Advisors using a Monte Carlo model [6]. Trump paid ¥8,415,000, roughly $58,000 at the time, for rights over 3,300,000 shares, and the notice states the exercise price was set partly in consideration of the fact that he takes no compensation for the advisory seat [6]. You can argue the price was too low, but "granted" is simply the wrong word
And the article places this immediately after its discussion of the 10th Series, as though one explains the other. They are separate instruments issued more than two years apart, to different people, on different terms, at different strike prices. Eric Trump has nothing to do with the 10th Series and no filing connects him to it
"Simon and management held roughly $600M in option value"
The magnitude is in the right area, so I am not going to quibble with it. The pool is 319,464,000 shares at a ¥10 exercise price [1]. At ¥271 that is worth about ¥83.4 billion once you subtract the strike the holders have to pay, roughly $540 million at ¥155 to the dollar. But two things about the number are wrong as the article uses it
It is not one person's. Gerovich holds 276,000 of the 459,000 units, 60.1% of the pool [1]; after the 28 August exercise that is 184,000 unexercised units plus 64,032,000 locked shares. His economic share is about $323M. The other $214M belongs to four people whose names have never been published
The article says "Simon and management" once and then attributes the whole figure to Simon for the rest of the piece, including in the LeBron James and Jensen Huang comparisons. Those comparisons are totally wrong
And it was not "paid" by shareholders in any sense that matches the word. Nothing left the company. It is dilution, it is unrealised, it moves with the share price by tens of millions of dollars a week, and following the August amendment every share arising from it is, in the filing's own words, in principle unsaleable until 17 August 2031 [1][7]. A number that changes when bitcoin moves is not a fee anyone was charged
"they are rare because they are an extreme form of executive compensation"
Paid stock options, are a standard and widespread instrument in Japan. Employees subscribe at an appraised fair value with their own money; the structure is used across the listed market. Gerovich paid ¥54,950,000 for his February 2023 package, declared in his own large-holding report as own funds with no borrowings [8], and the 18 August notice states that all the recipients subscribed at a third-party-appraised fair value with their own funds [1]
What was unusual was the adjustment clause attached to it, not the vehicle. The distinction matters because the article uses "rare options" to imply something furtive was constructed for this purpose, when the actual position is more mundane and more damning: an ordinary Japanese instrument carrying one extraordinary term that nobody re-explained at any subsequent raise
"has lost most investors most of their capital to date, some as much as 80%"
That depends entirely on when you bought, and the company publishes the number that settles it. Metaplanet's own April 2025 filing states the full-year 2024 average share price as ¥1,026, equal to ¥102.6 in current terms after the April 2025 split [5]. Against ¥271 today, anyone who bought at the average price across the whole of 2024 is up about 164%
The 86% decline is equally real, and it is measured from the June 2025 peak of ¥1,930, a figure that appears in the annual report's own high/low share price table [4]. Both facts are true. Only one appears in the article
"while what Madoff did was clearly illegal, the results, in practice, are almost the same"
They are not almost the same. Madoff's investors held claims against fabricated returns backed by nothing
Metaplanet's shareholders hold a company with 43,000 verifiable bitcoin, the third largest holding of any listed company in the world and the largest in Asia, on the company's cited source [3], and, as above, 63% more bitcoin per share than a year ago
A treasury company whose stock falls because bitcoin fell and its premium normalised is not a Ponzi
Reaching for Madoff is the point at which an argument stops being analysis
"Ten days later, on August 28, 2026, Gerovich exercised 92,000 units of the rights, converting them into 64,032,000 shares... So, roughly $600M and huge losses for common shareholders later, Simon is willing to... not do that anymore, but keep the $600M"
This is the sharpest line in the article and it is the one that falls apart fastest, because the sentence immediately before it does the refuting
The author correctly lists what the 18 August amendment did: fixed the pool at 319,464,000 shares, scrapped the 20% adjustment mechanism, added a five-year lock-up through August 2031
Read those three again
Fixing the pool means every share Metaplanet issues from here dilutes the holders alongside everyone else
Scrapping the adjustment removes the only term that made the pool grow
The lock-up attaches five years of illiquidity to shares that previously had none [1]
All three take value away from the people holding the rights, and the article states all three accurately before concluding that the beneficiary got to "keep the $600M"
The exercise runs the same direction. The rights run to 7 February 2033 [4]. Exercising in August 2026 surrendered 6.5 years of remaining option life, converted an option into stock, put ¥640,320,000 into the company (64,032,000 shares at the ¥10 exercise price), and took delivery of shares that in principle cannot be sold until 17 August 2031 [7]
Somebody keeping $600M does not pay the strike on 64 million shares and then hand himself a 5 year prohibition on selling them
And "keep the $600M" misdescribes what the number is. It is not cash and never has been. It is the market value of unexercised rights and locked stock, it moves with the share price, and 40% of it belongs to four people who are not Gerovich [1]. Between the article being written and today it has moved by tens of millions of dollars without anyone doing anything
What he did gain is voting stock, roughly 5% of the register, and that is worth naming rather than glossing over. It is a governance question. It is not a looting one, and it is a strange foundation for a Madoff comparison
Where the article is right, and I will say it plainly, is the first half of that same sentence: it took weeks of shareholder criticism to produce the amendment. It should not have. The pressure worked, and the fact that pressure was needed is the real indictment
What the article gets right, and it is not nothing
The mechanism was wrong. It paid for issuance rather than performance, it had no ceiling, and the people it paid controlled the lever. The company effectively conceded it on 18 August when it deleted the clause, stating in its own notice that the provision "amplifies the dilution borne by existing shareholders" and had made it difficult for investors to ascertain the fully diluted share count [1]
The disclosure was poor, though I want to be precise about how, because the article is not precise at all
The clause was not buried. It was in the December 2022 allotment notice, approved by special resolution at the extraordinary general meeting of 7 February 2023, and set out in every annual securities report since, formula included: adjusted number of shares = new fully diluted shares outstanding × 0.2 [4]. What was never done was explaining, at any individual capital raise, that the raise itself would enlarge the pool. That is the real gap. It is narrower than "hidden"
But an argument this serious deserved primary sources. Every correction above comes from a document that was publicly available while the article was being written. When you accuse named people of running something comparable to a Ponzi, the minimum standard is opening the filings
Sources
[1] TDnet, 18 August 2026, "Notice Regarding Partial Amendment to the Terms of Issuance of the 10th Series Stock Acquisition Rights (Paid Stock Options) (Fixing of the Number of Potential Shares), Execution of Lock-Up Agreements and Policy for Partial Transfer to a Long-Term Officer and Employee Incentive Plan". Issue price ¥18 per unit, exercise price ¥10 per share, approved by the extraordinary general meeting of 7 February 2023, subscribed at third-party-appraised fair value with the recipients' own funds. Originally allotted to seven officers and employees (two directors and five employees); on the departure of those five employees their units were reallocated with board approval "in their entirety to the other three SAR holders (the current one executive officer and two employees)"; 1,000 units exercised April 2026; unexercised balance at 30 June 2026 of 459,000 units held by five people, Gerovich 276,000, two executive officers 141,000, two employees 42,000. Adjustment provision deleted, potential shares fixed at 319,464,000, five-year lock-up. Reason given: the provision "amplifies the dilution borne by existing shareholders". Contains the quarterly table of issued shares, fully diluted shares and BTC per 1,000 fully diluted shares from June 2024 to June 2026, including 826,567,925 diluted and 0.016151 at June 2025, 1,434,392,925 and 0.021489 at September 2025, and 1,631,543,380 and 0.026355 at June 2026. https://t.co/i6YZXhZ7yP
[2] TDnet, 10 September 2025, "Notice Regarding Determination of Issue Price and Other Matters": international offering of 385,000,000 shares at an offer price of ¥553, aggregate ¥212,905,000,000, payment date 16 September 2025.
[3] Metaplanet consolidated interim results for the six months ended 30 June 2026, and Q2 2026 earnings presentation, 13 August 2026: 43,000 BTC held at 30 June 2026, ranking third globally and first in Asia among publicly listed companies (company's cited source: https://t.co/mKHenaYoGU); BTC reference price ¥9,523,104. https://t.co/1nrDRhSfYA
[4] Metaplanet annual securities report for FY2025, filed 26 March 2026 (EDINET S100XTWY). Officer remuneration: ¥57M to two inside directors and ¥10M to three outside officers, ¥67M in total, stock-option column blank in every row, with nine directors and three auditors in office at year end. Employees: 16 at the parent company at 31 December 2025, average annual salary ¥16 million, 35 across the consolidated group. Share price table showing the ¥1,930 high. 10th Series stock acquisition rights table: board resolution 28 December 2022, extraordinary general meeting approval 7 February 2023, exercise period 8 February 2026 to 7 February 2033, exercise price ¥10, and the adjustment formula written out as 調整後株式数=新完全希薄化後発行済株式総数×0.2. https://t.co/mjBfZMlLqw
[5] TDnet, 11 April 2025, "Notice Regarding the Issuance of the 18th Series of Stock Acquisition Rights (Paid Stock Options)": named allocations to nine directors (Gerovich 5,000 units, Abe 5,000, Edwards, Yusko, Kuwajima, Evans, Tsai, Eto and Kincaid 1,500 each) and three auditors 1,500 each, plus 20,750 units to 22 employees of the company and its subsidiaries as a group; exercise price ¥105 against a ¥348 close, set by reference to "the average share price for the full year of 2024 of ¥1,026 (¥102.6 after the share split)".
[6] TDnet, 9 May 2025, "Notice Regarding Issuance of Paid Stock Options (19th Series of New Share Subscription Rights) to External Collaborators and Conclusion of Paid Stock Option Purchase Agreement": Eric Trump 33,000 units, David Bailey 3,000 units, 36,000 units in total over 3,600,000 shares; issue price ¥255 per unit, ¥9,180,000 in total; allotment and payment date 26 May 2025; valuation by Tokyo Financial Advisors by Monte Carlo simulation; exercise price ¥105, set in part in consideration of the allottees taking no compensation for the Strategic Board of Advisors.
[7] TDnet, 31 August 2026, "Notice Regarding the Partial Exercise of the 10th Series of Stock Acquisition Rights": 92,000 units exercised into 64,032,000 shares, exercise date 28 August 2026, shares subject to the lock-up and in principle not saleable or transferable until 17 August 2031; personal holding 15,555,500 before and 79,587,500 after. https://t.co/CRL7uvRNWv
[8] Simon Gerovich initial large-holding report, obligation date 8 February 2023 (EDINET S100VL2I): total position 30,000,000 securities all acquired on 8 February 2023; acquisition funds ¥54,950,000, own funds, no borrowings. https://t.co/anrfXwjNgw