saul

VP engineering @anteriorai | | ex Apple Cloud | NYC

[saul @saulhoward]

https://t.co/fsp7YyT0S6

[saul @saulhoward]

Travelling New York -> Tokyo -> Bangkok is going from one extreme to the other and settling in a happy medium.

[saul @saulhoward]

I think it's completely fair to charge citizens and tourists different prices.

[saul @saulhoward]

"Not enough appetite for gambling" is not what's wrong with Britain. What's wrong with Britain is that 50k is a life-changing amount for most.

[Nate Silver @NateSilver538]

This is what's wrong with Britain tbh

[Brett Harrison @BrettHarrison]

In 2015 I wrote an interview question for Jane Street based on a real task I had given to new engineers. The problem was ostensibly simple but few candidates did well. The question was retired a few years ago so here it is, with related thoughts on LLMs building trading systems.

US securities exchanges have Designated Market Maker programs to ensure liquidity for their ETFs. The programs require that MMs maintain two-sided quotes 100% of the trading day, where each quote must be within 8% of the best-bid/offer (BBO). Satisfying these program requirements confers benefits to the MM, including improved capital treatment and the ability to short ETFs without locates.

The interview problem set up a hypothetical in which, in addition to the 8%/100% requirement above, the programs also require the MMs to be within 4% of the BBO 90% of the time, and within 1% of the BBO 70% of the time.

The problem asked the candidate to write code that could read a file containing a time series of market BBOs and another of the firm’s BBOs and produce a report measuring compliance with the MM requirements. Bonus points were awarded if the majority of the code could be reused for real-time reporting in addition to a file-based historical analysis.

Unlike what many expect of a Jane Street interview, this problem didn’t involve probabilistic reasoning or advanced computer science. It required a careful understanding of the data domain, including how to handle time boundaries, the optionality of market data prices, ordering inconsistencies, and online record keeping.

The pass rate was very low. We assumed that most candidates either hadn’t encountered enough real-world software in college/grad school, or else prepared too narrowly for their interview with questions about algorithms and computational complexity.

My benchmark for testing coding abilities of frontier LLMs is to ask the models to build infrastructure for a market making system from scratch. Even the most recent models, including Fable and GPT-5.6, do not produce systems with design patterns that are commonplace at HFTs. Even though the models’ outputs function in a narrow sense, the code architectures would not be able to form the foundations of even a modest production trading operation.

A production trading system is an institutional artifact, a representation of how a firm thinks about markets, failure, ownership, observability, deployment, and risk. Few of the inputs are ever written down in one place, some aren’t written down at all. The knowledge is distributed across code review norms, incident histories, operational procedures, and the accumulated instincts of experienced engineers over decades.

This helps explain why public LLMs can’t create effective trading infrastructure from a short specification, and why the interview candidates had particular difficulty with the question. A private LLM fine-tuned on an HFT firm’s repositories, incident reports, design documents, and review history might become highly capable within that firm’s engineering environment, but that’s far from how general-purpose LLMs are trained and operate today.

[Spooky JL @spooky_JL]

@TheStalwart But they don’t have an AI strategy

[saul @saulhoward]

Open source has to be weird if it’s to survive. Repellent in some way. Some adaptation that stunts growth, allows it to remain hidden.

[saul @saulhoward]

Job interviews aren’t a chance to show me what you can get Claude to do. I can already talk to Claude. What do I need you for?

[saul @saulhoward]

Incredible how many people still think showing LLM-generated work is enough. It’s only ever been about the marginal value you add. LLMs raised the baseline. They made your job so much harder.

[saul @saulhoward]

Soccer’s great because humans are really bad at it. Americans like to see continuous excellence. Soccer gives you fleeting glimpses of what we might be.

[saul @saulhoward]

If it wasn’t permanent siege mentality at any frontier lab I’d be worried tbh.

[saul @saulhoward]

The best thing about football is how bad humans are at it. It’s a sport really at the limits of physical ability.

[saul @saulhoward]

Sport and country music. America dominates with every cultural export, except those two.

[Per Bylund @PerBylund]

What causes #poverty? Nothing. It's the original state, the default and starting point. The real question is, What causes #prosperity?

[saul @saulhoward]

@napenforcer https://t.co/Qo8rf4iENw

[saul @saulhoward]

So all the frontend JS-framework-of-the-week guys pivoted into AI dev frameworks.

[saul @saulhoward]

@SOPHONTSIMP Having brothers helps.

[saul @saulhoward]

Automation doesn't advance by mimicking prior art. A new medium demands a new architectural approach. You'll know LLMs have changed software when the software itself is changed. Generative UI and ephemeral interfaces seem like the strongest direction.

[saul @saulhoward]

@BenjDicken Prose strives for ambiguity, code for unambiguity.

[saul @saulhoward]

@norpadon Prose is ambiguous by design, Code unambiguous.

[Justin Schroeder @jpschroeder]

Fable 5 is so ridiculously good that it boggles my mind how Claude desktop and Claude code were so buggy.

They’ve had this since April.

[saul @saulhoward]

@bnj