Jeremy Howard

🇦🇺 Co-founder: @AnswerDotAI/@FastDotAI ;
Prev: Professor@UQ; @kaggle founding president; founder @fastmail/@enlitic/…

[Jeremy Howard @jeremyphoward]

@jahskillen @badlogicgames Not a daft question! I haven't found any LLMs particularly good at editing ASTs. Also you really need a CST for stuff like comments - which gets a bit more complex. Also plenty of files can't be represented and edited as ASTs (e.g markdown - or at least, not easily)

[Keno Fischer @KenoFischer]

Obvious in retrospect, but I didn't really anticipate:
Fable: Performing Final Review of <Awesome Feature>
Also Fable: I appear to have introduced a critical security vulnerability.
<This model's safeguards flagged this message.>
Opus 4.8: Doesn't look like anything to me.

[Sara Dragutinovic @sara_drag]

Is Muon as good as they say? We looked beyond training speed and found a hidden cost: Muon loses the simplicity bias of older optimizers like gradient descent — and this matters for generalization.

[Yifan Wu @yifannnwu]

Introducing SWE-Together: a multi-turn benchmark built from real user–agent coding sessions.

Coding agents are often benchmarked like exam-takers: given the full spec up front, then graded on the final code. But real coding help is a conversation — users clarify goals, add constraints, and correct course along the way.

SWE-Together turns real coding work into a reproducible, verifiable benchmark: 109 repo-level tasks curated from 11,260 recorded sessions, replayed with a reactive LLM user simulator that preserves the original user’s intent.

We evaluate agents as collaborators, not just patch generators: final pass rate and how many user interventions were needed to get there.

In this evaluation snapshot, claude-opus-4.8 currently leads among the 7 agents we tested — achieving the highest pass rate while requiring the fewest user interventions.

📄 Paper: https://t.co/Zp5BSPpLTJ
💻 Code: https://t.co/NPgxCMLdHi
🌐 Website: https://t.co/BK50zRGReE

[Jeremy Howard @jeremyphoward]

@jm_alexia Wow that's awesome news! :D

[Hanchi Sun @ ACL @sun_hanchi]

https://t.co/gPwut02Ilj

People are missing out on how big a deal Longcat 2.0 by Meituan (aka "Chinese Doordash") is.

Near frontier performance, trained on 50k Chinese domestic accelerators! The first ever to achieve this!

[Jeremy Howard @jeremyphoward]

@simulx4 @jietang Confidently incorrect.

[Jeremy Howard @jeremyphoward]

@Pps831 @jietang Hahahaha.

No.

[Yuchen Jin @Yuchenj_UW]

GLM-5.2 is the open-source Claude moment.

The demand we’re seeing at Databricks is astonishing. The world is going to see massive adoption of oss LLMs.

Also, more companies will shift toward post-training their own models on top of oss models and owning the weights.

[Jeremy Howard @jeremyphoward]

@jsnover As with all models, you need to provide the current date in the system prompt, give it web search and url-reading tools, and tell it how and when to use those tools.

[Matthew Zirwas, MD @MattZirwas]

The take that frontier models aren't ready for use in medicine is dead wrong.

I've been using Opus 4.x in my clinical workflow every day for 6 months.

I'm a deep sub-sub-specialist in dermatology - the cases other derms refer out.

It's better than me, in my exact niche, after 25 years of reading and seeing patients. Not even close.

All it takes is a system prompt, some tool access, and a few references in the project files.

So when a Nature Medicine paper concludes frontier models are "not ready" for medicine, I read the methods.

They tested naked models. No system prompt. No tools. No references. Single-shot multiple choice.

Nobody would ever use them that way.

It's like taking somebody who just finished the first two years of med school, aced Step 1 of the USMLE, giving them zero access to reference materials and asking them to take the final board exam for someone who's had 6 additional years of clinical experience.

Then when they fail, pretending it's evidence that humans aren't ready to practice medicine.

That'd be obviously wrong and this take on frontier models in medicine is just as wrong.

[Jeremy Howard @jeremyphoward]

@williawa No. API.

[Jeremy Howard @jeremyphoward]

Did something happen to Opus 4.8 today? First time I've seen this happen (and I use it daily): it almost totally stopped thinking, even at highest reasoning setting. And answered terribly.

I repeated same prompts with Opus 4.6, and it worked great.

[Jeremy Howard @jeremyphoward]

@MLStreetTalk Yes I kinda agree - except for today. (I also thought it was quite good on launch day though.)

[Jeremy Howard @jeremyphoward]

@renegadesilicon Yeah I know, I use it every day -- but they still take just as *long* to complete the thinking process. Today it was returning nearly instantly every time.

[Albert Gu @_albertgu]

Transformers are better at copying, while RNNs are better at modeling "meaning-bearing words—the nouns, verbs, & adjectives that say what a sentence is about"

[Jeremy Howard @jeremyphoward]

@chrisalbon Also I don't even use zsh. Bash FTW.

Also also I have a .aliases file for aliases.

[Jeremy Howard @jeremyphoward]

@chrisalbon Nope mine is totes different :p