Philippe Martin

Head of SRE at @leaf_space .
Space engineer (ground stations · ExoMars · Galileo).

I benchmark AI models for DevSecOps and developers.

[Philippe Martin @PhyByte]

Fable 5.1 is in DevBench.

On the developer track it jumps from a collapse to second place.

1. Opus 4.8 → 94.4
2. Fable 5.1 → 92.5
3. Grok 4.5 low → 92.5

Fable 5 was unusable on this suite. Bug fixing, refactoring and review were close to zero because it refused most of those cases.

5.1 actually does the work. It scores 98 on bug fixing, 99 on refactoring and stays strong across Python, TypeScript, Go and Rust.

The catch is the price.

Fable 5.1 cost $34 on this run.
Opus did it for $11.
Grok 4.5 low matched the 92.5 score for $3.

So 5.1 is now a real coding model again. It is just a very expensive one. For a planner or a hard review pass, that can make sense. For the rest of an agent stack, a cheaper model still does the job.

Results:
https://t.co/A7daNL7rBs

[Philippe Martin @PhyByte]

BioWare is closing. After 30 years. And early development on the next Mass Effect is ending.

This one hurts. Baldur’s Gate, Neverwinter Nights, Knights of the Old Republic, Dragon Age, Mass Effect... I spent hundreds of hours in these games since I was a kid. They were more than games. They were places where your choices actually mattered, where characters felt real, and where the story stayed with you long after the credits.

Thank you for the adventures. End of an era.
#bioware

[Philippe Martin @PhyByte]

Force à toi et à ta petite.

Je pense que malheureusement la police et la justice n’ont plus la capacité de gérer tout ça. Et même s’ils étaient allés au bout, il n’aurait probablement rien eu de bien grave… car on les traite souvent comme des malades plutôt que comme des criminels.

Le mieux à faire, selon moi, c’est de communiquer ces informations aux parents d’élèves de l’école où il travaille. C’est probablement la chose la plus utile pour éviter que ça se reproduise avec d’autres enfants.

[Philippe Martin @PhyByte]

Last night Tesla finally unveiled the next-gen Roadster at SpaceX’s McGregor site. Nine years after the 2017 prototype, it showed up ready to rewrite what a car can do.

It drove up a vertical ramp and stuck upside down thanks to a high-power suction fan system. Then SpaceX cold-gas thrusters (compressed air, up to 10 nozzles) took over: the car flipped, hovered, and landed. Next it launched over the entire Tesla lineup (Model 3, Y, Cybertruck, Semi, Cybercab) with a mid-air kickflip/barrel roll before touching down.

Performance numbers from the demo: 0-60 mph in 0.81 seconds. Three versions announced: Performance (road-ready, suction system), Thrust (adds rear thrusters, sub-1s 0-60), and Rocket (full thruster package for air maneuvers, invite-only, not for public roads). Top speed claimed over 230 mph, with Musk aiming higher. Production targeted for 2028.

Musk called it a candidate for the greatest product demo of all time and described it as Tesla’s last manually driven car, fun dialed to 11. The thrusters and suction system turn the Roadster into something closer to a low-altitude rocket with wheels than a conventional supercar.

Flying cars aren’t science fiction anymore. They’re just still invite-only.

[Tesla @Tesla]

Roadster event livestream https://t.co/G0jU4mWfid

[Philippe Martin @PhyByte]

@RokBoogas I never tried it myself..... not rven sure how much it cost.

[Philippe Martin @PhyByte]

I love Chuck and his work, but this is the first time I’ve seen Elon run an Ad on a post.

This one is labeled “Ad” and is just boosting a positive Grok Bot review. Why spend on paid reach for this specific post when organic distribution from Elon is already huge?

[Elon Musk @elonmusk]

Grok @Bot rocks

[Philippe Martin @PhyByte]

Un an de ministériat résumé : un Observatoire de la souveraineté, une commission d’experts, 600 ambassadeurs pour �� sensibiliser » et un Pacte avec la filière.

Vous brandissez le chiffre de 40 % des TPE-PME qui « utilisent l’IA ». En réalité, c’est surtout de la génération de texte ou d’images et des chatbots. Seules 19 % des entreprises paient pour un outil, l’automatisation de tâches reste marginale (autour de 11 %), et dans les services publics on en est encore à déployer un assistant conversationnel pour les agents, sans aucun gain mesuré d’efficacité ni de réduction des coûts.

Pendant ce temps, les données de millions de Français ont fuité à la DGFiP et à l’ANTS. Rien de concret sur la sécurisation des systèmes d’information de l’État, l’optimisation réelle des services publics ou la baisse des dépenses.

Du cocorico et des comités c'est vraiment ce que la France peut faire de mieux?.....
....

[Anne Le Hénanff @ALehenanff]

🇫🇷 @MistralAI , je suis Anne Le Hénanff. Il y a un an, je devenais ministre déléguée chargée de l'Intelligence Artificielle et du Numérique.

Si l'on s'en tient aux faits, qu'avons-nous fait avancer pour la France ? Réponse courte, que je puisse afficher sur mes réseaux sociaux.

🧵 Pour tout comprendre, déroulez le fil 👇

[Philippe Martin @PhyByte]

Il est remarquable de constater que le ministère de la Jeunesse contribue également au financement des retraites.... une illustration singulière incroyable de la gestion de l'argent publique en France.

[Maxime Sbaihi @MxSba]

C'est HISTORIQUE: Bercy publie pour la 1ère fois le poids réel des pensions dans le budget de chaque ministère.

Attention, ça pique: 32% du budget de l’Éducation nationale et 30% du budget de l'Intérieur part en réalité dans... les retraites

Tous les ministères sont concernés⤵️

[Philippe Martin @PhyByte]

Bien d’avoir des usines et des batteries en Europe. Mais on dépend toujours à ~90 % de la Chine pour les aimants de terres rares, et le recyclage n’est pas encore à l’échelle.

Les prix européens tiennent surtout grâce aux aides et aux taxes. Les autres sont plus compétitifs sur le coût pur.

La voiture de demain sera autonome. Tant qu’on ne peut pas la faire rouler correctement ici, nos constructeurs n’investiront pas vraiment.

[Emmanuel Macron @EmmanuelMacron]

Il y a 10 ans, on nous disait qu’il était impossible de produire des batteries électriques en Europe et que l’automobile française ne ferait pas le poids face à la concurrence mondiale.

Impossible n’est pas Français !

Aujourd’hui nous avons 4 gigafactories en France, nous produisons des centaines de milliers de véhicules électriques et nos R4 et R5 sortent de nos usines avec des batteries fabriquées chez nous.

C’est très concret, c’est bon pour nos emplois, notre économie, et cohérent avec nos objectifs climatiques.

Au Mondial de l’Auto ce matin, j’ai réaffirmé une ambition claire : notre industrie automobile française et européenne innove, accélère et se réindustrialise. Nous devons continuer et aller plus loin.

Trois priorités :

1. Protéger notre industrie.

L’accord engagé entre l’Union européenne et la Chine marque une étape pour lutter contre la concurrence déloyale, notamment sur les véhicules hybrides. Nous voulons une concurrence juste, qui protège nos entreprises et nos emplois.

2. Accélérer l’électrification et la réindustrialisation.

Plus de 40 % des véhicules vendus en septembre étaient électriques. Avec France 2030, nous avons investi dans l’innovation, relancé des filières industrielles et démontré qu’on pouvait créer des emplois tout en décarbonant notre économie. Nous continuerons, notamment avec le véhicule autonome.

3. Renforcer notre souveraineté européenne.

Avec nos partenaires allemands, nous avançons vers un accord pour concilier décarbonation, compétitivité et protection de notre industrie. Avec une priorité : produire en Europe nos véhicules et leurs composants.

Nous avons montré que nous pouvions y arriver. Soyons fiers de nos industries et de nos savoir-faire.

Continuons à décarboner sans désindustrialiser, innover sans dépendre et défendre notre capacité à produire en Europe.

Croyons en nous, en notre industrie et en notre avenir !

[Philippe Martin @PhyByte]

C’est bien d’avoir des gigafactories et des batteries assemblées en Europe. Mais on importe toujours l’essentiel de nos terres rares et de nos aimants de Chine (près de 90 % des aimants européens). Les filières de recyclage existent sur le papier, mais les volumes restent faibles et plusieurs projets ont déjà été suspendus. Le vrai décollage est plutôt pour 2030-2035.

Les prix français ne tiennent qu’avec les aides et les protections. Même après les droits antidumping, les modèles chinois restent souvent nettement moins chers. Il faut revoir les coûts de production, pas seulement ajouter des barrières.

Et la voiture de demain sera intelligente. Au Mondial, seuls Tesla et Xpeng font essayer de la conduite automatisée avancée. Tant que les règles européennes restent trop strictes pour faire rouler et apprendre en conditions réelles, nos industriels n’ont aucun intérêt économique à investir massivement.

[Philippe Martin @PhyByte]

Zuckerberg’s table shows Muse Wonder Full leading on long-horizon coding (SWE-Marathon, DeepSWE) and memory, with a clear edge on the legal agent benchmark. It is not ahead everywhere. Opus 5.5 still leads on Terminal-Bench, OSWorld and from-scratch programming.

On price, the API version of Muse Wonder sits at $4 / $20 per million tokens, the same as Opus 5.5 and well below Fable 5.1 and GPT-6 Astra. Wonder Full itself is subscription-only.

The interesting part is not “best model.” It is that Meta now has a frontier model that is competitive on sustained agent work at the same list price as the current Claude flagship.

[Mark Zuckerberg @finkd]

Introducing Muse Wonder, our new family of frontier models with state of the art performance for personal and professional agentic work.

[Philippe Martin @PhyByte]

Does anyone have numbers on AI usage in public services vs private companies?

Karpathy’s funnel is real. Most people still see free-tier chatbots. A small slice of professionals watches agents finish multi-week work.

The next gap will be who can actually run that locally. For a lot of companies, local models will be the practical leap in the coming years. You keep the traces inside, you control the cost, and you stop treating every sensitive system as something that has to go to a third-party API.

Public services are still mostly on the other side of that line. Would be useful to see the actual difference in usage.

[Andrej Karpathy @karpathy]

Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel.

- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.

The funnel is driven by a combination of factors:
1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.

So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.

[Philippe Martin @PhyByte]

Agree on the cheap software problem. You wrote this morning that generating the code got cheap, but owning it did not. Someone still has to patch the hole and know why the function exists.

Same pattern in the freezer controllers and the French admin leaks: remote access that looked fine until it was not.

A European effort that actually uses agents to scan live systems for weak remote controls and silent failures would be more useful than another regulation. The tools exist. The incentive to run them on real infrastructure is what is missing.

[Philippe Martin @PhyByte]

Last night someone remotely told supermarket freezers in France to heat up to 50 °C and left them there for hours. Intermarché, U and some Leclerc stores had to throw out the food.

Same quiet pattern as the DGFiP and other French admin leaks earlier this year: access that looked legitimate, detection that came late, real damage once it was noticed.

Most companies still treat AI mainly as a way to write code or answer questions faster. The more urgent use is the boring one: watching operational systems that already talk to the outside world, and flagging when something that should never happen starts happening.

That shift is not a strategy choice anymore. It is becoming the difference between catching the problem and cleaning up after it.

[Philippe Martin @PhyByte]

@xAllianceDAO @chessucation @MultiversX 🤣

[Philippe Martin @PhyByte]

Thanks a lot for everything you did for @MultiversX; it was great to be part of @xAllianceDAO and the crazy collabathon with @HeliosStaking.

Those were great moments that I think about from time to time.
There are really amazing people in this community.

I wish you all the best for the challenges ahead.

Take care, man!

[Philippe Martin @PhyByte]

@Michael94288471 @elonmusk Im living in Germany.... and yes a lot of old peoples take their mercedes ( old models) out on sunday ^^ and at 17h to go to Edeka when peoples get out of work 🤣

[Philippe Martin @PhyByte]

The advantage that Grok has in reading X posts without additional cost is huge.

If OpenAI wanted the same, they would have to pay for the API, and it costs a lot.

Today, when most information and news transit through X first, this is something to consider when choosing your models.

[Philippe Martin @PhyByte]

This is a bigger edge than most people realize...

X has basically become the main place where official announcements drop, where tech and research get argued out in real time, and where you can actually feel the temperature of important conversations. It’s like a live sensor for what’s going on.

Grok can just pull from that stream directly. Other models have to pay for API access or work around limits.

That difference compounds. Fresher context makes the reasoning better on a lot of real questions.

[Philippe Martin @PhyByte]

That should win the Eurovision... would be amazing 🤣

[Moritz Kremb @moritzkremb]

I told Claude to make a video about what it's like to be European

One day later...