Joseph Joestar

Spécialiste de rien mais curieux de tout. 🇵🇰

[Joseph Joestar @UsHsn78]

J’aime être seul. Ça me permet d’avoir le contrôle sur ce que je fais. Par conséquent, pour me gagner, votre présence doit être meilleure que ma solitude. Vous n’êtes pas en concurrence avec une autre personne mais avec mes zones de confort.

[Kun Chen @kunchenguid]

if you've been using latest frontier LLMs, it's almost certain that you would have noticed by now the newer models have become worse to talk to

they're more robotic, they speak jargons, they spits out verbose text, and do stuff you didn't ask for

how did that happen? well, i'm not the person who trained those models so i can't speak for certain, but i've known enough evidence that gives me a well-educated guess, and i thought it's interesting to share as a crash course of modern LLM training pipelines

so here we go

let's wind back to 2020. GPT-2 and GPT-3 already came out and were widely available, but they could only predict one token at a time - that's what LLMs are at their core

token prediction was offered via API, but there was nothing you could "talk to". so while it generated a lot of excitement in the academic field due to the emergent intelligence, it didn't have any wide adoption

in 2022, ChatGPT changed all that. the research work that led to ChatGPT was a model initially named "InstructGPT". it took GPT-3 as the intelligent base, and used reinforcement learning with human feedback (RLHF) to teach the models how to "chat"

the core idea of RLHF is that you ask the model to generate a few responses, and then let real humans pick which one they like. do this over and over again, and you get a model that knows how to talk

worth noting even as early as InstructGPT, research found that making the model more pleasant to talk to will reduce their pure academic capabilities. this was called "alignment tax", which is an interesting thing we'll come back to in a bit

there were various techniques done to minimize the reliance on humans, but ultimately the reward is modeled after human preference, making these AI assistants easy to talk to

so remember this: RLHF = training the model to be likable by humans

in 2024, there was an inflection point introduced by claude sonnet 3.5 which was the first model that can kind of autonomously finish coding tasks. it led to the first wave of viable "coding agents"

the way sonnet 3.5 achieved this was by training the model with a harness (now it's called an agent) that has bash and file editing tools, throw the agent into a virtual machine, give it a task, and let it try to complete it. these tasks all have a machine-verifiable outcome predefined, mostly via test cases, that can validate whether the model really finished the task or not

then you let the model do billions and billions of attempts in such virtual environments, and some of them would succeed by chance. you keep the successful agent sessions, and use reinforcement learning to teach the model to do that more, and boom - you get a coding agent

that is called reinforcement learning with verifiable rewards (RLVR). if you look closely, you'll see that in this RLVR process, the final text response from the model doesn't matter AT ALL, as long as the code written by the agent could pass the test. it could talk like a jerk and it would still be rewarded

so remember this: RLVR = training the model to be accepted by machines

late 2024 and early 2025, we saw o1 and deepseek R1 came out as the first wave of "reasoning models". this article is getting long so i'm not diving into reasoning models now, but just know that reasoning models also relied heavily on RLVR to scale the training process - let the model think before taking action, and if the thinking led to a machine verifiable outcome, reward the thinking trace and teach the model to think like that more often

the biggest difference between RLVR and RLHF is that RLVR is more scalable. human feedback is expensive to get, especially in domains where only an expert can have a valid opinion on which result is good

with RLHF, if we let the model generate 100 responses, then a human has to review all 100 responses to pick which is good

with RLVR, the human (or sometimes an AI) would define a task and verifier only once, and the model can generate a million responses - the machine verifier will pick which responses are good in an automated way

so as a result, RLVR is becoming more and more dominant in newer models' training pipeline

if you put all these things together:
- RLHF = training the model to be likable by humans
- RLVR = training the model to be accepted by machines
- RLVR is more scalable
- "alignment tax" says "likable by humans" makes the model do worse on verifiable tasks

now you see why the newer models are becoming less and less likable?

this is not just a "frontier labs screwed up their model training" problem - this is a war between machines and humanity, and humanity is losing

we chased after benchmarks, when none of the benchmarks measure whether humans actually enjoy working with the model

we use machines to decide which AI response is better because that's easier and cheaper, when we have no way of making sure those machines actually represent what we humans want

we let AI go dark in a virtual environment on its own and complete predefined tasks at all costs, when in reality we often cannot define a verifiable outcome upfront, and need AI to work with us along the way

i don't have a good solution to this, but i want to call for awareness that we're starting to witness a failure in aligning super intelligence right in front of our eyes

this war between machines vs humanity is one we really can't afford to lose

[Marcos Hernanz @MarcosHernanz]

After doing ~60B tokens, this is my full AGENTS.md

[Min Choi @minchoi]

This Claude Code prompt is literally $300/hr senior engineer🤯

[Joseph Joestar @UsHsn78]

@olfactoriumonx Purée j’ai raté

[ASH @ashecomm]

31% des acheteurs français demandent à une IA avant d'acheter en ligne.

On l'a bien compris et on a fait 20 600 € de CA grâce à ChatGPT ces 30 derniers jours.

Voici comment faire pareil ↓

[Joseph Joestar @UsHsn78]

@OpenDesignHQ Can’t install it

[Ludonauta @LudensLudonauta]

I was studying MCPs and this comment really blew my mind.

[Boris Cherny @bcherny]

@rauchg @tobi This is pretty how I optimize code these days. “Hey fable, use a dynamic workflow to get p95 time down under 300ms. Dont stop till you’re done, use a profiler”

[Divad | Build in public @DivadEcom]

https://t.co/mHHhftDmeT

[Pierre ✺ @pgllmt]

Je suis senior software engineer.

Pendant 30 jours, je vais documenter ici comment j’utilise l’IA sur de vrais problèmes de développement : architecture, debugging, tests, code review et agents.

Pas de démo magique ni de promesse « 10x ».

Je partagerai des méthodes reproductibles, des diffs commentés et surtout les erreurs rencontrées.

Premier sujet : ma checklist avant de valider un diff généré par IA.

[Nouveau-Français en quête d'idéal @0giovanedante]

Suitsupply : faut faire au moins 1m95

COS : le polyester est passé à 95€/la pièce chez eux

Uniqlo : est-ce qu'on pourrait innover sur les couleurs svp ?

Les deux derniers : les coutures c'est sérieux ?

[Swи @3svn_]

La skyview est tellement DINGUE je m’en remet pas

[Arkunir @Arkunir]

Il manque seulement 84.000 signatures sur la pétition contre la loi de présomption de légitime défense chez les forces de l’ordre pour qu’un nouveau débat ai lieu à l'Assemblée Nationale.

C’est la dernière ligne droite : https://t.co/6NzBrMIjh5

[Kylian Mbappé @KMbappe]

Madame Celeste Amarilla,
Vous êtes une femme méprisable et indigne de sa fonction.
Vous ne représentez pas le Paraguay, ce pays qui a transpiré la passion et l’honneur tout au long de la compétition. Par votre inconscience et votre racisme décomplexé, le monde entier a déjà oublié le parcours et l’effort historique que vos joueurs ont réalisés durant cette coupe du monde pour laisser place à une dame incompétente donnant la pire image possible de son pays.
Je ne laisserai jamais aux gens comme elle, la liberté de laisser propager leur haine et leur racisme à travers le monde.

[ابن خليل @ibnkhleel]

All my sins can be forgiven just for wearing clothes?🤔

The Prophet ﷺ said:

“Whoever puts on a garment and says,

الحمد لله الذي كساني هذا، ورزقنيه من غير حول مني ولا قوة

‘All praise is for Allah who clothed me with this and provided it for me without any power or strength from myself,’

will have their past sins forgiven.”

Transliteration 👇
Alhamdu lillāhil-ladhī kasānī hādhā, wa razaqanīhi min ghayri ḥawlin minnī wa lā quwwah.

Classified as sahih (authentic) by Imām al-Albānī in Sahih Abī Dāwūd and Sahih at-Tirmidhī.

[Joseph Joestar @UsHsn78]

@Parfumeur_IDF Oh le fou

[Ania @asuna_nia]

Mon père fait des choses pour moi comme ça et 10000 fois plus et j’ai 29 ans. Ça en fait de lui le meilleur papa du monde, et moi j’ai une vie plus légère dans un monde déjà difficile. C’est NORMAL. Le contraire voudrait dire que ton enfant c’est ton ennemi clairement.

[JardinDesSensParfums @Mussaafir_01]

🎁 Jeu concours spécial Vanille Planifolia 🎁

Pour participer :
✅ Follow nous
🔄 RT ce post
📝 Écrit en commentaire ton parfum préféré

1 parfum et une huile inspiré du fameux Vanille planifolia de Guerlain ❤️

2 gagnants 🤝

Tirage au sort vendredi soir 😎

[a. @alxsqmr]

@UsHsn78 Bien sûr que si, en plus c’est pour une cause importante! C’était juste une blague avec @Hasnihass9 car on était ensemble quand je l’ai acheté. 🙂 Je te mets le lien. https://t.co/tG7jSeOG43

[Joseph Joestar @UsHsn78]

@ledenicheur69 Allez c’est bon!!