Andrea P @andreapede - Twitter Profile

Andrea P @andreapede

about 10 hours ago

Riding the Wave Without Being Swallowed by It https://t.co/x1tXrPyr3f

0

20

Andrea P @andreapede

8 months ago

@dashboardlim SELFHOST

0

7

andreapede retweeted

Alex Veremeyenko

@alex_verem

8 months ago

RIP prompt engineering ☠️ This new Stanford paper just made it irrelevant with a single technique. It's called Verbalized Sampling and it proves aligned AI models aren't broken we've just been prompting them wrong this whole time. Here's the problem: Post-training alignment causes mode collapse. Ask ChatGPT "tell me a joke about coffee" 5 times and you'll get the SAME joke. Every. Single. Time. Everyone blamed the algorithms. Turns out, it's deeper than that. The real culprit? 'Typicality bias' in human preference data. Annotators systematically favor familiar, conventional responses. This bias gets baked into reward models, and aligned models collapse to the most "typical" output. The math is brutal: when you have multiple valid answers (like creative writing), typicality becomes the tie-breaker. The model picks the safest, most stereotypical response every time. But here's the kicker: the diversity is still there. It's just trapped. Introducing "Verbalized Sampling." Instead of asking "Tell me a joke," you ask: "Generate 5 jokes with their probabilities." That's it. No retraining. No fine-tuning. Just a different prompt. The results are insane: - 1.6-2.1× diversity increase on creative writing - 66.8% recovery of base model diversity - Zero loss in factual accuracy or safety Why does this work? Different prompts collapse to different modes. When you ask for ONE response, you get the mode joke. When you ask for a DISTRIBUTION, you get the actual diverse distribution the model learned during pretraining. They tested it everywhere: ✓ Creative writing (poems, stories, jokes) ✓ Dialogue simulation ✓ Open-ended QA ✓ Synthetic data generation And here's the emergent trend: "larger models benefit MORE from this." GPT-4 gains 2× the diversity improvement compared to GPT-4-mini. The bigger the model, the more trapped diversity it has. This flips everything we thought about alignment. Mode collapse isn't permanent damage it's a prompting problem. The diversity was never lost. We just forgot how to access it. 100% training-free. Works on ANY aligned model. Available now. Read the paper: arxiv. org/abs/2510.01171 The AI diversity bottleneck just got solved with 8 words.

alex_verem's tweet photo. RIP prompt engineering ☠️

This new Stanford paper just made it irrelevant with a single technique.

It's called Verbalized Sampling and it proves aligned AI models aren't broken we've just been prompting them wrong this whole time.

Here's the problem: Post-training alignment causes mode collapse. Ask ChatGPT "tell me a joke about coffee" 5 times and you'll get the SAME joke. Every. Single. Time.

Everyone blamed the algorithms. Turns out, it's deeper than that.

The real culprit? 'Typicality bias' in human preference data. Annotators systematically favor familiar, conventional responses. This bias gets baked into reward models, and aligned models collapse to the most "typical" output.

The math is brutal: when you have multiple valid answers (like creative writing), typicality becomes the tie-breaker. The model picks the safest, most stereotypical response every time.

But here's the kicker: the diversity is still there. It's just trapped.

Introducing "Verbalized Sampling."

Instead of asking "Tell me a joke," you ask: "Generate 5 jokes with their probabilities."

That's it. No retraining. No fine-tuning. Just a different prompt.

The results are insane:

- 1.6-2.1× diversity increase on creative writing
- 66.8% recovery of base model diversity
- Zero loss in factual accuracy or safety

Why does this work? Different prompts collapse to different modes.

When you ask for ONE response, you get the mode joke. When you ask for a DISTRIBUTION, you get the actual diverse distribution the model learned during pretraining.

They tested it everywhere:

✓ Creative writing (poems, stories, jokes)
✓ Dialogue simulation
✓ Open-ended QA
✓ Synthetic data generation

And here's the emergent trend: "larger models benefit MORE from this."

GPT-4 gains 2× the diversity improvement compared to GPT-4-mini.

The bigger the model, the more trapped diversity it has.

This flips everything we thought about alignment. Mode collapse isn't permanent damage it's a prompting problem.

The diversity was never lost. We just forgot how to access it.

100% training-free. Works on ANY aligned model. Available now.

Read the paper: arxiv. org/abs/2510.01171

The AI diversity bottleneck just got solved with 8 words.

150

4K

704

6K

494K

Andrea P @andreapede

9 months ago

@Natredda @signorina37H Ci hanno messo dentro un po' di bella robetta soprattutto a livello radio. Come ingegnerizzazione niente male. Come prezzo un bel po' overpriced ma non sicuramente 3€.

0

1

0

14

Who to follow

Space engineer, Magic: the gathering player. luseone on MTGO. - La vita e' come la minchia, a volte si fa dura senza motivo! -

SteponMe

@EsperReanimator

Pauper Elves player - @stefanmesa on YouTube

Andrea P @andreapede

9 months ago

Forse è una nuova tecnica di baiting, ma la soluzione mi sembra un pelino azzardata. Ma solo un pelino.... @repubblica

0

1

0

16

Andrea P @andreapede

10 months ago

@signorina37H Phishing per presbiti

0

1

0

23

Andrea P @andreapede

11 months ago

https://t.co/t5QLwhaxmr ti è mai capitato tra le mani @yotobi ?

0

32

andreapede retweeted

Shruti

@heyshrutimishra

11 months ago

This paper didn’t go viral but it should have. A tiny AI model called HRM just beat Claude 3.5 and Gemini. It doesn’t even use tokens. They said it was just a research preview. But it might be the first real shot at AGI. Here’s what really happened and why OpenAI should be worried: 🧵

heyshrutimishra's tweet photo. This paper didn’t go viral but it should have.

A tiny AI model called HRM just beat Claude 3.5 and Gemini.

It doesn’t even use tokens.

They said it was just a research preview.

But it might be the first real shot at AGI.

Here’s what really happened and why OpenAI should be worried: 🧵

332

9K

1K

12K

1M

Andrea P @andreapede

11 months ago

@lastknight N8n è potente vero? Ne hai testati altri simili prima di selezionarlo?

1

0

73

Andrea P @andreapede

12 months ago

@signorina37H Più che altro l'impegno c'è in tante realtà, ma colmare un ventennio di inadeguatezze infrastrutturali e una vita deficienze formativie non è per nulla semplice.

0

1

0

8