Olga Grabek

JavaScript dev

[Andrej Karpathy @karpathy]

# On the "hallucination problem"

I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.

We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful.

It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does.

At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar "training documents" it has in its database, verbatim. You could say that this search engine has a "creativity problem" - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.

All that said, I realize that what people *actually* mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallcuinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.

TLDR I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.

</rant> Okay I feel much better now :)

[zeroheight @zeroheight]

📣 #HowWeDocument is back! 📣 We’re here kicking off the third edition of our report all about the #designsystems community. But, that means… we need you.
✍️ Take part in the survey: https://t.co/6aa5GWhJZy

[Ryan Bates @rbates]

I’ve started a dev blog! The first post is RailsCasts Retrospective Part 1: The Fuel. https://t.co/xEQQU67ju4

[Olga Grabek @ograbek]

@British_Airways Your website is down. It's not possible to check-in.

[Olga Grabek @ograbek]

@alyssaxuu I tried to use it. I couldn't find easing functions and UI is broken when selecting an object. However, it looks promising :)

[Olga Grabek @ograbek]

Budowa dróg rowerowych nie ma sensu. Najpierw trzeba zmienić przepisy https://t.co/lYctLDQUmT

[Olga Grabek @ograbek]

"When choosing a new habit many people seem to ask themselves, “What can I do on my best days?”

The trick is to ask, “What can I stick to even on my worst days?”

Start small. Master the art of showing up. Scale up when you have the time, energy, and interest."

–@JamesClear

[Olga Grabek @ograbek]

Going to read Mastering Modular JavaScript! 📕
It covers module thinking, patterns &amp; principles
https://t.co/RyOzCaUZ82

[Olga Grabek @ograbek]

I agree. Mercedes dominates and it made the f1 so boring.

[Olga Grabek @ograbek]

Konta Trumpa zbanowane we wszystkich istotnych serwisach społecznościowych. Amazon/Apple/Google blokuje Parlera. Cenzura na niespotykaną dotąd skalę czy normalność? https://t.co/fC2OThcoGd

[Paweł Sasko @PaweSasko]

Polish politicians in the middle of pandemic denied woman in my country basic human rights - and they expected nobody to stand up.

Since few days gigantic protests are gathering thousands of people, we do not agree to rob us of dignity.

Symbol of #StrajkKobiet is ⚡️

[Olga Grabek @ograbek]

One React mistake that's slowing you down, article by @kentcdodds https://t.co/y3PVos5udf

[Olga Grabek @ograbek]

@_marcba The one that fits the best the project specification.

[Olga Grabek @ograbek]

Unfortunately, my Internet provider, @libertyglobal (AS6830), does NOT implement BGP safely. Check out https://t.co/XVZqjoGaox to see if your ISP implements BGP in a safe way or if it leaves the Internet vulnerable to malicious route hijacks. via @Cloudflare

[Olga Grabek @ograbek]

@wrobel221 Adobe Flex - I fell in love in this technology 😁

[Olga Grabek @ograbek]

@excid3 Gym, motion design, books.

[Olga Grabek @ograbek]

How cool is that? I've just got a t-shirt, stickers and... HANDWRITTEN card from the @Cypress_io team! This is awesome! Thank you @Cypress_io for your daily effort to deliver such great e2e testing tool #cypress_io #e2e

[Olga Grabek @ograbek]

@bendhalpern XHTML, IE7 support for transparent png, css sliding doors technique to have rounded corners buttons, jQuery saved the world, Firebug, Ruby on Rails commercials :)

[Olga Grabek @ograbek]

@szafranek Jak widać to nie przyniesie dodatkowego przypływu elektoratu. Lepiej obiecać ludziom kolejne 500+ :( Kto by się tam Bałtykiem przejmował. Zaczną o tym mówić jak to się zacznie im opłacać 😢

[not just pixels @getifyX]

OMG @Oracle this #Java page is so horribly wrong. How embarrassing for you.