Vu

Generative AI @meta, previously @amazon

[Vu @vu_bui]

@pronounced_kyle @paulg They lie on purpose. They just want engagement.

[Vu @vu_bui]

@andrewchen @sharath_darsha we need this

[Vu @vu_bui]

@samswoora @meatballtimes Are these things not happening if you don’t take that deal?

[Vu @vu_bui]

@ylecun This makes me remember what Rick says to Morty: “Yeah, sure, I mean, if you spend all day shuffling words around, you can make anything sound bad, Morty.”

[Vu @vu_bui]

@dhh What a great view. Looks like somewhere in Chelsea?

[Vu @vu_bui]

@kirawontmiss Which episode?

[Years Progress @YearsProgress]

2024 is 0% complete. #HappyNewYear

[Vu @vu_bui]

@dastaan_hind @AbhayVenkatesh1 @0xkarmatic If he tells the truth, that won’t give him views 😂. So let him cook a little longer.

[Vu @vu_bui]

@villi A few more retweets, it will cure cancer. I believe in X.

[Vu @vu_bui]

@chrishlad Steve Jobs gave a talk at MIT and he kinda talked about people like these business experts. They know a “picture of a banana” but never taste it.
https://t.co/o8relxcfyS

[Vu @vu_bui]

@nsykesdotco @levelsio @damonchen He's not Vietnamese.

[Vu @vu_bui]

@CaneColvin @ZebraCat20 @techbronyc @PaulSkallas Get new friends

[Vu @vu_bui]

@brandonginsberg @volodarik 10 lessons I got from trying to get access to this chat and how it taught me to be resilient when facing insurmountable setbacks

[Vu @vu_bui]

@brandonginsberg @volodarik Too real 😂

[Vu @vu_bui]

@vkhosla Can you share the other modern public ideas?

[Andrej Karpathy @karpathy]

# On the "hallucination problem"

I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.

We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful.

It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does.

At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar "training documents" it has in its database, verbatim. You could say that this search engine has a "creativity problem" - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.

All that said, I realize that what people *actually* mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallcuinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.

TLDR I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.

</rant> Okay I feel much better now :)

[Vu @vu_bui]

@karpathy Thanks for elucidating the issue!

[Vu @vu_bui]

@LinusEkenstam Very impressive

[Vu @vu_bui]

@ronaldsvilcins @thekitze I think we need Pro Max for that.

[Vu @vu_bui]

@Merocle I wonder what’s the thought process behind implementing this.