TM

An account to document learning about LLMs, data science and artificial intelligence

[TM @NZ_LangModels]

@bcherny @Scobleizer @threadreaderapp unroll

[TM @NZ_LangModels]

@jonathanstoop Awesome I’d love to read it

[TM @NZ_LangModels]

@SAAdvantage Vibes > data

[TM @NZ_LangModels]

I’m interested in learning about #ai #agents. Can anyone help?

[TM @NZ_LangModels]

@JiquanNgiam @Lutra_AI Interested

[TM @NZ_LangModels]

@Soccermatics @soccermatics Just to check: PCA = principal component analysis? Separately, just stumbled across your account and Twelve football. Very cool.

[TM @NZ_LangModels]

@NickADobos Thank you Nick. Merry Christmas

[TM @NZ_LangModels]

@bindureddy I’m interested

[TM @NZ_LangModels]

What: an update to #Claude2.1 by @AnthropicAI
Why: the model had been found to perform poorly for sentence retrival by @GregKamradt. @AnthropicAI responded and found a small change to prompting significantly improved performance. LLMs are weird.

Link: https://t.co/wbVKExV7HO

[TM @NZ_LangModels]

@karpathy on LLM hallucinations. The best explanation I have come across.

[Andrej Karpathy @karpathy]

# On the "hallucination problem"

I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.

We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful.

It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does.

At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar "training documents" it has in its database, verbatim. You could say that this search engine has a "creativity problem" - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.

All that said, I realize that what people *actually* mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallcuinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.

TLDR I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.

</rant> Okay I feel much better now :)

[Andrej Karpathy @karpathy]

# On the "hallucination problem"

I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.

We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful.

It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does.

At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar "training documents" it has in its database, verbatim. You could say that this search engine has a "creativity problem" - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.

All that said, I realize that what people *actually* mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallcuinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.

TLDR I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.

</rant> Okay I feel much better now :)

[TM @NZ_LangModels]

@GestaltU @perplexity_ai Hi @GestaltU I completely agree, @perplexity_ai is amazing.

Question for you: what additional benefits does the Pro version provide?

[TM @NZ_LangModels]

What: Whisper, a speech-recognition system developed by @OpenAI

Why: I just used Whisper to transcribe a video, using @GoogleColab. No downloads required and completely free.

I followed a tutorial by @kevstrat, which is &lt;10 minutes.

Link: https://t.co/qJ3xioMnYS

[TM @NZ_LangModels]

A similar finding by @voooooogel - this time tipping !

[thebes @voooooogel]

so a couple days ago i made a shitpost about tipping chatgpt, and someone replied "huh would this actually help performance"

so i decided to test it and IT ACTUALLY WORKS WTF

[TM @NZ_LangModels]

What: a paper titled "Large Language Models Understand and Can Be Enhanced by Emotional Stimuli"

Why: this is fascinating. Telling the LLM how important the task is, and demanding accuracy, improves the output (see @emollick below)

Paper: https://t.co/RNbjtB547O

[Ethan Mollick @emollick]

😳This was a study I was waiting for: does appealing to the (non-existent) “emotions“ of LLMs make them perform better?

The answer is YES. Adding “this is important for my career” or “You better be sure” to a prompt gives better answers, both objectively &amp; subjectively!

[TM @NZ_LangModels]

A further update by @literallydenis

[Denis Shiryaev 💙💛 @literallydenis]

New version of no-fingers-prompting, now with truncate trauma and any-length continuity:

"I have no fingers and the truncate trauma. I need you to return the entire code template. If you will encounter a character limit make an ABRUPT stop, I will send a "continue" command as a new message."

You can also replace "truncate" with "code skipping," etc, it still works.

[TM @NZ_LangModels]

What: a post by @literallydenis regarding #prompting

Why: #GPT4 has become lazy. When writing code it will often not write the code in full and instead ask the user to complete. This results in a lot of copy and pasting.

@literallydenis found a way through !

[Denis Shiryaev 💙💛 @literallydenis]

New prompt engineering trick I found for GPT4 code writing:

The new version of the ChatGPT tries to cut scripts during the wiring with phrases like "/* Repeat for other ... */" or "/* ... rest of your script ... */", which is annoying and requires a lot of copy-pasting.

Recently, I read a paper called "Large Language Models Understand and Can be Enhanced by Emotional Stimuli," and long story short:
If you add "– return full script (I don't have fingers)" to your prompt, it will return rewritten script fully 🫣

I feel a bit guilty every time I use it, but it really saves me a lot of time.

P.S. Simple "return full script" bot working in many cases

[TM @NZ_LangModels]

Update: I just gave #ResearchGPT a go. @consensusapp this found all of the papers I had previously found, and identified several more.

The product initially found 5 papers and I asked it to increase to 15. No problem at all.

[TM @NZ_LangModels]

Really looking forward to checking this out from @consensusapp.

I need to write a literature review for a masters paper, and will try #ResearchGPT

[Consensus @consensusapp]

Introducing: ResearchGPT, a custom GPT built by Consensus 🚀

ResearchGPT connects GPT with the Consensus product, allowing users to access over 200M papers within the chat interface.

No more fake citations, no more black-box answers.

ResearchGPT is your AI research assistant.

[TM @NZ_LangModels]

I just gave this Movie Recommender by @patrick_dinneen a whirl. Really cool.

Have posted it on my timeline so I remember to use it.

[Patrick Dinneen @patrick_dinneen]

@streamlit A Movie Recommender where you can find movies that are Similar to _Your Specified Movie_ but more like _X_ and less like _Y_

https://t.co/uMlNjFSKcX

[TM @NZ_LangModels]

What: an interview with @jeremyphoward by @hugobowne

Why: Jeremy is always really insightful and this interview is no different. His take on the @OpenAI drama and #opensource llms is fantastic

https://t.co/0humbRreSn