Walid Rahman

ML Engineer & Scientist working on LLMs

[Walid Rahman @walidr314]

Made a fun time machine with the new openai GPT-Live-1 api. Talk to people from history with your own voice: https://t.co/zNBgZRSnr0

#openai #gptlive

[Earth Science Week @earthsciweek]

🎉 Theme reveal! 🌍✨ The 2025 #EarthScienceWeek theme is: "Energy Resources for Our Future" 💡🌱
Earth science empowers informed #energy choices for a sustainable future and better quality of life for generations! ⚡

https://t.co/zfOKzZURwI

[Walid Rahman @walidr314]

@netcapgirl In this gold rush, some devs took the advice of selling shovels. But now its becoming a shovel market rather than a gold market. Meanwhile, I'm digging with my bare hands (copy pasting from claude artifacts) because I am so tired of the shovel sellers.

[Vianai Systems, Inc. @VianaiSystems]

#GenAI is everywhere, but the true challenge for enterprises is making it work in their specific environments. The hila platform provides the tools needed to bring reliable, useful, domain-specific genAI solutions to life.

Download our brand new white paper today: https://t.co/vGgJ2idP8t
#EnterpriseAI

[Vianai Systems, Inc. @VianaiSystems]

Announcing the global launch of @boomi FinTalk powered by Vianai, enabling CFOs & finance users to ask ad-hoc, natural-language questions and get real-time, actionable insights for expense management, variance analysis and revenue strategies.

https://t.co/CzvGTywnIL #BoomiWorld

[Walid Rahman @walidr314]

@yacineMTB BERT to get extracted answers, textual entailment to grade the extractions, then maybe some model like GPT2 or BART to summarize the extracted answers. It was very easy to set up, answers were OK, but the buy-in for NLP based applications wasn't as great.

[Vianai Systems, Inc. @VianaiSystems]

Ask any question of systems of record using Conversational Finance powered by hila Enterprise, to get real-time insights as charts, graphs or text. With our anti-hallucination technology, responses from #LLMs are reliable, trustworthy, and correct.

https://t.co/gizduIrms1

[Yann LeCun @ylecun]

Lots of confusion about what a world model is. Here is my definition:

Given:
- an observation x(t)
- a previous estimate of the state of the world s(t)
- an action proposal a(t)
- a latent variable proposal z(t)

A world model computes:
- representation: h(t) = Enc(x(t))
- prediction: s(t+1) = Pred( h(t), s(t), z(t), a(t) )
Where
- Enc() is an encoder (a trainable deterministic function, e.g. a neural net)
- Pred() is a hidden state predictor (also a trainable deterministic function).
- the latent variable z(t) represents the unknown information that would allow us to predict exactly what happens. It must be sampled from a distribution or or varied over a set. It parameterizes the set (or distribution) of plausible predictions.

The trick is to train the entire thing from observation triplets (x(t),a(t),x(t+1)) while preventing the Encoder from collapsing to a trivial solution on which it ignores the input.

Auto-regressive generative models (such as LLMs) are a simplified special case in which
1. the Encoder is the identity function: h(t) = x(t),
2. the state is a window of past inputs
3. there is no action variable a(t)
4. x(t) is discrete
5. the Predictor computes a distribution over outcomes for x(t+1) and uses the latent z(t) to select one value from that distribution.
The equations reduce to:
s(t) = [x(t),x(t-1),...x(t-k)]
x(t+1) = Pred( s(t), z(t), a(t) )
There is no collapse issue in that case.

[Walid Rahman @walidr314]

We have some really cool things coming. Stay tuned.

[Vianai Systems, Inc. @VianaiSystems]

Dive into the complexities of #LLM hallucinations and the fine line between creativity and fabrication. Discover cutting-edge techniques for mitigating #hallucinations and how hila Enterprise is paving the way for reliable, factual responses.

Read more: https://t.co/KSXshXRi2X

[Vianai Systems, Inc. @VianaiSystems]

🌐🤖 Unlock the power of domain-specific #genAI applications with hila Enterprise.
Here are five key things to know about our platform 👇

https://t.co/33yTDOTwk9

[Vianai Systems, Inc. @VianaiSystems]

"The most thrilling aspect of my work lies in the challenge of integrating human intuition with the capabilities of artificial intelligence." Learn more about @walidr314 and his passion for video games and reading about the history of science & technology. https://t.co/FV1uohtIng

[Walid Rahman @walidr314]

This is the way.

[Andrej Karpathy @karpathy]

e/ia - Intelligence Amplification
- Does not seek to build superintelligent God entity that replaces humans.
- Builds “bicycle for the mind” tools that empower and extend the information processing capabilities of humans.
- Of all humans, not a top percentile.
- Faithful to computer pioneers Ashby, Licklider, Bush, Engelbart, ...

[elvis @omarsar0]

“Use LLMs as a reasoning engine to process information, rather than using it as a source of memorized information”

Great tip by Andrew Ng.

[Harrison Kinsley @Sentdex]

Working on a project getting LLMs to talk amongst themselves to come to a conclusion with the hopes of maybe achieving something substantial, I am left mostly realizing how LLMs by nature and/or RLHF have no real foundation of belief at all.

It's all just hallucination.

When you happen to get a model to appear smart, it's not smart, it's just appearing that way and in a minute it'll probably appear very stupid again.

[Walid Rahman @walidr314]

@Sentdex Projections of projections of the world interacting with projections of projections of the world. We need actual models interacting with actual models (models here being systems that plan, reason, critique, backtrack). We’ll get there one day…

[Andrej Karpathy @karpathy]

# On the "hallucination problem"

I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.

We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful.

It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does.

At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar "training documents" it has in its database, verbatim. You could say that this search engine has a "creativity problem" - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.

All that said, I realize that what people *actually* mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallcuinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.

TLDR I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.

</rant> Okay I feel much better now :)

[Walid Rahman @walidr314]

They grow up so fast. #OpenAI

[Walid Rahman @walidr314]

@tunguz And then they complain when you merge your fancy new way of reversing 2D linked lists with missing edges to prod. Why do we need user login or the database scripts anyway? They weren’t part of the interview! 🙄