voicebot.ai

Giving Voice to a Revolution. The most widely read generative AI and conversational AI news and analysis with over 15M page views. Get your industry data here.

[Matt Griswold @griswold]

@sama Found him, just as he's leaving. 😢https://t.co/ikZp6dLQ53

[voicebot.ai @voicebotai]

Listen to a song about writing #GenerativeAI stories produced using music startup @Udiomusic, who just emerged out of stealth with $10M from @a16z (plus artists @iamwill and @Common)
https://t.co/PO2c4FfCTz

[Bret Kinsella (Read Synthedia) @bretkinsella]

https://t.co/RomDq8sdpX raising $3B on $18B valuation, according to @WSJ. @amazon may be an ideal investor for @elonmusk. We break it all down in Synthedia.
https://t.co/4ruPRwstBd

[voicebot.ai @voicebotai]

.@OpenAI Hints at #GenerativeAI #Alexa and #Siri Rival With New ‘Voice Engine’ Trademark Filing
https://t.co/3mDHA2wA6g

[Alexa Developers @alexadevs]

Join us on March 7 at 10AM PST for a conversation with Bret Kinsella, Founder and CEO of @voicebotai, and Emerson Sklar, Chief Evangelist of Amazon Alexa.

Register now: https://t.co/4ZbURhzbSq

[voicebot.ai @voicebotai]

#GoogleBard (and Duet and #GoogleAssistant Mobile App) No More – #GoogleGemini Now The Star of @Google #GenerativeAI Show https://t.co/f9saFWfqIG

[voicebot.ai @voicebotai]

.@CerenceInc and @Nvidia Debut Automotive #LargeLanguageModel CaLLM to Power New #GenerativeAI Features
https://t.co/qGaLDgdamX

[Bret Kinsella (Read Synthedia) @bretkinsella]

I'm hosting the Generative AI News Rundown in 45 minutes - Gemini, Mistral, Phi, Grok, Meta, Snapchat and More. Join the discussion in the live stream or watch afterward on YouTube (link in the 🧵).

#generativeai @EricHSchwartz @voicebotai https://t.co/r4aOiO9nkR

[Eric Hal Schwartz @EricHSchwartz]

#GenerativeAI is in the stars (Gemini), in the wind (Mistral), and even on Mars (Grok). But even if it’s all Greek (Phi) to you, come watch @bretkinsella and I talk about the bits that really matter and what comes next. Then read all about it on @VoicebotAI.

[voicebot.ai @voicebotai]

#VoiceAI Startup @Tenyx_AI Solves #LLM ‘Catastrophic Forgetting’ During Fine-Tuning
https://t.co/t86SUmjfVT
#GenerativeAI

[voicebot.ai @voicebotai]

.@AnswerRocket Launches Custom Enterprise #GenerativeAI Copilot Platform
https://t.co/bF94Ao2xJf

[Bret Kinsella (Read Synthedia) @bretkinsella]

🧵Top Generative AI News of the Past Week.

- Google Gemini
- Mistral
- EU AI Act
- McDonald's
- Assembly AI

#GenerativeAI @voicebotai

[Bret Kinsella (Read Synthedia) @bretkinsella]

Generative AI News 12/1/23 - ChatGPT B-day, SoapBox Labs Acquired, Amazon and Open AI Go for Q, Inflection AI & More https://t.co/y4B3j2FSIc

@voicebotai @EricHSchwartz #generativeAI #OpenAI #chatgpt

[Jim Fan @DrJimFan]

In my decade spent on AI, I've never seen an algorithm that so many people fantasize about. Just from a name, no paper, no stats, no product. So let's reverse engineer the Q* fantasy. VERY LONG READ:

To understand the powerful marriage between Search and Learning, we need to go back to 2016 and revisit AlphaGo, a glorious moment in the AI history.
It's got 4 key ingredients:

1. Policy NN (Learning): responsible for selecting good moves. It estimates the probability of each move leading to a win.

2. Value NN (Learning): evaluates the board and predicts the winner from any given legal position in Go.

3. MCTS (Search): stands for "Monte Carlo Tree Search". It simulates many possible sequences of moves from the current position using the policy NN, and then aggregates the results of these simulations to decide on the most promising move. This is the "slow thinking" component that contrasts with the fast token sampling of LLMs.

4. A groundtruth signal to drive the whole system. In Go, it's as simple as the binary label "who wins", which is decided by an established set of game rules. You can think of it as a source of energy that *sustains* the learning progress.

How do the components above work together?

AlphaGo does self-play, i.e. playing against its own older checkpoints. As self-play continues, both Policy NN and Value NN are improved iteratively: as the policy gets better at selecting moves, the value NN obtains better data to learn from, and in turn it provides better feedback to the policy. A stronger policy also helps MCTS explore better strategies.

That completes an ingenious "perpetual motion machine". In this way, AlphaGo was able to bootstrap its own capabilities and beat the human world champion, Lee Sedol, 4-1 in 2016. An AI can never become super-human just by imitating human data alone.

-----
Now let's talk about Q*. What are the corresponding 4 components?

1. Policy NN: this will be OAI's most powerful internal GPT, responsible for actually implementing the thought traces that solve a math problem.

2. Value NN: another GPT that scores how likely each intermediate reasoning step is correct.
OAI published a paper in May 2023 called "Let's Verify Step by Step", coauthored by big names like @ilyasut @johnschulman2 @janleike: https://t.co/iAvXNjjhcK
It's much lesser known than DALL-E or Whipser, but gives us quite a lot of hints.

This paper proposes "Process-supervised Reward Models", or PRMs, that gives feedback for each step in the chain-of-thought. In contrast, "Outcome-supervised reward models", or ORMs, only judge the entire output at the end.

ORMs are the original reward model formulation for RLHF, but it's too coarse-grained to properly judge the sub-parts of a long response. In other words, ORMs are not great for credit assignment. In RL literature, we call ORMs "sparse reward" (only given once at the end), and PRMs "dense reward" that smoothly shapes the LLM to our desired behavior.

3. Search: unlike AlphaGo's discrete states and actions, LLMs operate on a much more sophisticated space of "all reasonable strings". So we need new search procedures.

Expanding on Chain of Thought (CoT), the research community has developed a few nonlinear CoTs:
- Tree of Thought: literally combining CoT and tree search: https://t.co/KM1P2ZJrjG @ShunyuYao12
- Graph of Thought: yeah you guessed it already. Turn the tree into a graph and Voilà! You get an even more sophisticated search operator: https://t.co/5ncT5tuTOY

4. Groundtruth signal: a few possibilities:
(a) Each math problem comes with a known answer. OAI may have collected a huge corpus from existing math exams or competitions.
(b) The ORM itself can be used as a groundtruth signal, but then it could be exploited and "loses energy" to sustain learning.
(c) A formal verification system, such as Lean Theorem Prover, can turn math into a coding problem and provide compiler feedbacks: https://t.co/vpOBOI2FR5

And just like AlphaGo, the Policy LLM and Value LLM can improve each other iteratively, as well as learn from human expert annotations whenever available. A better Policy LLM will help the Tree of Thought Search explore better strategies, which in turn collect better data for the next round.

@demishassabis said a while back that DeepMind Gemini will use "AlphaGo-style algorithms" to boost reasoning. Even if Q* is not what we think, Google will certainly catch up with their own. If I can think of the above, they surely can.

Note that what I described is just about reasoning. Nothing says Q* will be more creative in writing poetry, telling jokes @grok, or role playing. Improving creativity is a fundamentally human thing, so I believe natural data will still outperform synthetic ones.

I welcome any thoughts or feedback!!

[Bret Kinsella (Read Synthedia) @bretkinsella]

What is OpenAI's Q*? How Aligned Incentives are Fueling a Questionable Narrative. https://t.co/nIihH1LbYl

#GenerativeAI @voicebotai #Qstar #openai

[voicebot.ai @voicebotai]

After days of #GenerativeAI high drama, @SamA has returned as @OpenAI CEO with an (almost) all-new board.
https://t.co/A3m0QuuCNl

[Bret Kinsella (Read Synthedia) @bretkinsella]

Anthropic's Claude 2.1 LLM Has a 200K Context Window, API Tools and Poses a New Challenge to OpenAI

We have a full breakdown.

#generativeAI @AnthropicAI #LLM #LLMs #openai
https://t.co/8tEyyGQlHs

[Bret Kinsella (Read Synthedia) @bretkinsella]

Congrats to the entire @soapboxlabs team, and of course, to @NiamhBushnell! Good stuff.

[Bret Kinsella (Read Synthedia) @bretkinsella]

There has been a lot of talk about AI safety and a few mentions of conflicts of interest related to Altman. What about the clear conflict of interest of one OpenAI board member?

@TheRealAdamG @adamdangelo @Scobleizer @jowyang @mattturck

https://t.co/yEyLfHOEPm

[voicebot.ai @voicebotai]

Momentum.

[Sam Altman @sama]

we have more unity and commitment and focus than ever before.

we are all going to work together some way or other, and i’m so excited.

one team, one mission.