Leslie Smith ([email protected])

Researching life-long, deep learning on computers and for myself. I work for the US Navy and care about our democracy. I like my sense of humor.

[Leslie Smith ([email protected]) @lnsmith613]

This is great: "The Sound Of Music Nuns Have Their Say..." https://t.co/yXAiJnKlbv

[Leslie Smith ([email protected]) @lnsmith613]

Vote like your life depends on it

[Leslie Smith ([email protected]) @lnsmith613]

@JaneotN Based on recent events, it is looking likely that Trump will proclaim that nobody knew that Kamala was female and all of a sudden she turned female only for political reasons. Sigh - he is such a complete fool and total joke, only capable of racist slurs.

[Bindu Reddy @bindureddy]

The complexity in building AI agents is actually making them work in real world environments!

Only 10% is about the LLMs and its reasoning ability

The rest is the painful heavy lifting in terms of code, data and memory.

Evaluation and ongoing monitoring is an added layer of complexity

It’s like a beautiful orchestra when it comes together

[Leslie Smith ([email protected]) @lnsmith613]

Well said.

[Andrew Ng @AndrewYNg]

In AI, the ratio of attention on hypothetical, future, forms of harm to actual, current, realized forms of harm seems out of whack.

Many of the hypothetical forms of harm, like AI "taking over", are based on highly questionable hypotheses about what technology that does not currently exist might do.

Every field should examine both future and current problems. But is there any other engineering discipline where this much attention is on hypothetical problems rather than actual problems?

[Scientific American @sciam]

New training method helps AI generalize like people do https://t.co/bnFvRtjmoP

[Louis Anslow @LouisAnslow]

What AI doomers won't tell you: MODERNA uses AI to create its mRNA vaccines.

BEFORE AI it produced 30 mRNAs per month
AFTER AI it produced 1000 mRNAs per month
https://t.co/SNfCgv6591

[Jeremy Howard @jeremyphoward]

I was chatting on Discord with an anon account ,who had just come up with a really smart new approach to model merging and had some encouraging results to show.

But then he had to go.

Turns out he's in 10th grade and was in trouble because he wasn't focussing on English class.

[Andrew Ng @AndrewYNg]

Do Large Language Models really "understand" the world, or just give the appearance of understanding? Evidence (e.g., Othello-GPT) shows LLMs build models of how the world works, which makes me comfortable saying they do understand. More in The Batch: https://t.co/e0JGU2wUbf

[Frank Hutter @FrankRHutter]

Do you like #LLMs and #AutoML? Then you might like the Auto-Survey competition at #AutoML2023 (https://t.co/8kcRR7VU9x), which just launched officially. It's a very novel competition where LLM agents write survey papers and LLM agents judge them (final judges are human). IMO doing well in this challenge might give important insights into AutoML for alignment! Thanks to Isabelle Guyon and her team for setting up this cool challenge.

[PoliticsGirl @IAmPoliticsGirl]

Worth the time to listen to the wise words of @GovPritzker

[Leslie Smith ([email protected]) @lnsmith613]

This is a thoughtful comment regarding the issue of AI safety and any existential threat from any technology.

[Jeremy Howard @jeremyphoward]

The most powerful and compelling writing I've seen on AI safety was a random HN comment from some anon poster.

knodi123, whoever you are, thank you...

[Sebastien Bubeck @SebastienBubeck]

New LLM in town:

***phi-1 achieves 51% on HumanEval w. only 1.3B parameters & 7B tokens training dataset***

Any other >50% HumanEval model is >1000x bigger (e.g., WizardCoder from last week is 10x in model size and 100x in dataset size).

How?

***Textbooks Are All You Need***

[Leslie Smith ([email protected]) @lnsmith613]

Every DL researcher should listen to this video and spend time thinking about these questions.

[Andrew Ng @AndrewYNg]

Had an insightful conversation with @geoffreyhinton about AI and catastrophic risks. Two thoughts we want to share:
(i) It's important that AI scientists reach consensus on risks-similar to climate scientists, who have rough consensus on climate change-to shape good policy.
(ii) Do AI models understand the world? We think they do. If we list out and develop a shared view on key technical questions like this, it will help move us toward consensus on risks.

I learned a lot speaking with Geoff. Let’s all of us in AI keep having conversations to learn from each other!

[Republican Accountability @AccountableGOP]

Bill Barr on the classified documents investigation:

"This is not a case of the DOJ conducting a witch hunt...This would have gone nowhere had the president just returned the documents, but he jerked them around for a year and a half...There is no excuse for what he did here."

[Leslie Smith ([email protected]) @lnsmith613]

Useful. Also, note https://t.co/HsWDfDJBNg and https://t.co/QI9MeQNh8q.

[Sanyam Bhutani @bhutanisanyam1]

Another epic collection of prompting notes 🧑‍💻

This is a crispy read covering history, strategies and guidelines around LLM powered systems.

Note, it’s missing ToT 🌲 prompting, I’ll write more about it soon!

https://t.co/PQhK0y7q7Y

[Leslie Smith ([email protected]) @lnsmith613]

A lot of folks try but this one gets a gold start for having a truly clever name for their paper:
StarCoder: may the source be with you!
https://t.co/D2H7QXpdlc

[Leslie Smith ([email protected]) @lnsmith613]

IMHO, LLMs will/should disrupt many societal tasks and early adopters who get on this early will have an advantage over everyone else. Reminds me of the song title: "It's the End of the World as We Know It (And I feel Fine)" - R.E.M.

[Leslie Smith ([email protected]) @lnsmith613]

Personally, I am studying prompt engineering and gaining experience with fine-tuning open source LLMs.

[Andrej Karpathy @karpathy]

It's a great question. I roughly think of finetuning as analogous to expertise in people:
- Describe a task in words ~= zero-shot prompting
- Give examples of solving task ~= few-shot prompting
- Allow person to practice task ~= finetuning

With this analogy in mind, it's awesome that we have models that can reach high levels of accuracy across many tasks with prompting alone, but I also expect that reaching top tier performance will include finetuning, especially in applications with concrete well-defined tasks where it is possible to collect a lot of data and "practice" on it.

Rough picture to have in mind maybe. Small models are incapable of in-context learning and will benefit very little from prompt engineering, but depending on the difficulty of the task it may be possible to still finetune them into decent experts.
Big caveat all of this is still very new.