Brian

Brain enthusiast. tweets == my opinions

[Brian @sadacca]

In a year of getting used to big leaps forward, this still feels huge - LLMs crafting excellent and evolving RL reward functions

[Jim Fan @DrJimFan]

Can GPT-4 teach a robot hand to do pen spinning tricks better than you do?

I'm excited to announce Eureka, an open-ended agent that designs reward functions for robot dexterity at super-human level. It’s like Voyager in the space of a physics simulator API!

Eureka bridges the gap between high-level reasoning (coding) and low-level motor control. It is a “hybrid-gradient architecture”: a black box, inference-only LLM instructs a white box, learnable neural network. The outer loop runs GPT-4 to refine the reward function (gradient-free), while the inner loop runs reinforcement learning to train a robot controller (gradient-based).

We are able to scale up Eureka thanks to IsaacGym, a GPU-accelerated physics simulator that speeds up reality by 1000x. On a benchmark suite of 29 tasks across 10 robots, Eureka rewards outperform expert human-written ones on 83% of the tasks by 52% improvement margin on average. We are surprised that Eureka is able to learn pen spinning tricks, which are very difficult even for CGI artists to animate frame by frame!

Eureka also enables a new form of in-context RLHF, which is able to incorporate a human operator’s feedback in natural language to steer and align the reward functions. It can serve as a powerful co-pilot for robot engineers to design sophisticated motor behaviors.

As usual, we open-source everything! Welcome you all to check out our video gallery and try the codebase today: https://t.co/BHiNmqPoWE
Paper: https://t.co/bdh9TYQtHm
Code: https://t.co/lqKiaM2yYJ

Deep dive with me: 🧵

[Samuel Marks @saprmarks]

Do language models know whether statements are true/false? And if so, what's the best way to "read an LLM's mind"?

In a new paper with @tegmark, we explore how LLMs represent truth. 1/N

[carlos @closji]

Can LMs 🤖 replace programmers 🧑‍💻?
- Not yet!
Our new benchmark, SWE-bench, tests models on solving real issues from GitHub.
Claude 2 & GPT-4 get <5% acc.

🔗 See our leaderboard, paper, code, data: https://t.co/zZ0YpGo1te
🧵

[enrique morán @ennriiq]

all these announcements about wearables make me think of what top tier research like figlab @ CMU has cooked

WorldGaze (2020) is one of them— this is so impressive:

[Elizabeth Laraki @elizlaraki]

In 2006, I was 1 of 4 designers on Google Search.

For 20 years, every search engine has copied Google.

Now ChatGPT, Bard + Claude look like Google's offspring - "better” search engines.

But last week signaled we're on the brink of a design revolution.

ChatGPT unveiled incredible new features.

These could give us the opportunity to completely shift how we interface with AI.

Here's the full story:

–––

When I was a designer on Google Search, all major search engines looked the same – Google, Yahoo, MSN Bing.

Google was the market leader with a heavily optimized UI that supported billions of dollars in ad revenue.

Naturally, it became THE way to show search results.

Its success made it illogical for Google to consider big UI changes.

And any changes they did make were just mirrored by everyone else.

So 20 years later, we’ve only seen incremental changes to search engine UIs.

–––

Today, we have consumer-ready LLMs (Large Language Models) freshly in our hands.

As consumer products, these are in their infancy.

We’re very early in understanding their capabilities and defining how people interact with them.

These are uncharted waters.

And yet ChatGPT, Bard, Claude etc. all chose a text-based input box — just like Google’s search box — as the core interface.

Why?

The input box is simple, versatile, and familiar.

- It’s simple to understand → you type your questions into the box.

- It’s versatile → the box can handle all sorts of questions/queries.

- The paradigm is super familiar → people immediately know how to use it.

Because of this, LLMs have essentially become “a better Google.”

–––

But last week’s ChatGPT announcements thrust open the doors to new possibilities.

ChatGPT is now multi-modal — it can see, hear, and speak.

These are the recent announcements from @OpenAI :

Voice: https://t.co/hAeXxBTH9l

Photos: https://t.co/X3QbLnwT1V

The example of ChatGPT explaining how to lower a bike seat was incredible.

But, it could be so much better!

The video showed you'll have to post multiple new photos to keep adding new information and to progress the conversation.

It was still a linear conversation centered around the text box.

But what if we rethought the interface to center around the image?

What if ChatGPT supported both images AND voice simultaneously?

Could we end up with a more immersive experience?

–––

How else could interacting with LLMs mimic IRL conversations?

Could we (or the AI) pinch to zoom or rotate the image?

Could we interact in real time with video?

What new possibilities open up with context being preserved over time?

–––

There is so much energy and excitement around what AI can do.

But we are limiting the potential by assuming the conversation box is the best interface.

Right now, designers have the chance to create truly novel interactions and bust through the 20+ year old search UI paradigm.

The ideas above are just to illustrate some potential options.

But they are also intended to spark a flame.

Now is the opportunity to be creative and explore divergent UIs.

What are the craziest, coolest, most creative UI ideas we can unleash?

LFG 🚀

[Andrej Karpathy @karpathy]

An OS that boots to a baby Llama 2
https://t.co/yB0TMD0rXm
Standalone, Binary Portable, Bootable

I expected that my "Llama 2 inference code in a single .c file" would go places, but this really stretches the imagination :) And why not, do we really need all this stuff?

[Joshua Batson @thebasepoint]

In writing this paper, there were countless features we thought might be bugs. After careful inspection, ~all of them revealed surprising and subtle model properties.

To me this capacity for surprise is the true test of a new technique.

This thread is about my favorite finding.

[Jo Kristian Bergum @jobergum]

Made it to the orange site, I agree with this take in RAG context

[Wes Gurnee @wesg52]

Do language models have an internal world model? A sense of time? At multiple spatiotemporal scales?

In a new paper with @tegmark we provide evidence that they do by finding a literal map of the world inside the activations of Llama-2!

[Sergey Levine @svlevine]

We've updated the DDPO website with some new results for training diffusion models with RL! Our aesthetic bunny is now much more... aesthetic.

Latest here: https://t.co/5Mui7Wb8pB
Includes code, LoRA training for low memory, pretrained models, etc. Some highlights 👇

[TimDarcet @TimDarcet]

Vision transformers need registers!
Or at least, it seems they 𝘸𝘢𝘯𝘵 some…
ViTs have artifacts in attention maps. It’s due to the model using these patches as “registers”.
Just add new tokens (“[reg]”):
- no artifacts
- interpretable attention maps 🦖
- improved performances!

[Melanie Mitchell @MelMitchell1]

Short but incisive piece by @mcxfrank on why children are so much more efficient at learning than LLMs.

[Ida Momennejad @criticalneuro]

Delighted to share our #neurips2023 paper w @grockious @hmd_palangi et al
Evaluating Cognitive Maps & Planning in LLMs with CogEval

We test planning in 8 LLMs.
Failures like hallucinating invalid paths/falling in loops don't support emergent planning.
1/n
https://t.co/x4AdQyzekw

[Borriss @_Borriss_]

Less than 31 hours since OpenAI started dropping the ChatGPT vision feature on pro users...

People are scratching their heads in disbelief.

10 wild examples:

[Ethan Mollick @emollick]

@ResObscura @fakehistoryhunt My lord!

[Jessica Flack @C4COMPUTATION]

John Conway's A Survey of Life Forms—a taxonomy of Conway's discoveries from the Game of Life + sent in 1970 to Martin Gardner. The image is discussed in Ananyo Bhattacharya's (@Ananyo) new book, The Man from the Future: The Visionary Life of John von Neumann. 1/n

[Michael Bendersky @bemikelive]

Our paper "Out-of-Domain Semantics to the Rescue! Zero-Shot Hybrid Retrieval Models" (by @tao_chen @_Mingyang_Zhang Jing Lu @bemikelive @marc_najork; To appear in ECIR, 2022) is now on arXiv: https://t.co/fV3g7cByiW

[Yury Selivanov @1st1]

WOW.

Bloomberg finally opensourced memray—a new versatile memory profile for Python. Can't way to use it.

Thread 👇

[CT Hsu @caffeinatedcee]

I love this figure and have used it in two of my classes now. I hate to admit that before this paper came along I could never keep cortical cell types straight in my head. https://t.co/jGxMSQO6JI

[Iva Brunec @IvaBrunec]

Excited to share our work w/ @MelissaNantais @suttonjenn @epstein_lab @NoraNewcombe on how people explore new environments! We analyzed people’s free exploration trajectories to see if they predict cognitive map accuracy. A 🧵: https://t.co/jX4N98DXLd