Sean Oesch

Cybersecurity Researcher at Oak Ridge National Laboratory (@ORNLCyber). PhD @UTK_TCE. Applications of AI to cyber, usable security, network security.

[Jim Fan @DrJimFan]

Turns out LLMs can do some pretty esoteric things in-context!

My favorite paper this week systematically studies LLM beyond natural language. GPT can do complex 2D pattern recognition, fit a sinusoid, predict robot grasp, and even control an inverted pendulum in an online feedback loop.

Below is the extremely challenging ARC IQ benchmark by @fchollet. Without any finetuning, Both text-davinci-003 and PaLM can solve a non-trivial subset (85 and 42 out of 800) after flattening the 2D pattern into a 1D token sequence.

Consistent tokenization is critical. In fact, you can use random tokens sampled from the vocab and still get decent performance.

Next, GPT can predict robot grasp after converting a low-res image into a flat sequence of integer tokens! This suggests some basic computer vision emerging from an extremely powerful text pattern recognizer. Think of it as doing object detection on ASCII art.

Next, GPT can fit & extrapolate a sinusoid function by observing an array of continuous numbers and inferring its amplitude/frequency. This is done purely in-context and without the help of external tools like scikit-learn.

Finally, GPT can control a "left/right" button to balance an inverted pendulum (the classical "Cartpole" task in reinforcement learning) by reacting to the angle & velocity of the pole in an online feedback loop.

There're a lot more experiments & discussions in the paper, "Large Language Models as General Pattern Machines". Take a look!
- Link: https://t.co/1XjgkrRvGZ
- Authors rom Stanford & Google DeepMind: @suvir_m @xf1280 @peteflorence @brian_ichter @DannyDriess Montserrat Gonzalez Arenas, Kanishka Rao, @DorsaSadigh @andyzeng_

[Jim Fan @DrJimFan]

This is way more impactful than it sounds.

Python executes on a single thread regardless of how many CPU cores you have. Removing the notorious GIL (Global Interpreter Lock) means that Python can finally run true concurrency.

Can't wait to see this happen.

[Andrew White 🐦‍⬛ @andrewwhite01]

Code is here (including highly valuable prompts): https://t.co/IKcPVm5joQ
Still needs some work, but fun to play around with and has been legitimately useful for my own research. 5/5

[Boaz Barak @boazbaraktcs]

A paperclip maximizer, a stochastic parrot, king Midas, a jpeg of the web, markets, democracy, an alien, a shoggoth,...

So many metaphors offered for AI. I discuss these and their limitations.

https://t.co/nX4dfoFIrh

[Jim Fan @DrJimFan]

It’s fascinating that somewhere in Princeton’s fruit fly brain scan lies Nature’s algorithm of attention, working memory, and even basic consciousness.

Think of the connectome as a compiled executable binary. How much source code can we decompile from a full brain simulation? 🤔

[Akshay 🚀 @akshay_pachaar]

We've all dealt with activation functions while working with neural nets.

- Sigmoid
- Tanh
- ReLu & Leaky ReLu
- Gelu

Ever wondered why they are so important❓🤔

Let me explain it to you in this 🧵👇

[Lior Alexander @LiorOnAI]

GPT-Engineer just hit 12,000 stars on Github.

It's an AI agent that can write an entire codebase with a prompt and learn how you want your code to look.

▸ Asks clarifying questions
▸ Generates technical spec
▸ Writes all necessary code
▸ Easy to add your own reasoning steps, modify, and experiment
▸ Lets you finish a coding project in minutes.

[Andrew Ng @AndrewYNg]

I'd like to have a real conversation about whether AI is a risk for human extinction. Honestly, I don't get how AI poses this risk.

What are your thoughts? And, who do you think has a thoughtful perspective on how AI poses this risk that I should talk to?

[Malcolm Guite @malcolmguite]

In three in one and one in three, in rhyme,
In music, in the whole creation story,
In His own image, His imagination,
The Triune Poet makes us for His glory,
A Sonnet for Trinity Sunday https://t.co/nOL5RXBv6U

[Andrew Ng @AndrewYNg]

1/ Thrilled to announce: Our new course ChatGPT Prompt Engineering for Developers, created together with @OpenAI, is available now for free! Access it here: https://t.co/OaIpa6L2jn

[Lorenzo Green 〰️ @mrgreen]

R.I.P. Photoshop.

In just a few clicks, you'll be able to edit any image EXACTLY the way you want. 🤯

[Jiannan Xiang @szxiangjn]

🤔Humans can learn from embodied experiences in the physical world. Can Language Models also do that?

🔥Check out our new paper about enhancing Language Models with World Models!

👇https://t.co/41ez48ibp5

1/n

[Zhihao Jia @JiaZhihao]

Generative LLMs are slow and expensive to serve. Their much smaller, distilled versions are faster and cheaper but achieve suboptimal generative performance. We show it is possible to achieve the best of both worlds.
Code: https://t.co/bWc7jYjiFQ
Paper: https://t.co/YdA3nR79zg

[Yann LeCun @ylecun]

Remember when naïve hackers were called script kiddies?

What would be the equivalent designation for folks who think they are AI experts after a couple hours of LLM prompt engineering?

Prompt kiddies?

[Angry Tom @AngryTomtweets]

Open AI just released their most powerful tool yet.

The Code Interpreter.

Here are 10 insane examples I've found:

[George Mack @george__mack]

What is ignored by the media -- but will be studied by historians?

8 more examples:

[Rowan Cheung @rowancheung]

1. Hugging Face launches a ChatGPT alternative

Hugging Face launched a 30-billion-parameter open-sourced AI chatbot called HuggingChat

The chatbot aims to compete with OpenAI's ChatGPT.

[Alvaro Cintas @dr_cintas]

This paper is going viral.

Why?

Authors were able to find a way to enable the Recurrent Memory Transformer to retain information across up to 2 million tokens 🤯

In simpler words, it can process and remember vast amounts of data, significantly more than before.

Just so you have an idea, GPT-4 handles 32K tokens (~50 pages of documents), while the entire Harry Potter series is ~1.5M tokens.

Imagine the potential implications of this:

• AI could write entire novels, not just blog posts
• It could aid in complex scientific research by analyzing vast quantities of data
• Enhance customer service by retaining years of interaction history
• It could store and recall the entirety of your life experiences!!

[Mark Tenenholtz @marktenenholtz]

I am at least 3-5x more productive using ChatGPT to code.

Not only am I faster writing code I'm familiar with, but I've even shipped apps in tech stacks I'd never used before.

Here's my process, the prompts I use, and why it all works:

[Liv Ross @je_suis_liv]

My friend saw this book on @malcolmguite's shelf, decided that I needed one on my shelf, then hunted it down and surprised me with it today.

It contains works by favorites such as, Tennyson, Coleridge, Donne, Spenser, Solomon, Yeats.

And it was published the year I was born.