Navid Madani

NLP researcher @ University at Buffalo

[Raj Dabre @prajdabre]

As a senior researcher who has both published in A* venues and has written toolkits himself and overseen the development of several others, it is humiliating to have to do leetcode grinding for a research scientist role in big tech. I have written more code than most people and I still will be judged for not being able to solve a coding question, WHICH BY THE WAY IS NOT SOMETHING YOU WILL DO A LOT IN BIG TECH BECAUSE WE HAVE CODING ASSISTANTS NOW FFS!

[Navid Madani @namadvid]

@BuffaloBills I won't say the running game is too obvious cause a lot of factors can affect your decision to do that, BUT, why don't we see any improvement in hand over gesture by josh? I mean why should it be so obviously executed?

[Navid Madani @namadvid]

@Paul77031270 @BuffaloBills +1 I mean, can it be more predictable?! why don't we see a very basic faking or some gesture when running the ball? I keep seeing this every game and it sounds like a very simple method to at least delay the defense even for a split of a second.

[Kenny Joseph @_kenny_joseph]

Our* paper on embedding social media bios into dimensions of social meaning was recently accepted @icwsm.

We think bios are a unique place of self-expression worth studying/using!

* @namadvid @RabirajBandyop1 @mccabe_s @michaelmyoder @Briony_Swire

https://t.co/uuVqik68Jt

[Noam Brown @polynoamial]

Today, I’m excited to share with you all the fruit of our effort at @OpenAI to create AI models capable of truly general reasoning: OpenAI's new o1 model series! (aka 🍓) Let me explain 🧵 1/

[Prakash @8teAPi]

o1 personal testing megathread 🧵

Bookmark if you need to, just keeping track of reactions since a lot of us have held out personal test sets

[elvis @omarsar0]

Open Source LLM Tools

If you are looking for useful open-source LLM tools, this is a really useful resource.

It includes different categories like tutorials, AI engineering, and applications, among others. You can also see the # of GitHub stars.

[Yann LeCun @ylecun]

"Our country is being poisoned" by immigrants, like me, @elonmusk, @sundarpichai, @satyanadella, @vkhosla, Jensen Huang, and two of Trump's three wives.

[Andrej Karpathy @karpathy]

I feel like a large amount of GDP is locked up because it is difficult for person A to very conveniently pay 5 cents to person B. Current high fixed costs per transaction force each of them to be of high enough amounts, which results in business models with purchase bundles, subscriptions, ad-based, etc., instead of simply pay-as-you-go. As an example, I'd like my computer to auto-pay 5 cents to the article/blog that I just read but I can't, and I think we're worse for it.

In a capitalist system, transactions between entities are the gradient signal of the economy. Because our pipes don't support low magnitude terms in the sums, the gradients are not flowing properly through the system. I'm not familiar enough with payments to have an idea of specific solutions, but I expect we'd see a lot of positive 2nd / 3rd order effects if the gradients were allowed to flow properly, frictionlessly and with much higher resolution.

[Aran Komatsuzaki @arankomatsuzaki]

Meta presents UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling

- Scaling offers little benefit for reasoning or relations
- Best VLMs struggle on simple digit recognition and counting tasks, e.g. MNIST

repo: https://t.co/Jbulll38Ji
abs: https://t.co/hhM6TwDZ2w

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

LLM-as-a-Judge is one of the most widely-used techniques for evaluating LLM outputs, but how exactly should we implement LLM-as-a-Judge?

To answer this question, let’s look at a few widely-cited papers / blogs / tutorials, study their exact implementation of LLM-as-a-Judge, and try to find some useful patterns.

(1) Vicuna was one of the first models to use LLMs as an evaluator. Their approach is different depending on the problem being solved. Separate prompts are written for i) general, ii) coding, and iii) math questions. Each domain-specific prompt introduces some extra, relevant details compared to the vanilla prompt. For example:

- The coding prompt provides a list of desirable characteristics for a good solution.
- The math prompt asks the judge to first solve the question before generating a score.

Interestingly, the judge is given two model outputs within its prompt, but it is asked to score each output on a scale of 1-10 instead of just choosing the better output.

(2) AlpacaEval is one of the most widely-used LLM leaderboards, and it is entirely based on LLM-as-a-Judge! The current approach used by AlpacaEval is based upon GPT-4-Turbo and uses a very simple prompt that:

- Provides an instruction to the judge.
- Gives the judge two example responses to the instruction.
- Asks the judge to identify the better response based on human preferences.

Despite the simplicity, this strategy correlates very highly with human preference scores (i.e., 0.9+ Spearman correlation with chatbot arena).

(3) G-Eval was one of the first LLM-powered evaluation metrics that was shown to correlate well with human judgements. The key to success for this metric was to leverage a two-stage prompting approach. First, the LLM is given the task / instruction as input and asked to generate a sequence of steps that should be used to evaluate a solution to this task. This approach is called AutoCoT. Then, the LLM uses this reasoning strategy as input when generating an actual score, which is found to improve scoring accuracy!

(4) The LLM-as-a-Judge paper itself uses a pretty simple prompting strategy to score model outputs. However, the model is also asked to provide an explanation for its scores. Generating such an explanation resembles a chain-of-thought prompting strategy and is found to improve scoring accuracy. Going further, several different prompting strategies–including both pointwise and pairwise prompts–are explored and found to be effective within this paper.

Key takeaways. From these examples, we can arrive at a few common takeaways / learnings:
- LLM judges are very good at identifying responses that are preferable to humans (due to training with RLHF).
- Creating specialized evaluation prompts for each domain / application is useful.
- Providing a scoring rubric or list of desirable properties for a good solution can be helpful to the LLM.
- Simple prompts can be extremely effective (don’t make it overly complicated!).
- Providing (or generating) a reference solution for complex problems (e.g., math) is useful.
- CoT prompting (in various forms) is helpful.
- Both pairwise and pointwise prompts are commonly used.
- Pairwise prompts can either i) ask for each output to be scored or ii) ask for the better output to be identified.

[Sakana AI @SakanaAILabs]

Introducing Evolutionary Model Merge: A new approach bringing us closer to automating foundation model development. We use evolution to find great ways of combining open-source models, building new powerful foundation models with user-specified abilities!

https://t.co/QIDTHYHQrM

[Andrej Karpathy @karpathy]

# on technical accessibility

One interesting observation I think back to often:
- when I first published the micrograd repo, it got some traction on GitHub but then somewhat stagnated and it didn't seem that people cared much.
- then I made the video building it from scratch, and the repo immediately went through hockey stick growth and became a verty often cited reference for people learning backpropagation.

This was interesting because the micrograd code itself didn't change at all and it was up on GitHub for many months before, stagnating. The code made sense to me (because I wrote it), it was only ~200 lines of code, it was extensively commented in the .py files and in the Readme, so I thought surely it was clear and/or self-explanatory. I was very happy with myself about how minimal the code was for explaining backprop - it strips away a ton of complexity and just gets to the very heart of an autograd engine on one page of code. But others didn't seem to think so, so I just kind of brushed it off and moved on.

Except it turned out that what stood in its way was "just" a matter of accessibility. When I made the video that built it and walked through it, it suddenly almost 100X'd the overall interest and engagement with that exact same piece of code. Not only from beginners in the field who needed the full intro and explanation, but even from more technical/expert friends, who I think could have understood it if they looked at it long enough, but were deterred by a barrier to entry.

I think as technical people we have a strong bias to put up code or papers or the final thing and feel like things are mostly self-explanatory. It's there, and also it's commented, there is a Readme, so all is well, and if people don't engage then it's just because the thing is not good enough. But the reality is that there is still a large barrier to engage with your thing (even for other experts who might not feel like spending time/effort!), and you might be leaving somewhere 10-100X of the potential of that exact same piece of work on the table just because you haven't made it sufficiently accessible.

TLDR: Step 1 build the thing. Step 2 build the ramp. 📈
Some voice in your head will tell you that this is not necessary, but it is wrong.

[Andrej Karpathy @karpathy]

The most unknown most common shortcut I use on my MacBook is:

- Command+Option+Shift+4 to select a small part of the screen and copy it into clipboard as an image
- Command+Shift+4 to do the same, but save it as a file on Desktop as png

Life-changing.

[Zoubin Ghahramani @ZoubinGhahrama1]

A nice way to end the year with some data: 66 Good News Stories You Didn't Hear About in 2023 https://t.co/isA9gW1fzD

[elvis @omarsar0]

Really enjoyed NeurIPS!

After attending great sessions around LLMs, I documented a huge list of interesting LLM papers that were either presented or mentioned.

Here is a list of some of my favorite papers in no particular order. I have included papers that won awards and are pushing ideas that we will continue hearing more about:

--
Chain of Code: Reasoning with a Language Model-Augmented Code Emulator - https://t.co/d0MFRyIzsy
--
Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective - https://t.co/nAthjZLRMr
--
Scaling Data-Constrained Language Models - https://t.co/JWLFat0xDG
--
Language to Rewards for Robotic Skill Synthesis - https://t.co/KrwMXXyXDC
--
Tree of Thoughts: Deliberate Problem Solving with Large Language Models - https://t.co/nfZ2N6mslU
--
Why think step by step? Reasoning emerges from the locality of experience - https://t.co/TkffQfXsOh
--
Toolformer: Language Models Can Teach Themselves to Use Tools - https://t.co/u7GanCTUjI
--
Reasoning with Language Model is Planning with World Model - https://t.co/Gd7HH9hEkQ
--
ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings - https://t.co/fGssZNKECL
--
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models - https://t.co/owSD4bAqqC
--
QLoRA: Efficient Finetuning of Quantized LLMs - https://t.co/ILgUMZtk1x
--
Direct Preference Optimization: Your Language Model is Secretly a Reward Model - https://t.co/4DHb7Iot4T
--
Are Emergent Abilities of Large Language Models a Mirage? - https://t.co/HK4vDUTPmc
--
Reverse Engineering Self-Supervised Learning - https://t.co/DNUtj85ZOs
--
Learning Transformer Programs - https://t.co/b5kbJM16AY
--
OpenAssistant Conversations -- Democratizing Large Language Model Alignment - https://t.co/ji5ZFwZlNc
--
Privacy Auditing with One (1) Training Run - https://t.co/CmZYwMnVE5
--
Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning - https://t.co/uvNyqMnRiJ
--
Large Language Models as Zero-Shot Conversational Recommenders - https://t.co/oeSRM8kYeN
--
Zephyr: Direct Distillation of LM Alignment - https://t.co/9ihZGCm62I
--

I am also putting together a year in review later in the week or early next week so you will see other cool and important LLM papers that were published throughout the year. Stay tuned!

Feel free to comment with your favorite papers as well.

[Carlos E. Perez @IntuitMachine]

1/n Was December 8th, 2023, the day when we've come to realize that AGI technology has been democratized? That it cannot be confined to the few and the GPU-rich? Let me explain to you what happened yesterday.

[Navid Madani @namadvid]

@Lianhuiq @ucsd_cse Sounds interesting! sent you an email.

[Tanishq Mathew Abraham, Ph.D. @iScienceLuvr]

Language Model Inversion

abs: https://t.co/KO5DsUR75Z

Given the logits of the language model output (such as those returned by an API), a trained inversion model is able to reconstruct the original prompt. With Llama-2-7b-chat, their method is able to achieve a token-level F1 of 78 and able to reconstruct the prompts exactly 27% of the time.

[Sergey Levine @svlevine]

How do we get LLMs to interact with humans intelligently? Ask clarifying questions and reason about dialogue outcomes, vs. just single responses? Key idea: get LLMs to "simulate" human dialogues, then use offline RL on simulated data to learn optimal dialogue agents! A thread 👇