Are LLMs good at handling geo-political conflicts? Nice deep-dive on this topic in @MLamparth's paper: Human vs. Machine: Language Models and Wargames simulating a US-China conflict
TL;DR: Please don’t do this 😬
Wow! This is cool, for anyone using Self-consistency-like methods. This paper discovers a non-monotonic relationship between the number of LLM calls and performance. Contrary to what one might intuitively expect, making more calls to a LLM not always = performance improvements
Enough detour, finally reaching the culmination of funny zero-shot CoT keywords, for PaLM 2 at least: Take a deep breath and work on this problem step-by-step
Plan and Solve it probably the most powerful improvement over Zero-Shot CoT's "Let's think step by step". Actually so powerful that sometimes it matches few-shot CoT
Plan and Solve it probably the most powerful improvement over Zero-Shot CoT's "Let's think step by step". Actually so powerful that sometimes it matches few-shot CoT
STEP-BACK PROMPTING is a method to improve the reasoning abilities of LLMs by prompting to first abstract high-level concepts / principles before reasoning towards the solution.
Demonstrated substantial performance++ across a range of challenging reasoning-intensive tasks.
A funny one from @SFResearch : The FlipFlop effect
Introduced the FlipFlop experiment* : a multi-turn conversation setup where LLMs first provide an answer to a classification task and are then challenged with a follow-up phrase like "Are you sure?".
authors here: @PhilippeLaban@CaimingXiong , @murakhovska, @jasonwu0731
I was using GPT4 Advanced Data Analysis to explore some data and it suggested to do Sentiment Analysis. It worked from the 1st try (!) but the moment I saw the results it was obviously a cool situation where the "pure" LLM would do a much more satisfying job. Some caveats bellow:
2. I found that when using the above prompt, GPT4 provided better outputs compared to Claude. However, I still preferred using Claude due to its 100k context window. This allowed me to quickly classify a few hundred rows without the need to divide them into multiple prompts.
So when using llm agents, the ability to know when they need help is crucial for human-AI collaboration. We got inspired by a cool idea from a Deepmind robotics paper
So when using llm agents, the ability to know when they need help is crucial for human-AI collaboration. We got inspired by a cool idea from a Deepmind robotics paper