AI Pursuit

Join YouTube Membership:

[AI Pursuit @AiPursuit]

For more great research video~

Subscribe ⇢ https://t.co/awdU0UoMJl

[AI Pursuit @AiPursuit]

@realDonaldTrump 00:19 has shown signs of AI generated content

[Tim Denning @Tim_Denning]

Harsh truths I know at 39 I wish I knew at 29:

1. The solution to money problems is to make more money, not save money.

2. Entrepreneurship is hard. Being laid off as an employee is harder.

[AI Pursuit @AiPursuit]

Ilya Sutskever: Sequence to sequence learning with neural networks | NeurIPS 2024 Test of Time Award
https://t.co/3aoSU90mBo

[AI Pursuit @AiPursuit]

@NeurIPSConf Talk is cheap, action is needed, shame on NeurIPS.

[Thomas Wolf @Thom_Wolf]

a 3.25B params quantized gemini running locally in coming Google Chrome with less than 100ms latency while using less than 2GB of ram

that's less ram usage than many of my current Chrome page already use (my slack is using 4.8GB as I type this)

no doubt LLMs will be integrated everywhere in our world, from browsers to TV, to appliances, to Raspberry-Pi, to increasingly simpler and simpler devices as the tech is perfected

and what I'm most excited about is that they're all going to run locally, on-device, fast, decentralized

[Yujia Qin @TsingYoga]

我们离理想的AutoGPT还有多远？[1/4]

AutoGPT[1]已经163k star了，AutoGPT的开发者雕花了一年多，但它仍然停留在demo阶段，算不上产品（即使面向开发者)。这和传统开源软件的发展轨迹相差甚远，核心原因是Agent的上限由底座模型决定

GPT4从去年到现在发布了多个版本，实测起来对支持AutoGPT的能力并没有明显变化(甚至还有版本迭代后效果变差的情况）。可以明显感受到，OpenAI并没有为Agent任务专门优化，所谓GPT4“推理能力”的提升是面向评测起来比较简单的场景(例如代码、数学题等)，而Agent任务所需的“推理能力”和MATH / SWE-Bench任务的推理能力其实完全不是一个概念。因此，很大概率纯套壳类Agent工具day1的效果就是上限

我们离理想的AutoGPT还有多远？Agent能力细数起来其实很多，例如（1）选择适合的工具并填写正确的参数；（2）复杂场景下的推理能力；（3）反思上一个失败步骤的原因从而找到更优的解路径；（4）长序列memory问题等等——本质上其实都是幻觉问题：即底座模型没有见过该场景下的MDP决策序列（数据）。数据的缺失才是原罪

目前，无论是预训练阶段还是微调阶段的Agent数据都是几乎没有被收集/重视的。互联网上的数据本质上都是“静态快照”，或者说是“结果数据”，如何产生这些数据的用户行为过程（“过程数据”）是没有被记录下来的。后者没有统一的记录格式，仅零星地存在于做埋点测试的后端，从收集到被利用都还有很大距离。过程数据主动记录的成本也极高，参考学生团队[2]和大厂团队的标注效率[3]

在缺乏规模化数据的前提下想解决幻觉问题，常见的解决方案就是（1）RAG：即参考相似任务的解路径来辅助当前任务决策 [4]；或者（2）RLHF：基于先前的经验来迭代优化当前任务的路径。后者没有成熟的工作（[5]可参考），原因是通用Agent任务场景的reward model（evaluation）是该方法的卡点。去年在Tree-of-thought和Q*的消息出来后分别有一系列工作探索了用Monte-Carlo来bootstrap成功路径的工作（太多了，偷懒只列一篇[6]），都有涨点但没有做到特别实用的程度，核心原因也是evaluation的（1）准确率不够和（2）无法泛化或scalable[7]

=======
关于Agent还有挺多想分享的，后面有空了再写

[1] https://t.co/5Vva8PqJji
[2] https://t.co/Tc5zNylROP
[3] https://t.co/B6iqMpOJOH
[4] https://t.co/rw2LQ5ospm
[5] https://t.co/CoemfQRuhF
[6] https://t.co/j2UTPuR9zC
[7] https://t.co/CoJRRghGZ8

[AI Pursuit @AiPursuit]

@yangzhizheng1 @OpenBMB @zibuyu9 @yaoyuanAIR @Datou @ZHOZHO672070 @Gradio @AkshGarg03 @siddrrsh Perhaps they have a lot more works like this, anyone wanna dig into it?

[aarya @gd3kr]

I've been experimenting with embeddings a lot recently. Here's an interface I created that lets me interpolate between CLIP image embeddings and generate images from the intermediate latents

Latent Surfing. it's kinda like a color picker, but for images. (1/n)

[Assaf Elovic @assaf_elovic]

This is one of the most exciting releases yet. Proud to introduce the latest GPT Researcher x LangGraph integration showcasing the power of flow engineering and multi agent collaboration!

Generates a 6+ page research report in multiple formats such as PDF, Docx and Markdown. 🤯

Check out the open source here: https://t.co/JELbyOQHsR

Thanks to the @langchain team for this great framework!

[Yao Fu @Francis_YAO_]

From Claude100K to Gemini10M, we are in the era of long context language models. Why and how a language model can utilize information at any input locations within long context? We discover retrieval heads, a special type of attention head responsible for long-context factuality

[Sebastian Raschka @rasbt]

There's a new promising method for finetuning LLMs without modifying their weights called
proxy-tuning (by Liu et al. https://t.co/3PjF0NtlOM).
How does it work? It's a simple decoding-time method where you modify the logits of the target LLM. In particular, you compute the logits' difference between a smaller base and finetuning model, then apply the difference to the target model's logits.

More concretely, suppose the goal is to improve a large target model (M1).

The main idea is to take two small models:
- a small base model (M2)
- a finetuned base model (M3)

Then, you simply apply the difference in the smaller models' predictions (logits over the output vocabulary) to the target model M1.

The improved target model's outputs are calculated as M1*(x) = M1(x) + [M3(x) - M2(x)]

Based on the experimental results, this works surprisingly well. The authors tested this on
A. instruction-tuning
B. domain adaptation
C. task-specific finetuning

For brevity, focusing only on point A, here's a concrete example:

1) The goal was to improve a Llama 2 70B Base model to the level of Llama 2 70B Chat but without doing any RLHF to get the model from Base -> Chat.

2) They took a 10x smaller Llama 2 7B model and instruction-finetuned it.

3) After finetuning, they computed the difference in logits over the output vocabulary between 7B Base and 7B Finetuned

4) They applied the difference from 3) to the Llama 2 70B Base model. This pushed the 70B Base model's performance pretty close to 70B Chat.

The only caveat of this method is, of course, that your smaller models have to be trained on the same vocabulary as the larger model. Theoretically, if one knew the GPT-4 vocabulary and had access to its logit outputs, one could create new specialized GPT-4 models with this approach.

[HippoML @hippoml_com]

Introducing PrivateCanvas. Harness the power of your local GPU for contiguous editing and generating with cutting-edge models like Large Language Model, SDXL, Segment Anything, and GANs. Experience top-tier performance with minimal hardware demands. https://t.co/oo62q2LNcL

[Stella Biderman @BlancheMinerva]

Are you lost with all these new LLMs, trying to figure out what best practices are? Or even what practices different models follow? Do you wish documenting architectures didn't require delving into the code?

I've done all that work so you don't have to!

[Prof. Ambuj Varshney @ambuj123]

🚨 I am looking to mentor PhD students, postdocs and visiting students. There are openings in my group WEISER at @NUSingapore @NUSComputing

We are a leading group in the area of wireless embedded systems (aka IoT).

Please reach out and retweet 🙏

https://t.co/uwupvompVV

[Sasha Rush @srush_nlp]

Check out this wild talk tomorrow!

As far as I understand it... A couple of students proxy served chatgpt and collected 570k chats. This is what they learned.

[Andreas Madsen @andreas_madsen]

Since I started my PhD 3 years ago, I wanted to bridge the gap between post-hoc and intrinsic interpretability. As in, can we make general purpose models inherently interpretable?

Today, I'm happy to present: faithfulness measurable models, a new paradigm for interpretability!…

[AI Pursuit @AiPursuit]

@Stake Fiancé Chinese grandpa 97 years old birthday party! What a bang 🙈
angusstake

[AI Pursuit @AiPursuit]

@noahmorris Niche

[Jason Wei @_jasonwei]

Enjoyed visiting UC Berkeley’s Machine Learning Club yesterday, where I gave a talk on doing AI research. Slides: https://t.co/SGvVtL6Qks

In the past few years I’ve worked with and observed some extremely talented researchers, and these are the trends I’ve noticed:

1. When starting a project, average researchers tend to jump quickly to modeling proposals, architecture design, new ideas, etc. Great researchers often first spend time manually looking at data and playing with models to deeply understand the problem, before proposing an (often simple) approach.

2. Average researchers may often write hacky code that is not reusable and requires many separate steps. Great researchers are often also great software engineers—their code can be easily extended for future experiments, they write extensive tests, and they create infra to run many experiments quickly and visualize results with the fewest clicks.

3. While average researchers might work mostly by themselves or with one or two others, great researchers know that research is a social activity. They collaborate with people of varying experience, share results in writeups, and communicate their vision convincingly.

4. Average researchers might get stuck in rabbit holes—if they have experiments with only mediocre results, they spend 3 more weeks writing it up and submitting it to a conference. Great researchers quickly move on to something else when they know that one approach won’t be a breakthrough.

5. If an average researcher finds some success, they may try to keep doing that thing they are comfortable with for several more years, even if it becomes outdated. Great researchers pivot quickly and keep adapting to new advances and paradigms.

6. Average researchers often implement task-specific solutions, which are heavily optimized for a single task. Great researchers may also work on specific tasks, but they try to think of general approaches that can be applied to many other tasks.

7. Average researchers talk about and optimize for the number of papers or conference acceptances. I have never met a great researcher that still cares about such things.

(And by the way, being an average researcher shouldn’t be taken as an insult. It takes a lot of hard work to even do research at all :))