Imagine a population of machine agents. Each might be strong on certain tasks but fundamentally limited: partial tools, partial observations, finite context, bounded compute.
How can these agents self-orchestrate and self-evolve into stronger collective intelligence to solve tasks beyond any single agent's capability?
Instead of designing the multi-agent system itself, we propose designing the incentives that govern it.
We put agents in an economy. They compete, trade, get wealthy, go bankrupt, and mutate, forming an alive society where coordination and adaptation automatically emerge in a decentralized manner.
NEW POST
Thoughtworks internal IT use a workflow for agentic programming called Structured-Prompt-Driven Development (SPDD). @WeiZhang595190 and Jessie Jie Xia describe how this works with a simple example plus details in a github project.
https://t.co/6cHnSPWr6L
One of the most important skills I teach my students&postdocs is how to communicate their research. Last week at NSF, I gave a keynote to (mostly) junior faculty on how to pitch their work. The talk resonated with many folks, so I'm sharing it publicly: https://t.co/CpK88NfbPK
I gave a talk at Seoul National University.
I titled the talk “Large Language Models (in 2023)”. This was an ambitious attempt to summarize our exploding field.
Video: https://t.co/vumzAtUvBl
Slides: https://t.co/IidLe4JfrC
Trying to summarize the field forced me to think about what really matters in the field. While scaling undeniably stands out, its far-reaching implications are more nuanced. I share my thoughts on scaling from three angles:
1) Change in perspective is necessary because some abilities only emerge at a certain scale. Even if some abilities don’t work with the current generation LLMs, we should not claim that it doesn’t work. Rather, we should think it doesn’t work yet. Once larger models are available many conclusions change.
This also means that some conclusions from the past are invalidated and we need to constantly unlearn intuitions built on top of such ideas.
2) From first-principles, scaling up the Transformer amounts to efficiently doing matrix multiplications with many, many machines. I see many researchers in the field of LLM who are not familiar with how scaling is actually done. This section is targeted for technical audiences who want to understand what it means to train large models.
3) I talk about what we should think about for further scaling (think 10000x GPT-4 scale). To me scaling isn’t just doing the same thing with more machines. It entails finding the inductive bias that is the bottleneck in further scaling.
I believe that the maximum likelihood objective function is the bottleneck in achieving the scale of 10000x GPT-4 level. Learning the objective function with an expressive neural net is the next paradigm that is a lot more scalable. With the compute cost going down exponentially, scalable methods eventually win. Don’t compete with that.
In all of these sections, I strive to describe everything from first-principles. In an extremely fast moving field like LLM, no one can keep up. I believe that understanding the core ideas by deriving from first-principles is the only scalable approach.
Here's my conversation with Mark Zuckerberg, his 3rd time on the podcast, but this time we talked in the Metaverse as photorealistic avatars. This was one of the most incredible experiences of my life. It really felt like we were talking in-person, but we were miles apart 🤯 It's hard to put into words how awesome this was for someone like me who values the intimacy of in-person conversation. It gave me a glimpse of an exciting future with many new possibilities and fascinating questions about the nature of reality and human connection ❤
Timestamps:
0:00 - Introduction
0:52 - Metaverse
15:27 - Quest 3
30:16 - Nature of reality
34:54 - AI in the Metaverse
51:51 - Large language models
57:49 - Future of humanity
"Involution," which refers to China's hyper-competitive #work#culture, has gained traction with elite #students and younger white-collar #office#workers, especially those in the #tech industry. It has also caught the government's attention. https://t.co/5YyoSgJoz3
My course on differential privacy is public!
Check out the course page for: lecture videos 🎥, lecture notes 📝, and suggested readings 📖!
https://t.co/srlq0tGfzs
Be sure to subscribe on Youtube to catch the latest lectures!
https://t.co/rl6Tx6DSh7
#privacy
[3/7] Why is the datacenter in Quincy, WA?
AI eats compute which is fed by power and cooling (+=power). This is a massive Opex to cloud providers.
But in Quincy you pay 3ct/kWh! (SF=30ct/kWH). This is bc they have hydro (green!) power.
I guess that makes ChatGPT "green"?
Day 2 of the spring school #SocialXR! Coming up in a few moments: the lecture ‘Virtual Social Interaction and its Applications in Health and #Healthcare’ by @panxueni. Check out the livestream here: https://t.co/wvyyRog9cf
The more of an expert you are, the more you feel you’re “late to the party” when something new comes out.
I felt late to AI in 2015, late to generative AI in 2017.
Idk who needs to hear this but: it’s still just beginning. You’re not late at all. Come make some noise.
#CFP#networking The IEEE/ACM International Symposium on Quality of Service (IWQoS 2023) https://t.co/wraWcdnN1q,
19–21 June 2023, Orlando, FL, USA is calling for papers. This year's IWQoS is an affiliated conference with FCRC. Really amazing experience. Please join us. Cheers.
Hey, new faculty!! Congrats on your new jobs. If you’d find it helpful, I’ve made a list of the stuff that’s made my life a lot easier in this chaos job. I can’t promise I know what I’m doing myself, but I hope these might help you as you adjust! 1/probably too many
At CVPR and interested in INRs / Neural Fields / Neural Scene Representations? Come to our tutorial on Monday! We'll cover fundamental techniques and latest advances in Neural Fields, and reflect on what's next together with exciting invited speakers! https://t.co/Jz7NQYNkz9 1/4