Top Tweets for #deArena
"Welke rol spelen gevangenissen écht in onze samenleving?" dinsdag 10 februari #DeArena Debat met @lisimbo , @AnneliesVl , @bachrouriamir , @VincentVQ Walter Van Steenbrugge & Joëlle Rozie #viernulvier .... dat ik er naar uitkijk ! ;-)
@yasmienaciri 30 jaar geleden leidde de zaak ‘Bende van de miljardair’ tot een volledige doorlichting van het Gentse politiekorps! Nu zelfs geen doorlichting van de dienst waar betrokken rechercheur werkte 🙄 Enkel in #DeArena is er nog debat 😉
🚀🚀🚀 Finally releasing #DecentralizedArena #DeArena during the amazing week of #COLM2024!
Imagine a new paradigm of AI evaluation🤔: Can we have all LLMs govern themselves for a faster, smarter, and unbiased system to benchmark LLMs (and even future superintelligence models)?
🤖 Reflect the current LLM benchmarking system
Let's be realistic: traditional accuracy/dataset-based benchmarking systems have broken in the #LLM era. Human judgment and democracy seem to be the only trustworthy oracles, hence, the great Chatbot Arena https://t.co/nAXB0z5mUv.
But lately, people become obsessed with GPT-4-based evaluation 😶. Sure, it's fast, and yes, you can publish more, but we're all aware of the risks and biases of trusting a single (or a few) model. Numerous papers have pointed out these biases (e.g., https://t.co/U2nMkv0bAS; https://t.co/BoavmQ4svk; https://t.co/Ow6DPgQecq)
A natural question is that, can we train a better judge model to approximate human judgments? It's doable, and some folks are trying (e.g., Prometheus https://t.co/wGZvyJknKM, SFR-LLaMA https://t.co/9DmluoAx3J). But here's the kicker: democracy doesn't fit in a single model. That's like asking the king to write the constitution for a democracy. Weird, right?
🏟️ Enter the era of #DeArena
Instead, why not have all LLMs judge themselves? It's automated, scalable, and—get this—less biased. It's democracy at scale, powered by collective LLM intelligence. Welcome to a more transparent, autonomous, and reproducible future in AI evaluation.
🔧 Overcome the technical challenges
There’s no free lunch when it comes to democracy. Using all LLMs to judge each other sounds compute-heavy (N^3 complexity for pairwise judgments with N models).
Here comes the core technical novelty of DeArena, which is a sorting-based algorithm to sample pairs of models to compare and maximally distinguish models with close capabilities. It uses a coarse-to-fine-grained process to reduce the complexity to Nlog(N)! More details are in our blog https://t.co/abiYdOMFKB and upcoming papers.
For perspective, this novel algorithm can rank a new LLM in an incredible fast way. With a practical estimation, it takes about 1 week for Chatbot Arena to collect enough human votes to rank o1 models https://t.co/6Iw0Xxbhcq , while #DeArena can take about 2 days to rank the o1 with 100 LLMs.
🚀 Future of #DeArena: Scalable Oversight for Superintelligence?
At first glance, #DeArena is a fast version of Chatbot Arena (much faster of course), but what makes it extremely exciting is that it pictures a future for scalable oversight and benchmarking superintelligence systems. When humans can no longer reliably evaluate models, we’ll need to (or can only?) trust a democratic system where all LLMs participate in their own governance.
Already, #DeArena is showing it can rank LLMs on extremely challenging questions—those requiring expert-level human judgment, which, let’s face it, is both expensive and scarce.
🌐 Open-source commitment and involvement
Exciting times ahead! Stay tuned as we continue to expand #DeArena’s capabilities, covering even more evaluation dimensions while improving the algorithm for greater scalability and precision. 🚀
In the coming months, we'll be releasing more results, papers, and open-source tools to help the community better benchmark LLMs across any user-defined dimensions and understand their true boundaries. This is just the beginning of a more democratic and robust AI evaluation paradigm. 🔮
Join us on this journey to shape the future of AI oversight and benchmarking—together, we’re redefining how we trust and measure intelligence. 🌍💡
With @llm360, we're thrilled to release ⚔️Decentralized Arena built on Collective LLM Intelligence 🤖🤖
An automated "Chatbot Arena" for democratic LLM benchmarking on any fine-grained dimensions, e.g.:
🧮(Math) algebra, geometry, probability ...
💡(Reasoning) symbolic, social ...
🧪(Science) biology, chemistry ...
⌨️(Coding) python, C++, SQL ...
❓any other dims you care about!
🔥95% correlation with Chatbot Arena in "Overall" dimension (on 50+ models)
Key idea:
👉Chatbot Arena ⬅️Wisdom of the human crowds
👉Decentralized Arena ⬅️Wisdom of the LLM crowds
With @llm360, we're working to make Decentralized Arena fully transparent and reproducible
Blog: https://t.co/AmBQqtMztO
Leaderboard: https://t.co/0a9O520PQO
More interesting results:
1/
zeer hoogwaardig debat mogen meemaken @VIERNULVIERGent ‘Trial by media’ #dearena met @NicholasLataire #waltervansteenbrugge @RuttenGwendolyn @yasmienaciri moderator: @lisimbo #top
Was tot #Ajax' #Beursgang #DeArena #Adidas #TIPS
een diehard #Ajacied
maar ben NOOIT anti-#Feyenoord/#PSV geweest
vind #tribalisme nl #kortzichtig
ik was geïnteresseerd in de sport
#TrueSport Moraal
#sportief zijn
= de basis van alles
zodra dat weg is = het zinloos
@bartdecl @Karl_Drabbe Zelfs op de eerste debatavond #deArena @VIERNULVIERGent ‘Jongeren en Criminaliteit’ met @lisimbo @VSAdvocaten @yasmienaciri @JorgenVanLaer #ArnoVanOverberghe was hij aanwezig 😉
Eerste debatavond #deArena @VIERNULVIERGent ‘Jongeren en Criminaliteit’ met @lisimbo @VSAdvocaten @yasmienaciri @JorgenVanLaer #ArnoVanOverberghe doet al verlangen naar de volgende ‘Kerk en Justitie’ 👊
Estaremos en
@BuenaVibraFest
22 y 23 de #junio
En @huertoromaverde
Jalapa 234 , #colRoma
#buenavibra #Ganesha #decor #dearena #DeArena https://t.co/XVWhx2jH1v
De zo geroemde #verdediging van #nederland heeft het ff niet helemaal scherp #NedDui. Spelen in #dekuip lijkt voor het #nederlandselftal prettiger dan speen in #dearena
¡Todos a armar el arbolito! #CuentaRegresiva #DaleQueNoFaltaNada #EsHoy #MiráElPesebreInauguradoEnElVaticano #DeArena #ArtePura #ImágenesImperdibles #DinDonDan #LlegaNavidad #OhOhOh https://t.co/pVCAAoWzdP
Vente a nuestra tienda @En_laDelValle a conocer la nueva colección de #anillos de #DeArena los vas a amar #accesorios
Heej @HenkHoijtink Geniet jij ook zo van het sprankelende flitsende voetbal #deArena Mooi man , niet om aan te zien
#diciembre 2 y 3 #buenavibrafest #San Angel # cdmx
#DeArena
#mandalas #indianjewellery #onix… https://t.co/xYp6atz1CZ
Last Seen Hashtags on Sotwe
LaBurbujaBukele
Seen from United States
รับงานปากช่อง
Seen from Thailand
粘着拘束
Seen from Korea
teenageee
Seen from Turkey
4canseachonbus
Seen from United States
miseEnEchec
Seen from United States
brosis
Seen from United States
MikeWeatherleyIsaCoward
Seen from United States
darkzadie (())*******************************************
Seen from Canada
ASDSSkinMD
Seen from United States
Trends for you
Most Popular Users
Elon Musk 
@elonmusk
241.7M followers
Barack Obama
@barackobama
119M followers
Cristiano Ronaldo
@cristiano
114.4M followers
Donald J. Trump
@realdonaldtrump
111.9M followers
Narendra Modi
@narendramodi
107.2M followers
Rihanna
@rihanna
98.7M followers
NASA
@nasa
92.4M followers
Justin Bieber
@justinbieber
91.8M followers
KATY PERRY
@katyperry
90M followers
Taylor Swift
@taylorswift13
83.9M followers
Lady Gaga
@ladygaga
75.4M followers
Virat Kohli
@imvkohli
73.3M followers
Kim Kardashian
@kimkardashian
70.9M followers
YouTube
@youtube
68.8M followers
Neymar Jr
@neymarjr
66.3M followers
Bill Gates
@billgates
65.1M followers
Selena Gomez
@selenagomez
63M followers
The Ellen Show
@theellenshow
62.3M followers
CNN
@cnn
61.8M followers
X
@x
60.7M followers