Top Tweets for #DeArena
"Welke rol spelen gevangenissen écht in onze samenleving?" dinsdag 10 februari #DeArena Debat met @lisimbo , @AnneliesVl , @bachrouriamir , @VincentVQ Walter Van Steenbrugge & Joëlle Rozie #viernulvier .... dat ik er naar uitkijk ! ;-)
@yasmienaciri 30 jaar geleden leidde de zaak ‘Bende van de miljardair’ tot een volledige doorlichting van het Gentse politiekorps! Nu zelfs geen doorlichting van de dienst waar betrokken rechercheur werkte 🙄 Enkel in #DeArena is er nog debat 😉
🚀🚀🚀 Finally releasing #DecentralizedArena #DeArena during the amazing week of #COLM2024!
Imagine a new paradigm of AI evaluation🤔: Can we have all LLMs govern themselves for a faster, smarter, and unbiased system to benchmark LLMs (and even future superintelligence models)?
🤖 Reflect the current LLM benchmarking system
Let's be realistic: traditional accuracy/dataset-based benchmarking systems have broken in the #LLM era. Human judgment and democracy seem to be the only trustworthy oracles, hence, the great Chatbot Arena https://t.co/nAXB0z5mUv.
But lately, people become obsessed with GPT-4-based evaluation 😶. Sure, it's fast, and yes, you can publish more, but we're all aware of the risks and biases of trusting a single (or a few) model. Numerous papers have pointed out these biases (e.g., https://t.co/U2nMkv0bAS; https://t.co/BoavmQ4svk; https://t.co/Ow6DPgQecq)
A natural question is that, can we train a better judge model to approximate human judgments? It's doable, and some folks are trying (e.g., Prometheus https://t.co/wGZvyJknKM, SFR-LLaMA https://t.co/9DmluoAx3J). But here's the kicker: democracy doesn't fit in a single model. That's like asking the king to write the constitution for a democracy. Weird, right?
🏟️ Enter the era of #DeArena
Instead, why not have all LLMs judge themselves? It's automated, scalable, and—get this—less biased. It's democracy at scale, powered by collective LLM intelligence. Welcome to a more transparent, autonomous, and reproducible future in AI evaluation.
🔧 Overcome the technical challenges
There’s no free lunch when it comes to democracy. Using all LLMs to judge each other sounds compute-heavy (N^3 complexity for pairwise judgments with N models).
Here comes the core technical novelty of DeArena, which is a sorting-based algorithm to sample pairs of models to compare and maximally distinguish models with close capabilities. It uses a coarse-to-fine-grained process to reduce the complexity to Nlog(N)! More details are in our blog https://t.co/abiYdOMFKB and upcoming papers.
For perspective, this novel algorithm can rank a new LLM in an incredible fast way. With a practical estimation, it takes about 1 week for Chatbot Arena to collect enough human votes to rank o1 models https://t.co/6Iw0Xxbhcq , while #DeArena can take about 2 days to rank the o1 with 100 LLMs.
🚀 Future of #DeArena: Scalable Oversight for Superintelligence?
At first glance, #DeArena is a fast version of Chatbot Arena (much faster of course), but what makes it extremely exciting is that it pictures a future for scalable oversight and benchmarking superintelligence systems. When humans can no longer reliably evaluate models, we’ll need to (or can only?) trust a democratic system where all LLMs participate in their own governance.
Already, #DeArena is showing it can rank LLMs on extremely challenging questions—those requiring expert-level human judgment, which, let’s face it, is both expensive and scarce.
🌐 Open-source commitment and involvement
Exciting times ahead! Stay tuned as we continue to expand #DeArena’s capabilities, covering even more evaluation dimensions while improving the algorithm for greater scalability and precision. 🚀
In the coming months, we'll be releasing more results, papers, and open-source tools to help the community better benchmark LLMs across any user-defined dimensions and understand their true boundaries. This is just the beginning of a more democratic and robust AI evaluation paradigm. 🔮
Join us on this journey to shape the future of AI oversight and benchmarking—together, we’re redefining how we trust and measure intelligence. 🌍💡
With @llm360, we're thrilled to release ⚔️Decentralized Arena built on Collective LLM Intelligence 🤖🤖
An automated "Chatbot Arena" for democratic LLM benchmarking on any fine-grained dimensions, e.g.:
🧮(Math) algebra, geometry, probability ...
💡(Reasoning) symbolic, social ...
🧪(Science) biology, chemistry ...
⌨️(Coding) python, C++, SQL ...
❓any other dims you care about!
🔥95% correlation with Chatbot Arena in "Overall" dimension (on 50+ models)
Key idea:
👉Chatbot Arena ⬅️Wisdom of the human crowds
👉Decentralized Arena ⬅️Wisdom of the LLM crowds
With @llm360, we're working to make Decentralized Arena fully transparent and reproducible
Blog: https://t.co/AmBQqtMztO
Leaderboard: https://t.co/0a9O520PQO
More interesting results:
1/
zeer hoogwaardig debat mogen meemaken @VIERNULVIERGent ‘Trial by media’ #dearena met @NicholasLataire #waltervansteenbrugge @RuttenGwendolyn @yasmienaciri moderator: @lisimbo #top
Was tot #Ajax' #Beursgang #DeArena #Adidas #TIPS
een diehard #Ajacied
maar ben NOOIT anti-#Feyenoord/#PSV geweest
vind #tribalisme nl #kortzichtig
ik was geïnteresseerd in de sport
#TrueSport Moraal
#sportief zijn
= de basis van alles
zodra dat weg is = het zinloos
@bartdecl @Karl_Drabbe Zelfs op de eerste debatavond #deArena @VIERNULVIERGent ‘Jongeren en Criminaliteit’ met @lisimbo @VSAdvocaten @yasmienaciri @JorgenVanLaer #ArnoVanOverberghe was hij aanwezig 😉
Eerste debatavond #deArena @VIERNULVIERGent ‘Jongeren en Criminaliteit’ met @lisimbo @VSAdvocaten @yasmienaciri @JorgenVanLaer #ArnoVanOverberghe doet al verlangen naar de volgende ‘Kerk en Justitie’ 👊
Estaremos en
@BuenaVibraFest
22 y 23 de #junio
En @huertoromaverde
Jalapa 234 , #colRoma
#buenavibra #Ganesha #decor #dearena #DeArena https://t.co/XVWhx2jH1v
De zo geroemde #verdediging van #nederland heeft het ff niet helemaal scherp #NedDui. Spelen in #dekuip lijkt voor het #nederlandselftal prettiger dan speen in #dearena
¡Todos a armar el arbolito! #CuentaRegresiva #DaleQueNoFaltaNada #EsHoy #MiráElPesebreInauguradoEnElVaticano #DeArena #ArtePura #ImágenesImperdibles #DinDonDan #LlegaNavidad #OhOhOh https://t.co/pVCAAoWzdP
Vente a nuestra tienda @En_laDelValle a conocer la nueva colección de #anillos de #DeArena los vas a amar #accesorios
Heej @HenkHoijtink Geniet jij ook zo van het sprankelende flitsende voetbal #deArena Mooi man , niet om aan te zien
#diciembre 2 y 3 #buenavibrafest #San Angel # cdmx
#DeArena
#mandalas #indianjewellery #onix… https://t.co/xYp6atz1CZ
Last Seen Hashtags on Sotwe
نودز
straightbait
Seen from Germany
teenageee filter:videos
Seen from Australia
cnc video*
Seen from Australia
baited
Seen from Latvia
somno
Seen from Turkey
minichat
Seen from United States
RandomChat
Seen from Turkey
dogsex() +filter:native_video
Seen from Pakistan
teenageee candid
Seen from Netherlands
Trends for you
Most Popular Users
Elon Musk 
@elonmusk
241.7M followers
Barack Obama
@barackobama
119M followers
Cristiano Ronaldo
@cristiano
114.4M followers
Donald J. Trump
@realdonaldtrump
111.9M followers
Narendra Modi
@narendramodi
107.2M followers
Rihanna
@rihanna
98.7M followers
NASA
@nasa
92.4M followers
Justin Bieber
@justinbieber
91.8M followers
KATY PERRY
@katyperry
90M followers
Taylor Swift
@taylorswift13
83.9M followers
Lady Gaga
@ladygaga
75.4M followers
Virat Kohli
@imvkohli
73.3M followers
Kim Kardashian
@kimkardashian
70.9M followers
YouTube
@youtube
68.8M followers
Neymar Jr
@neymarjr
66.3M followers
Bill Gates
@billgates
65.1M followers
Selena Gomez
@selenagomez
63M followers
The Ellen Show
@theellenshow
62.3M followers
CNN
@cnn
61.8M followers
X
@x
60.7M followers