@Mila_Quebec @UdeM on ML for remote sensing, @mcgillu CS alum into: ML for climate & societal impacts, STS, FATE prev: @SSofCS she/her @[email protected]
🚀We launch Evaluation Cards (beta): a centralized public record of AI evaluation results 🚀
Not another leaderboard. Every score comes with who ran it, the settings they used, what the benchmark tests and the other results reported for the same model, side by side. 🧵👇
It’s official: the first large-scale inherently interpretable language model is here.
Steerling-8B from @guidelabsai is the first and largest model that can trace every token it generates back to:
→ Input Context
→ Training data
→ Human-understandable concepts
In other words, we've successfully trained Steerling-8B to trace its outputs and explain what has impacted that decision for more reliable manipulation. This isn’t post-hoc explainability. Interpretability is built directly into the model.
🔓Steerling-8B can self-monitor for memorized content and suppress it at inference time without retraining. That makes interpretability a first-class design principle, not an afterthought.
This is a major step toward models we can actually understand, debug, and trust.
Over the coming days, we’ll be sharing investigations into what Steerling-8B’s interpretability enables in practice. Stay tuned as we dive deeper into our research & how we are building LLMs we can trust.
🚨 Try it LIVE and help improve it:
Guide Labs: https://t.co/EyEMFz2p9O
GitHub: https://t.co/PIVwJgleFP
Hugging Face: https://t.co/0apB117l4o
Huge thank you to @TimFernholz and @TechCrunch for featuring this breakthrough. https://t.co/DIZpq5XqGS
#Steerling8B #GuideLabs #AI #MachineLearning
1/ 💻 Queer in AI is hosting a social at #ICML2025 in Vancouver on 📅 July 16, and you’re invited! Let’s network, enjoy food and drinks, and celebrate our community. Details below…
🚗💥Introducing Ctrl-Crash: controllable video generation for autonomous driving! SOTA models struggle to generate physically realistic car crashes. We propose an image2video diffusion model with bounding box and crash type control.
Website: https://t.co/vNBYhbx3c4
🧵->
Discover the Evaluating Evaluations: Examining Best Practices for Measuring Broader Impacts of Generative AI workshop, co-organized by Mila student @XMichelleLinX at @NeurIPSConf, East Meeting Room 16.
⚛️🤗 Announcing LeMaterial ⚛️🤗
@huggingface & @entalpic_ai are teaming up to release LeMaterial -- an open source initiative aiming to facilitate (AI for) materials discovery !
Datasets, hash function, tools to explore the chemical space & more !
https://t.co/7xJO2SxXqN
With less than a week to go for NeurIPS 2024, I wanted to make a small thread to celebrate our little workshop, EvalEval, and the incredible amount of interest and love we have received in our first time organizing it.
But first, What does Evaluate Evaluations mean?
🌈🌈We have an incredible day planned for you at our #QueerInAI workshop at #NeurIPS2024 in Vancouver on December 11! You can join us in person or virtually via live-stream! Find out more about our events and our lovely speakers at https://t.co/AawLvbQVqa!! ✨✨
Join us for our in-person social for #NeurIPS2024 on Thursday, December 12 at 8pm at The Metropole Community Pub! 🌈✨🌈✨ We can’t wait to see you all and to mingle while enjoying the lovely city vibes! Check out our website https://t.co/2QVhSBKiIu for more details!
If you don’t have good books to read over Thanksgiving break, read the excellent accepted tiny papers at the #Neurips#EvalEval workshop!
So proud of all the cool work that poured in and beyond excited to see everyone in Vancouver!
Le processus de demande de supervision de Mila ouvre le 15 octobre pour la maitrise de recherche et le doctorat, en vue de l'admission à l'automne 2025. Rejoignez notre communauté! Tous les détails ici: https://t.co/Dr0AhVN5pK
Due to the impressive submissions we've been receiving for our Workshop at NeurIPS 2024 (https://t.co/2QVhSBKQy2), we have decided to announce a ✨ SURPRISE DEADLINE EXTENSION ✨ Now you can submit your submissions on OpenReview up till Oct 7th EOD. https://t.co/muEfAIn5d2 (1/2)
We're nearly 2 weeks away from the deadline for Tiny Papers for our workshop on social impact evaluation of genAI.
If you have thoughts, critiques, WIP, or resources on that topic, now's the time to make them a quick 2-pager!
https://t.co/YmiC86U3fF
📢 Call for Tiny Papers!
Submit your 2 pager by Sept 20 on eval perspectives, challenges, validity
Don't miss our #NeurIPS2024 workshop!
more info: https://t.co/G6RB5HH5W4
Announcing NeurIPS Workshop: EvalEval 2024!
🚀 As generative AI rapidly transforms our world, a critical question looms: How do we measure and evaluate its broader societal impacts?
📄 Our recent collaborative paper (https://t.co/nwtU687g1Q) reveals a lack of standardized methods to assess the full range of effects of generative AI on society, culture, and individual lives.
🔍 To bridge this gap, we're excited to announce our NeurIPS 2024 workshop: "Evaluating Evaluations: Examining Best Practices for Measuring Broader Impacts of Generative AI" aka EvalEval 2024!
🌐 Our workshop website is live! Visit https://t.co/jbYSgHyENk to learn more and check out the call for tiny papers!
Key focus areas include:
• Conceptualization and operationalization of AI impact evaluations
• Ethical considerations in assessment methodologies
• Novel approaches for measuring social impact across different AI modalities
🌟 We're thrilled to have secured commitments from several stellar speakers in the field. Stay tuned for the full speaker list announcement coming soon!
This workshop aims to unite experts in evaluation science, AI practitioners, policymakers, and stakeholders. By fostering collaboration, we hope to develop comprehensive evaluation frameworks and policy recommendations for responsible AI development.
Are you passionate about ensuring AI benefits society? Join us in shaping the future of AI evaluation! Follow our page for updates and reach out with any questions or ideas.
See you at NeurIPS!
#EvalEval2024 #NeurIPS2024 #AIEthics #GenerativeAI #ResponsibleAI #AIPolicy