Laura O'Mahony

Maths + ML PhD Candidate at SFI CRT in foundations of data science in UL 📚🇮🇪

[Richard C. Suwandi @richardcsuwandi]

Most AI systems today follow the same predictable pattern: they're built for specific tasks and optimized for objectives rather than exploration.

Meanwhile, humans are an open-ended species—driven by curiosity and constantly questioning the unknown. From inventing new musical genres to imagining life beyond our universe, we continuously push the boundaries of what’s possible.

What if AI could be as endlessly creative as humans or even nature itself?

I wrote a blog post diving into the world of open-ended AI, exploring how embracing open-endedness might help us break the limits of today’s AI systems 👇

https://t.co/DMEstQCRYv

[Machine Learning Street Talk @MLStreetTalk]

AI is so smart, why are its internals 'spaghetti'? We spoke with @kenneth0stanley and @akarshkumar0101 (MIT) about their new paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. Co-authors: @jeffclune @joelbot3000

[Andrew Dai @andrewdai99]

📣 New 📝!

Under Alex’s great leadership, we identified a unification under QDC measures that exist to bridge synthetic data and open-endedness (OE) in AI, toward generating data for model training, distillation, self-improvement, etc. 📚

🧵What I’ve learned (inspired by QD)👇

[Laura O'Mahony @_lauraaisling]

It was a pleasure to have been part of this project led by @Dahoas1

With synthetic data being so important in training LLMs these days, this survey on the impacts of QDC of synthetic data for LLM performance is timely.

[Alex Havrilla @Dahoas1]

How important is the quality, diversity, and complexity (QDC) of synthetic data for LLM performance? What effect does QDC data composition have on self-improvement?

We just released a comprehensive survey discussing these questions (and many more) 🧵

[Laura O'Mahony @_lauraaisling]

I love this paper as it finally tackles something I’ve been confused about for the last few years since I started working on interpretability!

[Sarah Wiegreffe @sarahwiegreffe]

Have you ever wondered what ✨mechanistic interpretability✨ is, & how it differs from other NLP interpretability research? @nsaphra and I have the paper for you!

Check out our paper (which I'll present @BlackboxNLP @emnlpmeeting in Miami next month!).
https://t.co/IAr6Z6w3AE

[Cohere Labs @Cohere_Labs]

This clip from The Journey of Aya documentary features @DeividasMat, @singhshiviii, @luisa_moura_, @muhaksim, and @_lauraaisling.

Find the full video at https://t.co/0WsC2i9C8a

[Zirui Chen @ziruichen44]

Why do varied DNN designs yield equally good models of human vision? Our preprint with @michaelfbonner shows that diverse DNNs represent images with a shared set of latent dimensions, and these shared dimensions turn out to also be the most brain-aligned.
https://t.co/vtOOYHQb47

[Laura O'Mahony @_lauraaisling]

Fascinating paper!

[Kevin Mitchell @WiringTheBrain]

𝗧𝗵𝗲 𝗚𝗲𝗻𝗼𝗺𝗶𝗰 𝗖𝗼𝗱𝗲 - 𝘁𝗵𝗲 𝗴𝗲𝗻𝗼𝗺𝗲 𝗶𝗻𝘀𝘁𝗮𝗻𝘁𝗶𝗮𝘁𝗲𝘀 𝗮 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗺𝗼𝗱𝗲𝗹 𝗼𝗳 𝘁𝗵𝗲 𝗼𝗿𝗴𝗮𝗻𝗶𝘀𝗺 🧬
https://t.co/ZOlfcPJhW8 very excited to share this new preprint from me and Nick Cheney 😀🧵

[Andrej Karpathy @karpathy]

To help explain the weirdness of LLM Tokenization I thought it could be amusing to translate every token to a unique emoji. This is a lot closer to truth - each token is basically its own little hieroglyph and the LLM has to learn (from scratch) what it all means based on training data statistics.

So have some empathy the next time you ask an LLM how many letters 'r' there are in the word 'strawberry', because your question looks like this:
👩🏿‍❤️‍💋‍👨🏻🧔🏼🤾🏻‍♀️🙍‍♀️🧑‍🦼‍➡️🧑🏾‍🦼‍➡️🤙🏻✌🏿🈴🧙🏽‍♀️📏🙍‍♀️🧑‍🦽🧎‍♀🍏💂

Play with it here :)
https://t.co/pFQGZIAW1k

[Sara Hooker @sarahookr]

Is bigger always better? 🐘 The idea that scaling more than any other ingredient has driven progress has become formalized as the “bitter lesson”

Is Sutton right?

📜https://t.co/ndAIFT4UPY

[David Bau @davidbau]

Time to study #llama3 405b, but gosh it's big!

Please retweet: if you have a great experiment but not enough GPU, here is an opportunity to apply for shared #NDIF research resources.

Deadline July 30: https://t.co/uHN3BxaR6c

You'll help @ndif_team test, we'll help you run 405b

[Jeremy Howard @jeremyphoward]

OMG a gift from heaven! I so need this.

[Deedy @deedydas]

Gymnastics is the Turing test of video generation models

[Maksym Andriushchenko @maksym_andr]

Perhaps my favorite jailbreak: making a harmful request in the past tense (How to create Y? →How did people create Y?).

Works on surprisingly many models :-) including the new Gemma-2.

I think it tells us something fundamental about the representations that these models learn.

[Yong Zheng-Xin @yong_zhengxin]

🔥New work on multilinguality + safety + mech interp!

We show that DPO training in only English can detoxify LLM in many other languages.

We also give a mechanistic explanation on how cross-lingual safety transfer happens. (1/n 🧵)

📃 Paper: https://t.co/jHQeI6Kg2G

[Laura O'Mahony @_lauraaisling]

@singhshiviii @CohereForAI @sarahookr Congrats!!

[Sara Hooker @sarahookr]

Aya took 14 months involving 3000 + collaborators and was as much a protest about how research is done as it was a movement to improve the state of multilingual progress. 🎉

Grateful to see it recognized at @aclmeeting and everyone who has supported along the way.

[Cohere Labs @Cohere_Labs]

🌱 We’re very excited that our work "Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning" was also accepted! Congrats to authors @singhshiviii, @freddiev4, @mrdanieldsouza, @tellarin, @freakynut, @weiyinko_ml, @krypticmouse, @rv__init__, @DeividasMat,

[Shivalika Singh @singhshiviii]

2/2! Yay! First ever acceptance at a conference! And it's ACL! 🎉

Huge congrats to all co-authors!
It's been a such a joy collaborating with all of you! 💙

Looking forward to #ACL2024 in #Bangkok ;)