Roman Ring

[email protected]

Research Engineer @DeepMind

Large Scale Deep Learning Is All You Need?

AlphaStar, Gopher, Flamingo, ???

[Roman Ring @Inoryy]

After high school, I ended up working odd jobs, e.g., as a night shift cleaner at a fast-food restaurant. One especially nasty night, I decided that I had to at least try to get more out of my life and began self-studying coding every free minute I had, up to 20 hrs/day.

1/5

[Roman Ring @Inoryy]

I don't think Twitter will go down soon, mostly because I believe the (ex) eng and infra teams did a good job building resilient systems.
But it's unlikely I'll be posting from now on. To anyone stumbling on my profile seeking advice on AI, feel free to reach out via email.

[Roman Ring @Inoryy]

Truly visionary business moves this week. I wonder if more people will finally realize the emperor has no clothes.

[Elon Musk @elonmusk]

@mrddmia Thank you.

A thermonuclear name & shame is exactly what will happen if this continues.

[Roman Ring @Inoryy]

@_aidan_clark_ I think Mastodon existed before all of the web3 hype. Thinking of joining it as well!

[Roman Ring @Inoryy]

@0xWren I suppose the typical argument is that the runaround to make the ops vectorized isn't natural and the compiler should be able to do this by itself from a simple loop. It forces users to come up with clever tricks to make numpy work instead of just solving their problem.

[Roman Ring @Inoryy]

A bit late to the party but this subject comes up fairly often and I have some thoughts as I spend most of my time at the boundary of Python (JAX) and C++ (XLA), and in the past have been a (very minor) contributor to DeepMind's effort to speed up Python: https://t.co/EFNkdvUO6o.

[Talia Ringer 🕊🪬 @TaliaRinger]

It's really weird how much machine learning is done in Python, because numerical/array computing should in theory be like 50x easier in a compiled language where you can build abstractions and leave the optimization to the compiler, no?

[Roman Ring @Inoryy]

The unsatisfying reality is that the day-to-day issues researchers run into are usually either at the ML framework level or at the ML compiler level, and neither would be fully addressed by a language with a cleaner API for array computing.

[Roman Ring @Inoryy]

However, those switches have led to access to new major features or a significant boost in productivity and I just don't see opportunities to gain either in alternatives today. Python isn't great but all it has to be is *good enough* at serving as a DSL for ML compilers.

[Roman Ring @Inoryy]

@nathanbenaich Plaza accord 2.0 is inevitable at this rate...

[Roman Ring @Inoryy]

I feel like that blinking white guy meme coming back from a period of minimal social media to a Twitter full of stable diffusion / midjourney generated artwork. Genuinely in awe of the pace of progress.

[Roman Ring @Inoryy]

@dotstepan Not necessarily. AGI doesn't imply superhuman performance at all tasks, especially those involving embodiment and real-time interaction in complex environments.

[Roman Ring @Inoryy]

Hot? take: we might reach (some form of) AGI before we reach SAE J3016 L5 driving automation.

[Roman Ring @Inoryy]

@drjwrae It was a pleasure and a privilege working with you Jack, hope our paths cross again someday :)

[Roman Ring @Inoryy]

@nathanbenaich @londonai It was a great event, thanks for organising!

[Roman Ring @Inoryy]

@mark_riedl Nobody starts off at the deep end, people come to 4chan for the edgy jokes and are indoctrinated over time by bouncing increasingly toxic content off each other. Literally by definition of echo chamber it's clear that deploying AI into such ecosystem has a net negative effect.

[Roman Ring @Inoryy]

@_joaogui1 Genuinely baffled how anyone can argue there's no harm in doing this.

[Roman Ring @Inoryy]

@ykilcher It self-evidently contributed to 4chan's echo chamber, amplifying and solidifying their opinions. It's not impossible that gpt-4chan pushed somebody over the edge in their worldview. Whether a specially tuned LM can do it more efficiently than a regexp is a weird defense to make.

[Roman Ring @Inoryy]

@zoombapup I think it depends a lot on the lab and the team. Some consolidate all of their efforts in a specific narrow direction (not necessarily a bad thing!), while others have more leeway to explore various options. This applies to both research and engineering sides of things.

[Roman Ring @Inoryy]

People underestimate how much AI research is bottlenecked by engineering.

[Roman Ring @Inoryy]

@_arohan_ Interesting! Can you test it at larger scale, e.g. 10B?