Hamid Maei

AI researcher & engineer @ Netflix. Developing AI. CS/RL PhD with Rich Sutton—developed foundational gradient-TD (TDC, GTD, GQ) algos in RL

[Hamid Maei @HamidMaei]

@mehdiy_fa Also they need to define a margin for error. Like a couple of centimeters or so— this decision was total nonsense

[Hamid Maei @HamidMaei]

To my experience AI coding tools are useful for repetitive, mundane tasks—workflows, orchestration, or common algorithms or modules. But for novel work, or when you care about a specific code style or architecture, they often get in the way. Sometimes it’s faster to write it from scratch by hand.

[Hamid Maei @HamidMaei]

I was wondering, by reading the TLDR description, how is this different from temporal-difference learning? @RichardSSutton’s original TD paper already provides similar idea and algorithm in terms of learning — it updates the current prediction based on the next prediction(bootstraps) and ads a transition value to it ( which could be reward or any feature—the paper doesn’t mention reward or RL.) not sure if I’m missing some details.

[Hamid Maei @HamidMaei]

@_wenlixiao this is very interesting. can you elaborate more on actor-critic method? is it pre-trained and how LLM modifies it?

[Hamid Maei @HamidMaei]

AI was built through decades of publicly funded research, universities, students, engineers, taxpayers, and governments, including Canada’s major role. Companies later scaled it for profit, which is great and turned it into some amazing products.

However, some visible few now are shaping the public debate about AI. Some warn young people that jobs will disappear; others dismiss AI as empty hype. I hope both extremes are wrong. But when elites make sweeping claims, they should carry more responsibility. If they are wrong, they will not pay the price — young people who believed them will.

This is an interesting clip to watch: https://t.co/Xb8ZvWRiBu

[Hamid Maei @HamidMaei]

Great and insightful interview—probably one of the best interviews I’ve watched in a long time.

[Richard Sutton @RichardSSutton]

Michael Jordan brings a strong and thoughtful perspective on AI:
https://t.co/0EVjoDW5Hd

[Hamid Maei @HamidMaei]

The AI interviews are totally broken. As someone who did my PhD with Rich Sutton in RL almost 2 decades ago and has done both research and engineering in Bay Area tech, I think interviews should be based on understanding and engineering mindset rather than a bunch of buzzwords and hacks that will fade away by next year.

[Hamid Maei @HamidMaei]

This isn’t unique to CVPR—it’s everywhere. Perhaps because most of academia, though not all, has spent three decades chasing solutions in ML ( Bayesians vs frequentists and vs Connectionism!) rather than actual problems! ( I like this slogan: approximate solutions and not problems)

[Hamid Maei @HamidMaei]

@yacineMTB Pretty good at python since it is super constrained and high level but I can imagine in lower level languages such as C or even worse assembly

[Hamid Maei @HamidMaei]

Fair point, but I don’t think most people go to those top elite $300k schools to acquire skills.

Re education itself: we need to provide an environment where kids can learn, discover new things, and learn how to succeed through their own experience, exploration, and trial and error.

There are already many online resources where kids can acquire top-notch skills and knowledge, but they need to know how to do research, find those resources, and collaborators and stay motivated. LLMs still do not have the ability of discover new and novel things through experience as humans can do.

[Jitendra MALIK @JitendraMalikCV]

I want to offer some unsolicited advice to computer vision researchers jumping into robotics. Don't focus too much on VLMs, VLAs etc. That's fine, but the real action is at the sensorimotor level. Most of the open problems in robotics are in manipulation, which is about hand-object interaction, and contacts and forces are central. Proprioception and tactile sensing are as important as vision. Don't get seduced by cherry-picked demos. You can't do robotics without doing robotics.

[Hamid Maei @HamidMaei]

@pmddomingos Methods whose computational costs scales linearly in terms of number of learning parameters — both in terms of memory and per-time-step computation — would ultimately win. Transformers can use abstractions over the coarse of tokens to overcome quadratic costs. That is TBD

[Daniel Jeffries @Dan_Jeffries1]

AI will become our interface to the world.

It will sit higher in the stack than the OS. It will collapse current SaaS layers, chat, communications, apps, app creation, into a single new kind of interface that doesn't exist yet.

It's got to be open. It's got to be a cypherpunk solution that makes privacy and security the number one priority.

If a closed source solution wins this layer, it's a disaster for the world. Especially if it's built by a single company with a single closed source model.

Why?

Because what we share with AI will be more intimate than anything we've ever shared with a machine.

It will be our friend, our sounding board, our advisor. It will know our business ideas before we've told anyone. Our medical issues. Our financial picture. We'll talk about the fight we had with our partner. About feeling lost or depressed. Our kids will talk to it about problems at school, about bullying, about heartbreak, things they won't tell us.

It will know us more intimately than we know ourselves.

Right now the world runs on a surveillance economy. We traded free stuff for apps that peer deeply into our lives.

If we replicate that model in the AI era, it's not just surveillance economy 2.0. It's surveillance economy squared. Social scoring. Legal conversations you thought were privileged showing up in court. Random people making $2 bucks an hour on the backend from God knows where reading the most intimate details of your life. Every insecurity, every fear, every half-formed thought you whispered to your AI buddy at 2 AM, sitting in a database somewhere, searchable.

This interface might eventually become an OS, like the OS in Her. But it will take a long time to reach down to that layer and it will require a fundamentally new kind of operating system design. You can't retrofit this onto Linux or Windows or Android or iOS. It's a new layer of the stack entirely.

And whoever controls that layer controls our lives.

We've got to make sure it's us. Not them.

[Hamid Maei @HamidMaei]

I have seen this mistake over and over again: people think RL is a solution, whereas RL is really defined as a problem. So abandoning RL in favor of method X mentioned in this talk does not make much sense to me.

I would instead argue that the current solutions to RL are still crude and not yet practical in real experiential world, largely because they lack the right abstractions — both for representing past experience of agent, and for temporal abstractions over future coarse of actions.

In fact, there is a significant overlap between what this talk advocates for and what much of the RL community already believes is necessary for scalable and practical intelligence.

[Kosta Derpanis (sabbatical in Zurich) @CSProfKGD]

.@ylecun making his usual impassioned case for LLMs and RL 🍒 for world models ... just kidding 🤓

Thanks Yann for the thought-provoking talk.

[Hamid Maei @HamidMaei]

This is the best description I’ve ever heard, especially given how much overlap there is between the terms engineer and scientist in Silicon Valley tech companies

[Yann LeCun @ylecun]

Major difference in my mind:
- an engineer, given a problem, invents and tries multiple solutions and stops when the solution is good enough. The goal is product innovation and shipping.
- a scientist asks new questions, proposes various new solutions, compares them (sometimes with old ones), and writes about it. The methodology must be sound or else peers will sneer. The goal is scientific breakthroughs and technological progress.

Both can be called "researchers". Many people can do both: these are activities, not identities.

Importantly, most product innovations are built on scientific breakthroughs and technological innovations that happened 2, 5, 10, or 20 years earlier.

[Richard Sutton @RichardSSutton]

The bitter lesson in 26 words:

Don’t be distracted by human knowledge, as AI has been historically.
Instead focus on methods for creating knowledge that scale with computation, like search and learning.

[Hamid Maei @HamidMaei]

@chesterzelaya Cool. Are they being trained though any learning mechanism for control?

[Hamid Maei @HamidMaei]

@GaryMarcus I find it interesting that many commenters are blaming the messenger and saying you should have had backups. As soon as you give write access, which you need to in this agentic workflow, all other kinds of problems can pop up even if you have back ups.