James

Ask me about #ReinforcementLearning
#AI research @SonyAI_global
RL for games, robotics, and other real-world applications
Views and tweets are my own.

[James @jmac_ai]

Reinforcement learning in #AI is hard, so I’ve made a website to collect answers I’ve given to common RL questions.

It's named Decisions & Dragons. It’s launching with 8 questions and answers, but I will add to it in the future.

A 🧵to give a preview with the link below.

[Governor Tim Walz @GovTimWalz]

I’ve seen the video.

Don’t believe this propaganda machine.

The state will ensure there is a full, fair, and expeditious investigation to ensure accountability and justice.

[James @jmac_ai]

@seohong_park Link to GT Sophy work we did at Sony AI:
https://t.co/1XbpIxXibE

[James @jmac_ai]

@seohong_park Great post! I'll note that our work on GT is a real-world off-policy RL success. (It's in the game!)

However, I share a similar conclusion. Off-policy Q-learning is brittle. I think MBRL where you learn from off-policy data but optimize on-policy in the model is more promising.

[James @jmac_ai]

@bimald @coolkoon @GaryMarcus I’m not going to tell you LLMs will have no impact on coding. For simple things it can give non-programmers more flexibility.

However, it’s not like chess, because the limitations if English in specifying behavior is an inherent bottleneck, where no such thing exists for chess

[James @jmac_ai]

@bimald @coolkoon @GaryMarcus I think you misunderstand the nature of code. Code isn’t for computers, it’s for people. English is *bad* at specifying clear behavior. In fact, regularly in design discussions, we move to writing bits of code because it’s more clear than words.

[James @jmac_ai]

@kevinroose AI Expert here.

First, a great many experts do not believe in the extreme hype/doom & the loudest hypers/doomers are usually not AI *scientists*.

Second, unlike climate change, the hype/doom is not based on a scientific model. It is speculative rhetoric.

They are not the same

[James @jmac_ai]

I was *not* expecting vocals for this song to go so hard.

Absolutely incredible vocal power and control. Pernelle needs to be much more popular.

[Pernelle. 🦚 Suteki Da Ne @PernelleMusic]

BOW WOW WOW, had the pleasure to try my voice on rock music with @huskybythegeek and I love it!
Happy late PC release to Final Fantasy VII Rebirth 🐶
#FF7Rebirth

[Pernelle. 🦚 Suteki Da Ne @PernelleMusic]

BOW WOW WOW, had the pleasure to try my voice on rock music with @huskybythegeek and I love it!
Happy late PC release to Final Fantasy VII Rebirth 🐶
#FF7Rebirth

[James @jmac_ai]

@chrisprucha @bradneuberg The bigger limiter has always been that you always need to do experiments in the real world to advance, and that means both resources and contending with the speed of reality.

Science is necessary, not a kludge.

[James @jmac_ai]

@SenMastriano Resign. You are unfit for a booster seat.

[James @jmac_ai]

@agrimgupta92 Very impressive. Although it feels like this person is moments away from cutting off their finger and it’s giving me an anxiety attack :p

[James @jmac_ai]

@rao2z @natolambert I think this may be a misunderstanding of the bitter lesson. Rich Sutton is very focused on online learning and not pretraining, to a fault.

[James @jmac_ai]

@Intrinsic29 I don't think I've ever seen a non-doomer say "Some things are impossible, therefore AI cannot be a threat."

It's always been a reaction to magical thinking where a doomer will literally be equating a future AI with a "god."

[James @jmac_ai]

@Intrinsic29 Often, if a non-doomer brings up limits, its because a doomer presented an argument assuming no limits.

Doomers regularly hand wave "of course the AI finds a way to do x because it's superintelligent." You can't automatically conclude that without making magical assumptions.

[James @jmac_ai]

@scrumtuous @clattner_llvm @tsoding @Modular Lot's of reasons, but here's a simple one if you're actually skeptical: Modular's twitter account retweets that account.

[James @jmac_ai]

@scrumtuous @clattner_llvm @tsoding @Modular You know you're responding to Chris Lattner, right? :p

[James @jmac_ai]

@Miles_Brundage I've been all aboard the "we can solve the AI problem" train my entire adult life. But claiming it's not hard shows a lack of respect for the challenging problem we've tackled and is falling victim to hype instead of science.

[James @jmac_ai]

@Miles_Brundage What unwillingness? The timeless pattern is AI researchers thinking it will be solved soon and being wrong every time.

The founders of the field thought they'd solve much of the problems in a few months with a small team.

There is far too much willingness to believe it s easy.

[James @jmac_ai]

@ErbunnNinja @Intrinsic29 An an expert in decision-making agents, you are entirely incorrect.

[James @jmac_ai]

@ErbunnNinja @Intrinsic29 There will be a world of difference. People who use the word god end up confusing themselves precisely because they're assuming the difference.

This is a scientific and technical issue. The word "god" has no value in discourse of it and can -- and does -- confuse matters.