Nathan Lichtlé

PhD @UCBerkeley (@berkeley_ai)

[Nathan Lichtlé @nathanlichtle]

We ran 5,600 hyperparameter sweeps to compare RL algorithms on hidden-information games with billions of states.

In our benchmark, we found that properly tuned policy gradient methods, such as PPO, performed the best.

Paper: https://t.co/ADYIrqss8Z

[Samuel Sokota @ssokota]

Model-free deep RL algorithms like NFSP, PSRO, ESCHER, & R-NaD are tailor-made for games with hidden information (e.g. poker).

We performed the largest-ever comparison of these algorithms.

We find that they do not outperform generic policy gradient methods, such as PPO. 1/N

[Berkeley AI Research @berkeley_ai]

We could not be prouder of this years BAIR PhD graduates! Follow their journey to see where they go next @baifeng_shi @sea_snell @databoydg @enfleisig @graceluo_ @zhuhl98 @HaozhiQ @jdzamfi @LianJiachen @joshminwookang @junhaobearxiong @kaylolittlejohn @KentKChang @kvablack @KunheYang @lisabdunlap @LongTonyLian @maulik__bhatt @michaelpsenka @nathanlichtle @neerjathakkar @nikita_mehandru @NiklasLauffer @qiyang_li @vbenara05 @vonekels @xiuyu_l @Yiheng_Li_Cal @ZheFu1 Sampada Deglurkar Wei-Jer Chang Yichen Xie Yigit Erginbas

https://t.co/6axNQ64vDg

[Berkeley AI Research @berkeley_ai]

New Blog Post:

Scaling Up Reinforcement Learning for Traffic Smoothing: A 100-AV Highway Deployment https://t.co/bYijROnsOb

[Eugene Vinitsky 🦋 @EugeneVinitsky]

This paper has everything: large-scale empirical testing, new benchmarks, and subsequent empirical recommendations that we think might make solving imperfect information games a little easier

[Nathan Lichtlé @nathanlichtle]

Tic-Tac-Toe... but the opponent's moves are hidden.

Can you outsmart our top RL agents? Play here: https://t.co/uVtva67Y08

[Nathan Lichtl�� @nathanlichtle]

We ran 5,600 hyperparameter sweeps to compare RL algorithms on hidden-information games with billions of states.

In our benchmark, we found that properly tuned policy gradient methods, such as PPO, performed the best.

Paper: https://t.co/ADYIrqss8Z

[Joseph Suarez 🐡 @jsuarez]

https://t.co/KHYZt70Lgp

[Nathan Lichtlé @nathanlichtle]

Coded in C with the amazing Raylib! Try it out at https://t.co/jIGs4I3lxJ

[Nathan Lichtlé @nathanlichtle]

Fast turn-based tactical combat env I've been working on -- RL training incoming!

[Nathan Lichtlé @nathanlichtle]

@jsuarez Nice! It looks easier to play now

[Amaury Hayat @Amaury_Hayat]

Can language models be trained to find solutions to as yet unsolved mathematical problems? The answer is yes! Check our new article 🙂 1/n

[Nathan Lichtlé @nathanlichtle]

@EugeneVinitsky Going for a multi-lane version of the trajectory simulator. Maybe a fast highway-env?

[Nathan Lichtlé @nathanlichtle]

Another driving sim??

[Nathan Lichtlé @nathanlichtle]

@BonesaiDev This looks so cool

[Eugene Vinitsky 🦋 @EugeneVinitsky]

We’re open-sourcing and arxiving GPUDrive, a GPU-accelerated 2.5D multi-agent driving simulator that runs at over a million FPS. Hundreds of scenes on one GPU means scalable multi-agent planning

[Nathan Lichtlé @nathanlichtle]

@JamesLucasIT @threadreaderapp unroll

[Nathan Lichtlé @nathanlichtle]

Excited to share our work about the MVT, a large-scale highway field test for which we trained autonomous vehicles to smooth out traffic flow using deep reinforcement learning, then deployed our AI controllers onto 100 vehicles in busy morning traffic. Read more below! (1/n)

[Nathan Lichtlé @nathanlichtle]

My amazing co-first authors: @JangKathy @EugeneVinitsky @aditshah00

Read more about our adventures in the RL team:

📄 Paper 1 (Controller design in simulation) - https://t.co/lyUFJriCfQ
📄 Paper 2 (100 AV field test) - https://t.co/JCEFldslwd

[Nathan Lichtlé @nathanlichtle]

During training, we placed an agent behind a real-world trajectory and tasked it to optimize the energy consumption of everyone driving behind. The result: significant smoothing and fuel savings (>15%) on stop-and-go waves with only 4% of AVs! (Special thanks to our beloved PPO!)

[Nathan Lichtlé @nathanlichtle]

Extracting the data from dozens of overhead cameras from a week of experiments is a complex computer vision endeavor. Preliminary results suggest a trend of reduced fuel consumption behind AVs. Many more details and nuances available in the paper!

[Nathan Lichtlé @nathanlichtle]

Optimizing for both energy savings and movement is a careful balance! Cars shouldn’t stop (which is energy-optimal) or open excessive gaps. We introduced dynamic gap thresholds, giving AVs leeway to freely optimize for energy while being constrained to safe and reasonable gaps.