Ravi Tej Akella

Exploring world epsilon greedily
TeslaAI | Past: CMU, @IITRoorkee

[Ravi Tej Akella @ravitej_17]

Misc: Drifting models normalize the RBF kernel (basically softmax) but this is simply comes from the denominator in score fn expansion (∇log p=∇p/p). So they are just using the RBF kernel, effectively speaking.

[Ravi Tej Akella @ravitej_17]

Lots of hype around Kaiming et al’s Drifting Models (SOTA 1-step inference FID, connections to iMLE/MMD/InfoNCE), but the core idea is simple: take real + fake samples, fit RBF-KDEs to both, and optimize reverse KL on the KDE-derived scores. That’s the essence. More details in 🧵

[机器之心 JIQIZHIXIN @jiqizhixin]

New paradigm from Kaiming He's team: Drifting Models!

With this approach, you can generate a perfect image in a single step.

The team trains a "drifting field" that smoothly moves samples toward equilibrium with the real data distribution.

The result? A one-step generator that sets a new SOTA on ImageNet 256x256, beating complex multi-step models.

[Ravi Tej Akella @ravitej_17]

For the same reason, it’s hard to see this extending to other modalities like audio or robot action spaces. Moreover, I think it will be challenging to do RL on this sort of KDE derived likelihoods without some serious reward hacking.

[Ravi Tej Akella @ravitej_17]

@rosinality @GlenBerseth @veds_12 @pcastr I believe a similar vectorized variant for K1 as reward also exists where we compute the analytical KL over the full vocab size rather than just the sample inds.

However, I never saw any open LLM implementation do this. Is this because reduction over vocab size is too expensive?

[Ravi Tej Akella @ravitej_17]

@rosinality @GlenBerseth Basically, we teacher force the sample from pi theta through pi ref and compute the analytical KL over the entire vocab size. This would be a much lower variance gradient estimator than the sample based K3 approximation.

Curious to hear what the authors think @veds_12 @pcastr

[Ravi Tej Akella @ravitej_17]

@dwarkesh_sp @svlevine Can we achieve useful general purpose robotics with just current methods or are we still 1-2 breakthroughs aways?

Also, thoughts on today's near flat policies vs better hierarchical methods for subproblem identification and planning in this space

[Phil Duan @pduan]

Many more talented people who made this possible aren’t pictured here - they are the unsung heroes.

I want to give a special shoutout to the legend @patrick_cho_1 , the mastermind behind a lot of our breakthroughs.

[Ravi Tej Akella @ravitej_17]

@zmkzmkz Cool result! Curious if softpick suffers from numerical instability as we can no longer subtract the max term

[Glen Berseth @GlenBerseth]

Offline RL aims to learn high-value policies from static data. We find that using latent diffusion models to sample from complex multimodal distributions over long horizons enables better credit assignment, leading to improved performance.

More details in the 🧵 below:

[Brandon Trabucco @brandontrabucco]

Can we use Stable Diffusion for data augmentation?

Introducing DA-Fusion: a data augmentation strategy using pretrained diffusion models to semantically modify image attributes. This complements existing data augmentation and improves few-shot learning.

https://t.co/pBTDtjVVqu

[Ravi Tej Akella @ravitej_17]

@cwolferesearch @sgondala2 Is there a way for ChatGPT to separate factually correct output vs hallucinating? I don't think so, since ChatGPT doesn't quantity it's prediction uncertainty.

So simply using RLHF to reduce hallucinations cannot have any affect on hallucinations as far as I can tell.

[Behnam Neyshabur @bneyshabur]

We often prefer collaborating with people we know or those of high status. That makes it very difficult for hardworking and motivated junior researchers to get enough support to flourish.

Is it possible to reduce this barrier?
I'v been running some experiments to find out!

1/6

[Yisong Yue @yisongyue]

We have released a new version of the Caltech OPE Benchmarking Suite (COBS), along with an accompanying evaluation paper (appearing at @NeurIPSConf):

https://t.co/p4DpAwvf7W
https://t.co/mA2HJmnJuo

Joint w/ Cameron Voloshin (1st author), @HoangMinhLe @nanjiang_cs

[Ben Eysenbach @ben_eysenbach]

Unsupervised skill learning methods based on mutual information (e.g., DIAYN) learn a wide range of skills, which are often useful for solving downstream tasks. Are these methods a provably good way to do pretraining? Do you learn skills for all reward-maximizing behaviors?

[Olexandr Isayev 🇺🇦🇺🇸 @olexandr]

log(😅) =💧log(😄)

[Zachary Lipton @zacharylipton]

Heads up to all @purdueCS students interested in ML research. Your new prof @kazizzad is among the most brilliant, energetic, and creative researchers I've ever known and doing exciting work across the theoretical-empirical spectrum. Find a way to work with him.

[Ravi Tej Akella @ravitej_17]

BONUS: Monte-Carlo method, which is widely used for PG estimation, is a degenerate case of Bayesian Quadrature (BQ). In other words, BQ is a more general method that, in practice, offers more accurate PG estimates.

#neurips2020 #ReinforcementLearning

[Ravi Tej Akella @ravitej_17]

How to estimate Policy Gradients more accurately, with the same number of samples and similar computational cost?

Check out our work at the #NeurIPS2020 Deep RL workshop tomorrow.

Paper:https://t.co/43RxFRx7U6
Video:https://t.co/qAr99tXw8Q

@kazizzad @AnimaAnandkumar @yisongyue