Janice Lan

Research Engineer at FAIR, previously Resident at Uber AI Labs

[Tianhao Wu @WthThao]

🚀 We've found that optimizing a model's internal thought process yields gains not just in reasoning, planning, and math, but across all instruction-following tasks!

🔍 How it works:
1️⃣ We prompt the model to think (as an initialization)
2️⃣ Then, we use RL to optimize the thought process based only on the reward of the final answer.

⚡️ We don't teach the model how to think— instead, we incentivize it to refine its thinking process using answer rewards.

💡 This approach beats GPT-4 & Llama 3 (70B) on AlpacaEval and delivers huge gains on ArenaHard with an 8B model!

📄 Read more: https://t.co/sidklPjjYR

[Jason Weston @jaseweston]

🚨New work: Thinking LLMs!🚨
- Introduces Thought Preference Optimization (TPO)
- Trains LLMs to think & respond for *all* instruction following tasks, not just math
-Gives gains on AlpacaEval (beats GPT-4 & Llama3-70b) & ArenaHard with an 8B model
https://t.co/4MB0D79zYH
🧵1/4

[AI at Meta @AIatMeta]

Today we're releasing the Open Catalyst Demo to the public — this new service will allow researchers to accelerate work in material sciences by enabling them to simulate the reactivity of catalyst materials ~1000x faster than existing computational methods using AI.

Demo ⬇️

[Saro Passaro @pas_saro]

Are you always complaining about the high computational cost of SO(3)-equivariant networks?

📣Excited to present "Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs" or eSCN (in short) at ICML!

Paper: https://t.co/gvCoD6YyBo
Code: https://t.co/ArhqsuYtYa

1/6

[Brandon Wood @bwood_m]

Excited to present Spherical Channels Network (SCN) at #NeurIPS2022 today!

SCN is a GNN for modeling atomic energies and forces, which demonstrates state-of-the-art performance on @OpenCatalyst. The work was led by Larry Zitnick.

Poster 117 in Hall J at 11am-1pm CT

🧵 1/5

[AI at Meta @AIatMeta]

Today, we’re announcing a new data set focused on oxide catalysts for the Oxygen Evolution Reaction (OER), a critical chemical reaction used in green hydrogen fuel production via wind and solar energy. https://t.co/DO63cXH1WP

[ML Collective @ml_collective]

What's next after open-source and open-access? We believe it's *open collaboration*. The field is fairer, more inviting, diverse and inclusive only when we make mentorship and collaborators accessible. But how? We are discussing it on Tue at #NeurIPS2020: https://t.co/XQ7AE4po3D

[Janice Lan @lanjanice]

@savvyRL @jhung0 @jasonyo @IHaveSweaters @oh_that_hat @katherine1ee trash can with its lid open?

[ML Collective @ml_collective]

ICLR 2021 deadline is 10 days away! Working on a draft but short on labmates or peers to proofread? ML Collective can help: send us a draft ([email protected]) between now and Oct 2 and one of our volunteer readers will read+comment. Undersupported researchers... (1/2)

[Mitchell Wortsman @Mitchnw]

sharing Supermasks in Superposition (SupSup):

A model that sequentially learns thousands of tasks with negligible forgetting---even without access to task identity information.

arxiv: https://t.co/9eFj4TzamK
code: https://t.co/DvMSkDMwSO
blog: https://t.co/sEW0izlkiF

(1/8)

[Janice Lan @lanjanice]

@oh_that_hat @stanislavfort Similarly to aphantasia, some people don't have inner monologues (didn't find a term for it, but see https://t.co/nefu2dBssG). It's definitely possible to have thoughts without words, I also do that sometimes and find it faster/easier

[Janice Lan @lanjanice]

@iraphas13 @KerenGu @ACAIWorkshop +1 I don't play AC but still want to see everyone's islands... also lol growing plants both virtually and in real life?

[Janice Lan @lanjanice]

@w4nderlus7 good idea, I have indeed forgotten about bookmarks for years...

[Janice Lan @lanjanice]

What a day of virtual conferencing looks like: too many papers that I want to check out, but since no one is taking down physical posters I guess I'll just leave my bajillion tabs open... (I can't be the only one who does this, right?) #ICLR2020

[Rosanne Liu @savvyRL]

Poster session againnnnn! Come have a drink w/ us and chat about #PPLM at #ICLR2020

[Matthew Leavitt @leavittron]

Class selectivity is often used to interpret the function of individual neurons. @arimorcos and I investigated whether it’s actually necessary and/or sufficient for deep networks to function properly. Spoiler: it’s mostly neither. https://t.co/XTy7XIRuIf (1/10)

[Jonathan Frankle @jefrankle]

@davidjschwab @arimorcos and I have a new paper on BatchNorm. It's not exactly a typical BatchNorm paper: we study the accuracy when freezing all weights at random init and "Training BatchNorm and Only BatchNorm." How did this happen? It's a funny story... https://t.co/P7xAzCd6uL

[Sumanth Dathathri @sdathath]

Plug and play language models accepted to #ICLR2020. Super excited about this work! Thanks to the amazing @jasonyo and @savvyRL for mentoring and hosting me, and @AndreaMadotto for NLP wisdom! Thanks to @lanjanice @jhung0 @IHaveSweaters @w4nderlus7 for invaluable contributions!

[Sumanth Dathathri @sdathath]

@jasonyo @savvyRL @AndreaMadotto @lanjanice @jhung0 @IHaveSweaters @w4nderlus7 @_KarenHao wrote an amazing short excerpt about the paper! Check the article here: https://t.co/pSmH2C1BNR

[ML-Bioinfo-CEITEC @MCeitec]

Interesting research paper introducing Loss Change Allocation from @UberAILabs. LCA allows identifying good and bad parameters affecting the model training process.

https://t.co/5UNJCK8t9m

@lanjanice, @savvyRL, @oh_that_hat, and @jasonyo

#NeuralNetworks #NeurIPS2019