Yao

Researcher/engineer working on LLMs and agents. ex Google Deepmind.

[Peter J. Liu @peterjliu]

https://t.co/Ch1dWZQpH1

[Yao @yaozhaoai]

We're building Compound, an AI Analyst built for finance. It works like a team of analysts reporting to you — researching, delivering insights, and producing work. Start compounding your impact.

[Compound AI @getcompoundai]

Meet Compound - the world’s first AI Analyst for finance you can trust.

AI for spreadsheets and financial analysis is finally here, but most tools are too brittle for real use cases.

Compound is different - built for scale, accuracy, and auditability - so you can 10X your output.
- Upload unlimited number of files to analyze
- Kick off multiple AI Analysts at the same time
- Audit and edit the work output in the browser

Comment for access to the beta.

For more on what Compound can do, see below 🧵

[Peter J. Liu @peterjliu]

One of the top machine learning conferences #ICLR2025 is this week. But there’s 3000+ accepted papers, which is a lot to sift through. Use RadPod to chat with them all and quickly hone in on your interests.

Examples queries:

“find papers with more than one OpenAI-affiliated author”

“find papers that propose alternatives to Transformer architecture in LLM”

“give an overview of all spotlight or oral papers with Yann Lecun as author”

You can even get a link to the OpenReview reviews easily.

[Peter J. Liu @peterjliu]

Recently a huge new batch of files on the JFK Assassination was released by the National Archives as a result of a presidential executive order. A whopping ~80,000 pages of scanned PDFs -- available but not accessible. AI to the rescue!

Except none of the AI apps can handle this type and amount of context ... until now.

We built RadPod AI to enable highly-accurate, deep research on your (possibly huge) data.

[Aran Komatsuzaki @arankomatsuzaki]

Transformer decoder with MoE and efficient attention has been available at tensor2tensor library since 2017

A paper that trained Transformer decoder with MoE, efficient attention and up to 11k context length was released in September 2017 (https://t.co/VRfBBTT5PZ).

[Yao @yaozhaoai]

@Francis_YAO_ @denny_zhou So true, nearly forgot about it, used to spent so much time on it.

[Yao @yaozhaoai]

@GriffinAdams16 @peterjliu Thanks for sharing your paper, super interesting. Same scrutiny on HF data composition is much needed too!

[Yao @yaozhaoai]

Key of learning from feedback is a different signal than supervised fine-tuning: distinguish better/worse seqs vs generate plausible seqs

Evidence: contrastive learning and RL learn from feedback equally well, both much better than fine-tune on only positive feedback.

[Peter J. Liu @peterjliu]

Here is our “slick” RLHF-alternative without RL: https://t.co/D8H7fPnD5Z (SLiC-HF)

TL;DR: Works as well as RLHF, but a lot simpler.

About as easy and efficient as fine-tuning. Much better than simply fine-tuning on good examples.

From great collaborators: @yaozhaoai, @rishabh_joshi4, Tianqi Liu, @khalman_m, @Mohamma78108419, @peterjliu.

[Yao @yaozhaoai]

@zhansheng @peterjliu That would be the probability

[Peter J. Liu @peterjliu]

We are hiring for a full-time researcher/engineer in the Brain (Google Research) team who will focus on text generation research and its applications. A wide variety of backgrounds and experiences will be considered. DM if you're interested or have leads.

[Sam Shleifer @sam_shleifer]

DistilBERT by @SanhEstPasMoi is one of the most popular models on the @huggingface model hub, but there wasn’t a clear equivalent for Seq2Seq models. Now there is! I'm happy to introduce our paper on “Pre-trained Summarization Distillation”: w @srush_nlp https://t.co/xUMxjhNPvs

[Sam Shleifer @sam_shleifer]

Excited to release PEGASUS in @huggingface transformers: 12 new SOTA summarization models: https://t.co/gzc414Gjf6
from Google Brain (@GoogleAI) intern @JingqingZX , and colleagues @yaozhaoai, ,Mohammad Saleh, and Peter Liu (@peterjliu). 👇

[Peter J. Liu @peterjliu]

New SOTA results for abstractive summarization just posted to https://t.co/pLSEE8ems5! We have a new way to pre-train for summarization, and evaluated our PEGASUS model on 12 diverse downstream summarization tasks, achieving SOTA on all, in some cases by a significant margin.