Can Balioglu

Research Engineer at FAIR. Previously PyTorch at Meta.

[Can Balioglu @cbalioglu]

@giffmana @SunnySanyal9 @soldni @mechanicaldirk @epwalsh @tyleraromero @AkshitaB93 fairseq2 as well ;) It was used to pretrain Omnilingual models (7b+) as well as some internal derivatives of llama 2/3 back in the day. Unfortunately it is mostly in maintenance mode nowadays. We still have teams using it internally which might likely get OSS’d at some point.

[Jason Weston @jaseweston]

🌉 Bridging Offline & Online RL for LLMs 🌉
📝: https://t.co/tkK5pi1aGP
New paper shows on verifiable & non-verifiable tasks:
- Online DPO & GRPO give similar performance.
- Semi-online (iterative) DPO with sync every s steps (more efficient!) works very well also.
- Offline DPO is way behind.
- Combining verifiable + non-verifiable works! Cross-transfer gains.
- Recipes for how to make this work.
🧵1/4

[Juan Pino @juanmiguelpino]

We're introducing Seamless Interaction, a research project dedicated to modeling interpersonal dynamics, with potential applications to Avatars and Virtual Agents. We built a family of audiovisual motion models on the Seamless Interaction Dataset, a first-of-its-kind large-scale in-person interaction dataset that we are releasing to the community. Congratulations to the Team!

Download on github: https://t.co/DCOJuFy4x6
Download on @huggingface : https://t.co/5LuxaykMWU
Blog: https://t.co/Z64GK0XZYV
Website: https://t.co/U7DEoBiIjw
Data explorer demo: https://t.co/jSr4gyeENk
Paper (will come soon on arxiv!): https://t.co/yLrCeBEjVg

[fairseq2 @fairseq2]

🚀 Big news for LLM researchers! #fairseq2 now has native support in @vllm_project. Deploy your fine-tuned language models with #vLLM in just one command for lightning-fast performance. Ready to accelerate your research like in @AIatMeta ? Check this out: https://t.co/mPVJaDRYmb

[fairseq2 @fairseq2]

🖼️ A gallery of open-source projects and papers powered by #fairseq2! 🚀

Seamless Communication and Large Concept Models are 2 vivid examples that showcase the potential of what we are building.

More exciting @metaai research built on fairseq2 is on the way!

[Can Balioglu @cbalioglu]

We also have a pretty packed roadmap for 2025 and the list of our supported models might give you a clue about what is coming soon:
https://t.co/ZqhrE2XTL9

[Jason Weston @jaseweston]

Today we are releasing code, models & data from the Self-Taught Evaluator paper, a method to train LLM judges with synthetic preference data.

Better model trained with DPO: https://t.co/oGW56tGXNR

Synthetic preference data: https://t.co/ixz4kSlVgg

Code for training and inference: https://t.co/3NQtG6RY9y

Since we released the paper detailing the method to generate synthetic data, the community has also used this approach to build improved RMs (https://t.co/4sLVBwuKxk).

Our DPO model is a strong LLM judge on RewardBench, despite not using any human annotation in training data creation. We are releasing the training data, training and evaluation code to reproduce our results.

The model is also available as an evaluator on the AlpacaEval leaderboard, as one of the top-ranked evaluators in terms of human agreement rate: https://t.co/KJCrL5Ssid

You can also read the original paper here: https://t.co/aFpnt56TEk

[Can Balioglu @cbalioglu]

@TacoCohen Welcome to FAIR!

[Can Balioglu @cbalioglu]

@antirez @gall0ws Check out https://t.co/x2BaTwt9xN

Our GitHub repo also has a ggml implementation for low resource devices (i.e. smartphones)

[vicki @vboykis]

LLMs are so weird because one side is people with five PhDs who have been studying neuron activations for the past three decades and on the other side is someone called leetm5n with an anime avatar just casually releasing increasingly better performing fine tunes of mistral

[AI at Meta @AIatMeta]

SeamlessExpressive, a new AI translation model by research teams at Meta, enables high-quality speech translation that maintains the speaker's vocal style, tone and unique expressions in translated outputs.

Try the demo with your own voice ➡️ https://t.co/Mwr6oatD4i

[AI at Meta @AIatMeta]

Yesterday we introduced SeamlessExpressive — a new model that preserves unique vocal styles & expression for speech translation, built on our SeamlessM4T v2 foundation model.

More details on the family of Seamless Communication models ➡️
https://t.co/pPgkrTvryg

[Yann LeCun @ylecun]

Seamless: speech to text, text to speech, text to text, and speech to speech, transcription and translation in 100 languages.
From FAIR.

[AI at Meta @AIatMeta]

Today we're sharing the next milestone in our Seamless Communication research — a new family of AI translation models that preserve expression and deliver near-real time streaming translations.

More on this new work ➡️ https://t.co/KNZCEEPk9v

More on the individual models 🧵

[Can Balioglu @cbalioglu]

@NamanGoyal21 Glad that the sessions are over Naman! Wish you quick recovery!

[Maha Elbayad @melbayad]

Excited to share our work on SeamlessM4T. An all-in-one, Massively Multilingual and Multimodal Machine Translation model.
This 🧵 discusses some of the contributions of SeamlessM4T

[Can Balioglu @cbalioglu]

@marksaroufim @rvarm1 Very soon :)

[Can Balioglu @cbalioglu]

[Addendum 7/7] Check out our repo (https://t.co/a6oAe4rlLw) to learn more about Fake tensors and Deferred Module Initialization!

[Can Balioglu @cbalioglu]

[1/2] We recently soft-launched our new package PyTorch Distributed Experimental (a.k.a. torchdistX) https://t.co/a6oAe4rlLw. One nice little feature we have is Fake tensors which act like regular tensors but don’t allocate any storage https://t.co/QiZBi710xg.

[Can Balioglu @cbalioglu]

[Addendum 6/7] Here I take the same Blenderbot model and initialize only its encoder: