Akash Srivastava

Director, Core AI, IBM. Chief Architect . Founder, Red Hat AI Innovation Team. PI @MITIBMLab. ❤️ Density Ratios.

[Isha Puri @ishapuri101]

come check out poster #5518 at NeurIPS morning session today to learn about how you can encourage diversity / prevent early-pruning during inference-time scaling and boost the performance of any model without additional training!

[Akash Srivastava @variational_i]

@annadgoldie @Azaliamirh @RicursiveAI Congrats @annadgoldie and @Azaliamirh — very exciting!

[Akash Srivastava @variational_i]

What does it take to scale AI beyond the lab?

At #RedHatSummit, @ishapuri101 and I spoke with Red Hat CEO Matt Hicks & CTO Chris Wright on inference-time scaling, open infra (LLMD), and making AI affordable for enterprise.

🎧 https://t.co/HDtJcEFF28
#NoMathAI @RedHat_AI

[Akash Srivastava @variational_i]

🚀 How is generative AI transforming the way we design cars, planes, and entire systems?

In Ep 2 of No Math AI, @ishapuri101 and I chat with Dr. @_faezahmed (@MIT DeCoDE Lab) about how AI boosts creativity, cuts design time, and works with engineers—not against them.

[Red Hat AI @RedHat_AI]

How is generative AI reshaping engineering design?

In Episode 2 of No Math AI, hosts Dr. Akash Srivastava (@variational_i) and MIT PhD student Isha Puri (@ishapuri101) sit down with Dr. Faez Ahmed (@_faezahmed) from MIT DeCoDE Lab to explore just that. 👇

[Akash Srivastava @variational_i]

SQuat: KV-Cache for making reasoning models go 🚀
📄paper: https://t.co/SCYVEz8YKL
💻 code: https://t.co/ZLVKIAEgmc
From my awesome collaborators @RedHat_AI

[Hao Wang @HW_HaoWang]

[1/x] 🚀 We're excited to share our latest work on improving inference-time efficiency for LLMs through KV cache quantization---a key step toward making long-context reasoning more scalable and memory-efficient.

[Red Hat AI @RedHat_AI]

Excited to share our preliminary work on customizing reasoning models using Red Hat AI Innovation’s Synthetic Data Generation (SDG) package!

📄 Turn your documents into training data for LLMs.

🧵👇

[Isha Puri @ishapuri101]

had a great time giving a talk about probabilistic inference scaling and the power of small models at the IBM Research ML Seminar Series - the best talks end with tons of questions, and it was great to see everyone so engaged : ) https://t.co/zr09shHGT7

[Akash Srivastava @variational_i]

Come along and help us build reasoning in small LLMs

[Kai Xu @xukai92]

🚀 Exploring LLM reasoning—live!
We, the @RedHat AI Innovation Team, are working on reproducing R1-like reasoning in small LLMs without distilling R1 or its derivatives.

We’re documenting our journey in real-time:

🔗 Follow along: https://t.co/89HLcgrtVt

[Akash Srivastava @variational_i]

Excited to share our latest work with @ishapuri101 et al.! 🚀 We introduce a probabilistic inference approach for inference-time scaling of LLMs using particle-based Monte Carlo methods—achieving 4–16x better scaling on math reasoning tasks and O1-level performance on MATH500.

[Isha Puri @ishapuri101]

[1/x] can we scale small, open LMs to o1 level? Using classical probabilistic inference methods, YES! Joint @MIT_CSAIL / @RedHat AI Innovation Team work introduces a particle filtering approach to scaling inference w/o any training! check out https://t.co/Iz8zoVbZPn

[Seungwook Han @seungwookh]

🧩 Why do task vectors exist in pretrained LLMs?

Our new research uncovers how transformers form internal abstractions and the mechanisms behind in-context learning(ICL).

[Cole Hurwitz @cole_hurwitz]

Neural activity is correlated among animals performing the same task and across sequential trials.

Led by @zhang_yizi and @hl3616, we develop an reduced-rank model that exploits shared structure across animals to improve neural decoding.

https://t.co/Ip7nO0q4yS

[Cole Hurwitz @cole_hurwitz]

What will a foundation model for the brain look like?

We argue that it must be able to solve a diverse set of tasks across multiple brain regions and animals.

Check out our preprint where we introduce a multi-region, multi-animal, multi-task model (MtM): https://t.co/eaC4jyFsBN

[Seungwook Han @seungwookh]

🚀 Stronger, simpler, and better! 🚀

Introducing Value Augmented Sampling (VAS) - our new algorithm for LLM alignment and personalization that outperforms existing methods!

[Seungwook Han @seungwookh]

Excited to give a talk on our hottest, newest work “Value Augmented Sampling for Language Model Alignment and Personalization” at 2:30p Halle A3 in #ICLR2024 Reliable and Responsible Foundation Models Workshop 🥳🥳

[Akash Srivastava @variational_i]

Attending #ICLR2024, interested in continual learning and like probabilistic modeling? Lazar from the @MITIBMLab, will be presenting our latest work that takes a probabilistic approach to modular continual learning on Tuesday, 7 May, Halle B #222 (https://t.co/dVYKhvtkM7).

[Lazar Valkov @lazarvalkov]

I’ll be presenting our #ICLR2024 paper on a probabilistic approach to scaling modular continual learning algorithms while achieving different types of knowledge transfer. (https://t.co/IbhoqPmjkI, in collaboration with @variational_i @swarat @RandomlyWalking ). A tldr (1/8):

[Faez Ahmed @_faezahmed]

Check out our work titled "From Automation to Augmentation: Redefining Engineering Design and Manufacturing in the Age of NextGen-AI", where we highlight the requirements for NextGenAI suitable for design, engineering, and manufacturing.
https://t.co/vHY6l39nqO

[Akash Srivastava @variational_i]

@jeethu @Geronimo_AI Good point, model card has been updated with v2 results.

[Akash Srivastava @variational_i]

New work from @MITIBMLab researchers on large scale alignment of LLMs. Check out the models at HF https://t.co/AlujBOwM0O

[David Cox @neurobongo]

Hey, we did a thing: "LAB: Large-scale Alignment for chatBots"—a new synthetic data-driven LLM alignment method that yields great results without using large-scale human or proprietary model data.

https://t.co/QdrAzgD9Kr
models: https://t.co/Q1eHsPOHUv, https://t.co/ipNIwpJPuf

[Akash Srivastava @variational_i]

New work on automated red-teaming in LLMs using curiosity-driven exploration! #iclr24

[Zhang-Wei Hong @ZhangWeiHong9]

(1/4) 🎉 Excited to share our ICLR'24 paper on "Curiosity-driven Red-teaming for Large Language Models"! We bridge curiosity-driven exploration in reinforcement learning (RL) with red-teaming, introducing the Curiosity-driven Red-teaming (CRT) method. #ICLR24 #AI #LLMSecurity