Roee Hendel

Algorithm Developer @AI21Labs

[AI21 Labs @AI21Labs]

1/5 As part of our work on improving the efficiency of our LLM online-RL training pipelines, we cut policy update step time by ~70% by introducing a model-agnostic padding minimization method. 🧵

[AI21 Labs @AI21Labs]

1/4 🚀Introducing Jamba2, a memory-efficient open source model family built for total enterprise reliability and steerability.

[AI21 Labs @AI21Labs]

1/5 Releasing Jamba Reasoning 3B under Apache 2.0: Hybrid SSM-Transformer architecture that tops accuracy & speed across record context lengths. e.g. 3-5X faster than Llama 3.2 3B and Qwen3 4B at 32K tokens.

[AI21 Labs @AI21Labs]

We released the #Jamba 1.5 open model family:

- 256K #contextwindow
- Up to 2.5X faster on #longcontext in its size class
- Native support for structured JSON output, function calling, digesting doc objects & generating citations

https://t.co/tebBJW09c5

#AI #LLM #AI21Jamba

[AI21 Labs @AI21Labs]

Introducing Jamba, our groundbreaking SSM-Transformer open model!

As the first production-grade model based on Mamba architecture, Jamba achieves an unprecedented 3X throughput and fits 140K context on a single GPU.

🥂Meet Jamba https://t.co/f2XZFOQbxh

🔨Build on @huggingface

[AK @_akhaliq]

AI21 Labs presents Jamba

SSM-Transformer open model

production-grade model based on Mamba architecture, Jamba achieves an unprecedented 3X throughput and fits 140K context on a single GPU.

[Roee Hendel @RoeeHendel]

@yalishandi @_akhaliq We haven't explored this in depth, but it's a promising direction. Studies linking ICL to SGD could offer clues. In related tests, ICL sometimes acts like an empirical risk minimizer, fitting random examples, while other times ignoring them. Further exploration is needed.

[AK @_akhaliq]

In-Context Learning Creates Task Vectors

paper page: https://t.co/RdQiiHkA3I

In-context learning (ICL) in Large Language Models (LLMs) has emerged as a powerful new learning paradigm. However, its underlying mechanism is still not well understood. In particular, it is challenging to map it to the "standard" machine learning framework, where one uses a training set S to find a best-fitting function f(x) in some hypothesis class. Here we make progress on this problem by showing that the functions learned by ICL often have a very simple structure: they correspond to the transformer LLM whose only inputs are the query x and a single "task vector" calculated from the training set. Thus, ICL can be seen as compressing S into a single task vector theta(S) and then using this task vector to modulate the transformer to produce the output. We support the above claim via comprehensive experiments across a range of models and tasks.

[Roee Hendel @RoeeHendel]

@_akhaliq 🙏 @_akhaliq

[Roee Hendel @RoeeHendel]

@LouisKirschAI Great work! Did you release the code?

[Roee Hendel @RoeeHendel]

@kushal_tirumala Is it possible that the different baseline values simply arise from the fact that larger models have an overall better language modeling capability, rather than memorization? It would be interesting to check the memorization value of the "special batch" prior to training on it.

[Roee Hendel @RoeeHendel]

@elonmusk @Carnage4Life There you go, ChatGPT as Hitler. MtH<=4 days