Casey A. Fitzpatrick

@ContextualAI MTS | Quantum Information PhD

[Casey A. Fitzpatrick @caseyfitz]

Hard to believe it’s barely been a year since @douwekiela called to order our very first all hands, then @apsdehal and I spent hours in a tiny room with a whiteboard laying out the technical vision for what we were about to do. So proud of the team we’ve built and what’s to come!

[Contextual AI @ContextualAI]

We’re excited to share today that we’ve raised $80M in Series A funding to accelerate our mission to change the way the world works through AI. Read more at our blogpost: https://t.co/aTLx0NmQfr

[Casey A. Fitzpatrick @caseyfitz]

Another great example of how we frame the problems of enterprise ai from a systems perspective... and this component is a key player 😎 (which is also SOTA by itself 🙌

[Douwe Kiela @douwekiela]

AI struggles with messy, conflicting, ever-changing data. Today's AI ranking methods can't prioritize clearly, because they lack human guidance. Introducing the world's first instruction-following, SOTA reranker!

Give our reranker instructions to control exactly how it ranks:
• “Prioritize recent documents”
• “Prefer PDFs over other sources”
• “The boss is always right”

Can’t wait to see what people build with it!

[Casey A. Fitzpatrick @caseyfitz]

@douwekiela Psyched to show the world a little bit more about how we're tackling real problems that will unblock AI's true potential.

[Casey A. Fitzpatrick @caseyfitz]

Here, have a truly open MoE 🤲. More amazing work by @Muennighoff! Tons to dive into here.

[Niklas Muennighoff @Muennighoff]

Releasing OLMoE - the first good Mixture-of-Experts LLM that's 100% open-source
- 1B active, 7B total params for 5T tokens
- Best small LLM & matches more costly ones like Gemma, Llama
- Open Model/Data/Code/Logs + lots of analysis & experiments

📜https://t.co/Vpac2q90CS
🧵1/9

[Stas Bekman @StasBekman]

Here is a new Machine Learning Engineering chapter: Network debug

https://t.co/4g90Eq8o14

The intention is to help non-network engineers to figure out how to resolve common problems around multi-gpu and multi-node collectives networking - it's heavily NCCL-biased at the moment. Will extend with RCCL and others when I get access to those.

Your feedback and corrections are always welcome.

[Casey A. Fitzpatrick @caseyfitz]

ty @charles_irl and gang

[Stas Bekman @StasBekman]

The kind folks from @modal_labs have just shared with me this 10-100x faster drop in replacement for pip

https://t.co/GYzRtkZpsH

If you want a much faster CI startup switch to uv now!

To use you just add `uv` before `pip` and everything else is the same, so:

pip install uv
uv pip install -e .
uv pip install torch
uv pip compile ...
etc.

[Casey A. Fitzpatrick @caseyfitz]

@hugobowne @sh_reya @VanishingData Haha with none other than the legendary @charles_irl in the mix. Small world.

[Douwe Kiela @douwekiela]

Maximizing expected human utility (i.e., KTO) is the natural way to do alignment. Cool to see how well this works even in diffusion models.

[sarah guo @saranormous]

from @douwekiela (a pioneer of Retrieval-Augmented Generation) on the value of end-to-end co-optimization of systems (language models + retrievers):

[Casey A. Fitzpatrick @caseyfitz]

😎

[Nathan Lambert @natolambert]

contextual . onlybangers . ai

[Casey A. Fitzpatrick @caseyfitz]

🤩

[Omar Khattab @lateinteraction]

Not everyone gets to call their new system RAG 2.0 — exciting announcement by the @ContextualAI team, who have been thinking about these problems for many years.

[Casey A. Fitzpatrick @caseyfitz]

rip FRAG 🪦 we'll never forget the many meetup demos

[Aaref Hilaly @aaref]

2023: RAG vs no RAG
2024: RAG 2.0 vs frozen RAG

As AI moves into production, simple RAG systems are not enough. @douwekiela & @ContextualAI show that with data.

[Casey A. Fitzpatrick @caseyfitz]

Last year I joined Contextual AI to focus on designing and building production-grade AI systems, from first principles, focused on real world workflows not demos. Today I'm excited to share some of what we've done so far!

[Contextual AI @ContextualAI]

Today, we’re excited to announce RAG 2.0, our end-to-end system for developing production-grade AI.

Using RAG 2.0, we’ve created Contextual Language Models (CLMs), which achieve state-of-the-art performance on a variety of industry benchmarks. CLMs outperform strong RAG baselines built using GPT-4 and top open-source models like Mixtral, according to our research and customers.

Read more in our blog post: https://t.co/YUFgTS3izT

[Casey A. Fitzpatrick @caseyfitz]

👋hiiii we’re here to play!

[Winnie Xu @winniethexu]

Excited to share a new model with @ContextualAI that tops the AlpacaEval 2.0 leaderboard!

How did we manage to rank higher than models like GPT4, Claude 3 and Mistral Medium? Enter iterative alignment… 🧵

[Kawin Ethayarajh @ethayarajh]

The Orca-Math paper does a comparison of DPO and KTO for mathematical reasoning, finding that KTO is slightly better when all data is used and 25+ pts better when you have fewer positive examples than negative examples.

[Omar Khattab @lateinteraction]

I'm glad that a lot more people understand the key ideas behind ColBERT and DSPy now.

My only remaining goal is to make sure people can also say them correctly; both are quite tricky😆

* Col-BAIR (it's "the late" interaction retriever, get it?)
* Dee-Ess-Pie (like num-pie)

[Casey A. Fitzpatrick @caseyfitz]

😊 🙏

[Andrew Carr 🤸 @andrew_n_carr]

I know everyone is excited about Mixtral and the new Hyena models - but @ContextualAI just dropped a pile of cool new models and a new alignment framework

https://t.co/Mmn3Vvff3T

[fraser @Fraser]

I believe strongly that:

1) The best products that will emerge from this moment are “full stack”, with teams training their own models, and the models & UI informing one another.

2) This requires researchers who care deeply about what’s best for the product, including data

[Casey A. Fitzpatrick @caseyfitz]

🌠🌠🌠

[Contextual AI @ContextualAI]

We're proud to be a 2023 #IA40 Intelligent Applications Rising Star winner!