Juan Manuel Ciro

[George Halal @halal_george]

Excited to share that we trained rerankers at the cost/performance frontier and are open sourcing them!

Contextual AI Reranker v2
🚀 Best performing, most efficient reranker
🤗 Open weights (1B, 2B, 6B)
🫡 Instruction-following (including recency-awareness)
🌐 Multilingual

1/4

[Douwe Kiela @douwekiela]

Context engineering has become the critical bottleneck for enterprise AI. Your AI agent works perfectly in demos but breaks down with real-world data complexity. Why? I see 6 fundamental challenges that every AI engineer faces: from the "needle in a haystack" problem where models lose critical information buried in long contexts, to the token cost explosion that makes production deployments prohibitively expensive. These are more than just technical hurdles, they're the difference between AI experiments and transformative business impact. Read my full thoughts below.

[Rajiv Shah @rajistics]

Introducing: Contextual AI MCP Server (now hosted)

After great feedback with our local MCP server, we have added a hosted MCP server inside the platform!
This mean every RAG agent is easily accessible via MCP.

Added updated info on the hosted MCP server here:
https://t.co/obM5ZwaREe

[CircleCI @CircleCI]

Tired of guessing if your LLM responses are “good enough?”

With @ContextualAI’s LMUnit and CircleCI, you can run natural language unit tests directly in your CI/CD pipeline—turning subjective evals into automated, testable checkpoints.

Read on: https://t.co/WM4ARflwv2

[Juan Manuel Ciro @ciropython]

The future of CI

[Contextual AI @ContextualAI]

🔥 Introducing the most reliable way to evaluate LLMs and agents in production! It's time to stop “vibe testing” your AI systems.

Our latest developer's guide shows you how to rigorously test AI systems so that they hold up in production, using Contextual AI's LMUnit evaluation model and @CircleCI’s CI/CD pipeline. You’ll learn how to:
• Write natural language unit tests that anyone on your team can understand
• Leverage LMUnit – Contextual AI's state-of-the-art, specialized evaluation language model that outperforms frontier models with greater interpretability at lower cost
• Implement @CircleCI's CI/CD pipeline to catch regressions before they reach users

See our complete developer’s guide here: https://t.co/LgMhAAOTyi

Stop relying on "vibes" and start building AI you can trust!
#AITesting #LLMOps #DevOps #Agents #LLM #Evaluation

[Douwe Kiela @douwekiela]

$20k/month for an AI agent is an interesting pricing decision. We're in new territory, where potential buyers are almost explicitly being forced to choose between a human vs an AI to do a particular job..

Not sure if this is the right strategy, but certainly interesting times ahead. I think we can all agree on the specialization part though. Glad to see a giant in the industry embrace something we've been saying from the start.

[Contextual AI @ContextualAI]

Introducing LMUnit: Natural language unit testing for LLM evaluation

How do you really know if your language model is behaving the way you expect?

When evaluation is this critical, your best methodology shouldn't just be vibes. With SOTA results on FLASK & BigGenBench and top-10 on RewardBench, LMUnit brings the rigor and familiarity of traditional software engineering unit testing to LLM evaluation.

Read on to learn how we built it and try it for free using our API 👇 🔗

🧵 (1/5)

[Hannah Rose Kirk @hannahrosekirk]

A real honour and career dream that PRISM has won a @NeurIPSConf best paper award! 🌈 One year ago I was sat in a 13,000+ person audience of NeurIPs '23 having just finished data collection. Safe to say I've gone from feeling #stressed to very #blessed 😁

[Juan Manuel Ciro @ciropython]

@hannahrosekirk @NeurIPSConf Congrats 🎉🎉

[Hannah Rose Kirk @hannahrosekirk]

Wahoo PRISM will officially be taking a trip to @NeurIPSConf this year as an oral presentation 🤩 (and my first ever 10/10 in a conference review process 🤯)

[Juan Manuel Ciro @ciropython]

@SaraDurango_ @joseluisintilde Totalmente de acuerdo, nos vemos por allá, muchos años sin hablar

[Juan Manuel Ciro @ciropython]

@SaraDurango_ @joseluisintilde Vas a ir al concierto?

[Juan Manuel Ciro @ciropython]

@jobergum Would be great a tutorial step by step showing how you created the data and the code for evaluation.

[Juan Manuel Ciro @ciropython]

@AsociacionMi Pequeño aporte

[Juan Manuel Ciro @ciropython]

@LiLiPuRpLe @FundacionToby Esta perfecto

[Juan Manuel Ciro @ciropython]

@AsociacionMi Un pequeño aporte

[Juan Manuel Ciro @ciropython]

@nana1gc Un pequeño aporte

[Hannah Rose Kirk @hannahrosekirk]

Today we're launching PRISM, a new resource to diversify the voices contributing to alignment. We asked 1500 people around the world for their stated preferences over LLM behaviours, then we observed their contextual preferences in 8000 convos with 21 LLMs https://t.co/P7nMKuvT23

[Contextual AI @ContextualAI]

Today, we’re excited to announce RAG 2.0, our end-to-end system for developing production-grade AI.

Using RAG 2.0, we’ve created Contextual Language Models (CLMs), which achieve state-of-the-art performance on a variety of industry benchmarks. CLMs outperform strong RAG baselines built using GPT-4 and top open-source models like Mixtral, according to our research and customers.

Read more in our blog post: https://t.co/YUFgTS3izT

[Stas Bekman @StasBekman]

RAG 2.0 is turning LLMs from being an awesome toy to a tool that one can safely rely on - so businesses can actually start using AI in their workflows.

We at Contextual AI have done an awesome groundbreaking work to make it work. Please see the break down of how and why it works here: