Omar Khattab

prof @MIT CSAIL | ColBERT, (@DSPyOSS), GEPA, RLMs, Pedagogical RL

[Omar Khattab @lateinteraction]

@dbreunig ah i’m worried we’ll have a rise of a misguided “threshold engineering” industry as a result

[Ben Clavié @bclavie]

I think I spy something new on there

[Christopher Potts @ChrisGPotts]

The wonderful “Don’t just Fix it in Post” paper identifies predicting memorization as a key open problem.

A progress report: we can now accurately predict verbatim memorization using just three well-understood variables. Experiments by Jing Huang, using the Hubble model suite:

[Denis Yarats @denisyarats]

you can now run perplexity's decider model locally on your mac. it's private, fast, and free, with no api key needed. https://t.co/etN9lWDTQk

decider for mac runs pplx-decider-flash-v1.2, our 4B decision model, fully on device. ask a yes/no question about text or an image and get a probability back. after a one-time 5.8 GB download, it serves the decisions api at localhost:8080/v1/decisions.

use it to label files, route messages, check screenshots, or run pre-commit checks, without any data leaving your mac.

[Omar Khattab @lateinteraction]

@srchvrs really great figure!

[Omar Khattab @lateinteraction]

confession: i've yet to have that moment of feeling that frontier LLMs are geniuses, or super-human at ~anything broadly coherent that i deeply prize

[Omar Khattab @lateinteraction]

@omaclaren i mean this very unironically but they’re computers after all

[Omar Khattab @lateinteraction]

one way to understand what i mean is the sheer amount of expertise AND (expert) effort required to extract high-caliber work out of them - i guess all the slop research complaints out there are a common consequence of this

[Omar Khattab @lateinteraction]

i'm not sure what it is that my work needs or what i'm presumably 'good' at, but despite extremely elaborate (& happy!) use and having worked on nothing but language models for essentially my entire career, i can only see that kind of vibe abstractly from others' (e.g., mathematicians) reactions, not in the vicinity of anything that i need or do

they're the most impressive tools i've ever used, and i essentially use my computer mostly from them now, but yeah, not yet

anyway, back to cc and codex cuz we have work to do

[Braden Hancock @bradenjhancock]

It's like self-supervised learning at the higher abstraction of research contributions instead of words: take a naturally occurring thing, mask out a part, see if the AI can recover it. What's the next level up of abstraction? "Pretend we know nothing about the field of acoustics; what can you recover?"

Congrats to @MangQiuyang, @lihanc02, and team. Another @LaudeInstitute Slingshot we're proud to have supported.

[Ben Clavié @bclavie]

On this note, if you've ever thought "Man, I wish I had BEIR and NanoBEIR, but considerably better and very discriminative of model quality!", you might also want to stay tuned.

i hear it might also be fully open

[Ben Clavié @bclavie]

but you do need to evaluate them better, BEIR considered harmful (but we thank it for its services)

[Ben Clavié @bclavie]

extremely excited for our upcoming releases. retrievers aren't even vaguely close to anything resembling a ceiling

[Antoine Chaffin @antoine_chaffin]

We've released a technical report on the released late interaction models
It's purposely kept short, simple and direct, because most of the recipe is not new (shared space is not really new as well, but pretty cool for ColBERT still)
There are still a lot of technical information if you are interested beyond the BP
We are preparing much more thorough papers for things where we did really bring a lot of novelty, pretty sure you'll enjoy those!
https://t.co/73gLjJd4oi

[Bo @bo_wangbo]

Here we have a minimal tech report on 2 Colbert models:

[Manning Publications @ManningBooks]

Save 50%, only on Oct 21st!

Today, you can save on Building LLM Applications with DSPy by Serj Smorodinsky and Brett Kennedy — now in print — and other related titles: https://t.co/ln788YfqJJ

Learn how to build DSPy into fully agentic pipelines, integrating conversation memories, and connecting agents to external environments via MCP.

[Andy Konwinski @andykonwinski]

applications for slingshots more than doubled from batch 3 to batch 4.

funding open science is critical to democracy.

never in the past have scientists had to turn down so much money to do research they can talk about openly. top PhD students are receiving offers of millions of dollars per year from closed labs.

i'm proud that at Laude we support open research.

great work on slingshots batch four @bradenhancock and the @laudeinstitute team

[humans& @humansand]

No one changes the world alone
Introducing Ampersand - the first model built for us. Ampersand is a collaborator trained to help people achieve their individual and collective goals. In multi-month simulations, users collaborating with Ampersand accomplished more than with Fable and other frontier models
Available to everyone through our API

[Matthew Russo @RussoMatthew]

Incredibly grateful to @LaudeInstitute for supporting our work on WaggleHaus!

TL;DR: We believe that supporting agentic queries over massive data lakes requires rebuilding the data system from the ground up.

This includes ideas ranging from:
- making the data layer more "agent-friendly"
- to specializing the inference engine for the query workload
- to running agents within the system that continuously update views and indices for faster data access

If you're interested in a demo or becoming an industry design partner you can sign up here: https://t.co/w6NMA09apD

[Veronica Qing Lyu @veronica3207]

Check out our latest research: MemCurrent, a persistent agent memory architecture, which consistently stays on the cost-accuracy Pareto frontier on our internal and three public benchmarks. Try it in the new Agent Bricks Memory and Genie 🚀