Mo

SVP of Engineering @ Trilogy

[Miles Turpin @milesaturpin]

🚀New paper!🚀

Chain-of-thought (CoT) prompting can give misleading explanations of an LLM's reasoning, due to the influence of unverbalized biases. We introduce a simple unsupervised consistency training method that dramatically reduces this, even on held-out forms of bias.
🧵

[Mo @mo_trilogy]

At Trilogy, our AI agents solve problems iteratively: they collect context, generate an output, evaluate, and iterate until a goal is achieved.

Biggest impact we see is in tool names & definitions rather than the system prompt. (i.e., nodes & edges) -- so there is truth here

[Itamar Friedman @itamar_mar]

@langchain When building real world AI empowered systems, we see a shift from prompt engineering to flow (/graph) engineering.

LangGraph and AlphaCodium are a perfect fit 👏

AI tools will help us busy developer to code with on the fly generated tests.

[Mo @mo_trilogy]

This is a game changer -- we're approaching a point where RAG will no longer be necessary for most use cases.

[Jeff Dean @JeffDean]

Needle in a Haystack Tests Out to 10M Tokens

First, let’s take a quick glance at a needle-in-a-haystack test across many different modalities to exercise Gemini 1.5 Pro’s ability to retrieve information from its very long context. In these tests, green is good, and red is not good, and these are almost entirely green (>99.7% recall), even out to 10M tokens. Great! A bit more on needle-in-a-haystack tests later in the thread.

[Mo @mo_trilogy]

If you haven't tried @sweepai , you're missing out. It's like having a Junior Software Engineer on your team!

Just create a GH Issue starting with "Sweep: ", and it'll churn out a PR with an explanation in 5 minutes.

[Robert @Tantalon]

I don't know how, but Sweep just fixed an unrelated bug in a separate file while working on a feature. I didn't ask it to - I didn't even know the bug was there. Sweep just found and fixed it.

[Mo @mo_trilogy]

If you're not using AI to 10x your productivity in 2024, you're missing out

[Muhammad Usman @SBrainAI]

Just transformed a tedious task into a breeze! Leveraged the power of AI with @OpenAI's ChatGPT to effortlessly convert raw DNS data for 15 domains into an import-ready format for Amazon Route 53. Minutes of AI magic saved me hours of manual work! 🚀 #AI #Automation #TechWin

[Mo @mo_trilogy]

Cursor is like magic, I didn't even tell it my intent here, I just asked for a textbox that renders conditionally

It was able to generate the placeholder, value and onChange fields as if it read my mind

Those guys at @cursor_ai really know what they're doing

[Mo @mo_trilogy]

We're obsessed with agents at Trilogy -- we have Level 3 & Level 4 agents in production right now that are DOing & QCing a subset of our Engineering & Support work units

[Jerry Liu @jerryjliu0]

🪜 4 Levels of Agents for RAG 🤖

There’s an entire spectrum of agentic capabilities offering simple-to-advanced reasoning on top of your data.

Here’s a mini-🧵 on the 4 different levels of agents and how they can augment your RAG pipeline 👇

1️⃣ Tool Use: A linear pipeline that can query a set of tools given a user query.

The simplest example is a router: https://t.co/hlwMh4QbDE

2️⃣ Reasoning Loop + Memory: Simple loop around a RAG pipeline. Given a user query and previous conversation history, infer a new query to query the pipeline.

We add a simple retry-layer on top of a text-to-SQL query engine: https://t.co/MRKrhorCVe

3️⃣ Reasoning Loop + Memory + Tool Use: A loop around a pipeline designed to pick a set of tools given a user query.

Here we combine reasoning loops with tool use. A simple example here is a retry layer on top of a router: https://t.co/vyGtpplbSl

4️⃣ Fancy Reasoning Loop + Tool Use + Memory: The reasoning loop can get arbitrarily complicated, from sequential ReAct chain-of-thought to LLMCompiler-style parallel planning and tool use.

ReAct from scratch: https://t.co/eGetiu0qj5

Check out our LLMCompiler implementation here: https://t.co/sp0qfpWo9f

[Mo @mo_trilogy]

This is really cool for anyone interested in confidence score elicitation out of an LLM

[Google AI @GoogleAI]

Introducing ASPIRE, a framework that enhances the selective prediction capabilities of large language models, enabling them to output an answer paired with a confidence score. Learn how it outperforms state-of-the-art methods on a variety of QA datasets. → https://t.co/NQeIV1fXJ6

[Mo @mo_trilogy]

Good stuff

[Aman Sanger @amanrsanger]

At Cursor, we’re fascinated by the problem of deeply understanding codebases.

One useful primitive we’ve been focused on is code graph construction and traversal.

Here's how/why we're tackling this... (1/12)

[Jerry Liu @jerryjliu0]

Using LLMs for text-to-SQL is flaky (e.g. should it be `WHERE country = “Japan”` or `WHERE country = “japan”`?)

Here’s a simple but useful idea to use RAG to fetch few-shot examples for less flaky text-to-SQL (or…less flaky structured RAG itself). Calling it dynamic metadata retrieval:

1️⃣ Index and embed each row
2️⃣ In the text-to-SQL prompt (or auto-retrieval prompt), add *few shot examples of rows*: given the first k rows in the prompt, retrieve the top-k rows matching the user query.
3️⃣ Execute text-to-SQL prompt (or auto-retrieval prompt) to infer the right query (SQL or metadata filters).
4️⃣ Execute query to get back result.

The key intuition is that you “pre-fetch” a sample of relevant rows to help infer the query which will then fetch the full-set of relevant rows!

Credits: This came up in conversation with Tianyang (author of “Rethinking Tabular Data Understanding” paper which we’ve posted about): https://t.co/7xirt4XHhc. I’m sure this isn’t a new idea, and if you have suggestions leave it in the comments below 🙂

(Full trace in screenshot through @arize_phoenix)

We’ve implemented this in TWO guides:

Text-to-SQL: https://t.co/Uig5srT4EP

Auto-retrieval: https://t.co/8RSqsfPSIV

[Jozsef Czapovics @jczapovics]

@CodeWisdomPL @FabianoLucchese try @perplexity_ai

[Matthew Berman @MatthewBerman]

Open Interpreter is like ChatGPT Code Interpreter, but you can run it locally on your computer, and it is 100% open-source.

Write and execute code, analyze data, control your computer, and more.

Great work @hellokillian

Here's how to install and use it:

👇

[Robert @Tantalon]

Natural language AI prompts have a lot of unnecessary information. You can eliminate up to 95% and still get the same results. It's even faster and cheaper! Check out LLMLingua. https://t.co/8zxp6y6BaU

[Mo @mo_trilogy]

@DoWhileAI

[Mo @mo_trilogy]

If you haven't tried https://t.co/sjZveaSN4S, you're missing out.

It can ingest your repos, and answer complex questions.

We gave it the following query: "Customers can schedule an interview with inactive users".

Its response? A perfect, code-level RCA.