My company got acquired. So now I'm going to teach.
4 years running a software innovations agency → a lot of hard-won lessons about building software that actually works.
This account is where I share them. No fluff. Just the real stuff.
Sounds reasonable and mimics how humans go about doing the task:
The paper proposes STAIR, a structure-aware retriever that uses a document’s table of contents and section hierarchy rather than chopping everything into arbitrary token-sized chunks.
RAG is officially cooked.
IBM built a retriever that hallucinates 65x less than fine-tuned RAG systems.
Right now, every company is using Retrieval-Augmented Generation (RAG) to stop their AI from hallucinating.
The idea is to give the AI a private database to read so it doesn't make things up.
But there is a massive structural flaw.
Standard RAG blindly grabs documents based on vector similarity and forces the AI to read them. If the retriever pulls irrelevant garbage, the AI doesn't stop.
It just hallucinates with citations.
IBM just dropped a paper that changes the math entirely.
They built a new retrieval architecture that hallucinates 65x less than state-of-the-art fine-tuned RAG systems.
65x is the difference between a fun prototype and a production-grade enterprise system.
Instead of blindly dumping context into the model and hoping for the best, their system actively filters and validates the retrieved data before the generation step even begins.
It cuts out the noise. It forces absolute precision.
For businesses, this is the holy grail.
Zero hallucinations means zero liability. It means autonomous agents can finally be trusted with sensitive company data without a human babysitting every output.
We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:
Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:
Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:
Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:
Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!
In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.
Retriever 0.5.0 🐕
A couple of people asked for the file browser and SSH terminal to stay in sync. This new version does exactly that! cd in SSH, and Files view follows, or navigate in Files and the terminal follows.
Also new: drag-and-drop and multi-file/folder transfers.
See demo video here and update your app 👇
Retriever 0.4.0 is out! 🐕
My favourite addition: right-click a folder → SSH into Folder. A terminal opens inside the file browser. Switch back to files without ending the session.
Oh and uploads (⌘U) are here too, but we'll still call it Retriever!
https://t.co/BHFAXFZUac
@_andrewthecoder BASIC on my uncle’s c64 while my cousins hated the fact that I hijacked their game console to write hello world 🤣
Professionally (paid): C on Borland C
Now, what's most amazing about LLMs is this!
I asked Codex to hand-craft an Arm-Mac assembly version to match the functionality of https://t.co/gPIyhUqXXE... and the result is super awesome and elegant.. instead of the 3.5MB resulting assembly via Cython-generated C, we have a beautiful 60-line Assembly file.. see image in reply 👇
This is a great recap. The most important takeaway, true in pre-AI era as much as post: 𝐓𝐡𝐞 𝐦𝐨𝐬𝐭 𝐞𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭 𝐬𝐡𝐢𝐩𝐩𝐢𝐧𝐠 𝐮𝐧𝐢𝐭 𝐢𝐬 𝐚𝐧 𝐞𝐧𝐠𝐢𝐧𝐞𝐞𝐫 𝐰𝐢𝐭𝐡 𝐠𝐫𝐞𝐚𝐭 𝐩𝐫𝐨𝐝𝐮𝐜𝐭 𝐭𝐚𝐬𝐭𝐞
Turns out Gemini and Google Maps (and other services) share the same API keys... but Google Maps API keys are designed to be public, since they are embedded directly in web pages. Gemini API keys can be used to access private files and make billable API requests -@simonw
@madsf88 Like riding a bike or skiing, by doing! But also “put on a helmet” and start slow:
- isolated VM or VPS
- only private/paired Telegram BOT
- no access to your email or socials, if you must give it a dedicated/separated email
OpenAI WebSocket mode shines when a workflow needs lots of model‑tool round‑trips (e.g., agentic coding or orchestration loops).
By keeping the connection open and sending only incremental input each turn, it cuts per‑turn overhead and slashes latency:
https://t.co/omalEPRxV7
Firefox 148 ships with an AI kill switch: One toggle to turn off all AI features.
This highlights an important fact: AI integrations are rolling out faster than user trust is being earned.
https://t.co/7CmSYsshSx