1/2 You don't need a vision model to ask a question, only to read the page. That split is worth stealing for any retrieval system with expensive documents and cheap queries.
Documents are visually complicated. Questions about them usually aren't.
NanoVDR exploits that asymmetry: a big 2B vision model indexes your PDFs once, offline — but a tiny 69M-parameter, CPU-only text encoder handles every query afterward, at 51ms, retaining 95% accuracy.
A useful rule in AI engineering:
Don’t start with the model. Start with the failure mode.
Ask:
• What can go wrong?
• Can rules solve part of it?
• Where does ML add value?
• How will we know it works?
Good AI starts with good problem framing.