Charts contain important information in enterprise documents, but agents often struggle to retrieve and reason over them accurately. Chart images can be misread, while captions often omit the precise values and labels needed for retrieval.
That’s why our team at @DbrxMosaicAI developed structured chart extraction for ai_parse_document, converting chart content into JSON that can be indexed and used by Genie and other RAG systems.
Across two chart-heavy benchmarks, our approach, which uses a lightweight 300M-parameter text embedding model, outperformed multimodal embedding baselines which were up to 10× larger.
@aaxsh18@DbrxMosaicAI Hi @aaxsh18 ! We are not planning to release the dataset, as we created it with the intent of testing more realistic scenarios at a smaller scale rather than as a full benchmark. However, the documents we created the questions from are linked in the blog post!
Cool counterintuitive result from our team: on 2 hard chart-retrieval benchmarks, a small text embedding + up-front parsing (ai_parse_document) matches or beats a big multimodal embedding model.
Big congrats to @amruthaaa___ , who led this work! 👏
https://t.co/Bl0W6RCH5H
Are you using Claude Code or Codex to:
- Ask questions over a dataset of PDFs
- Turn your docs folder into structured tables or JSON
- Classify documents like customer notes over a set of labels
If so, you could wait an hour for Claude to run a for loop + multimodal model on your documents
or you can add the Databricks ai_parse_document function to one of your skills and batch process your documents faster and more accurately 💃🏻
Awesome opportunity for students excited to apply their research to real-world enterprise problems — and compete for $100k in model credits 💰
Apply by April 3 → https://t.co/qBmWGVmqTg