Dr G

I'm Sarah, founding devrel engineer @tensorlake

[Dr G @drguthals]

I haven't posted yet about my new position because, no shade, I felt like the "I have a new role" just didn't do it justice.

There are a lot of reasons I decided to join @tensorlake

Here's a 🧵, but a huge part is this announcement today 👇

(1/4)

[Tensorlake @tensorlake]

Announcing Tensorlake Cloud

Up-leveling Document Ingestion and Workflows for building agentic applications and complex business workflows.

[LangChain @LangChain]

🚀 LangSmith Agent Builder is now available in Public Beta

Now anyone can create production ready agents without writing code, just chat.

Agent Builder guides you from initial idea to deployed agent, creating detailed prompts, selecting required tools, and even creating subagents.

Our Beta release also includes:
🧰 Bring your own tools via MCP server
🧩 Browse, copy, and customize agents across a shared workspace
🧠 Use your preferred model

Try Agent Builder free: https://t.co/N2O8g0LDt1
Read more on the launch: https://t.co/m763CWM3l8

[Dr G @drguthals]

Quickly process SEC filings so that you can see what AI risks NYSE companies are considering

[MotherDuck @motherduck]

Query unstructured documents with SQL??

We built a tutorial (complete with Colab notebook!) showing how Tensorlake and MotherDuck turn unstructured documents into analytics-ready datasets:

📄 Classify relevant pages in 100+ page SEC filings
🔍 Extract structured AI risk data with industry-leading accuracy
💾 Load into MotherDuck for instant SQL analysis
📊 Query trends across companies in seconds

The workflow is surprisingly simple: a few Python functions to extract data, then familiar SQL to analyze it. No complex pipelines or data engineering required.

Follow along in this post by Diptanu Gon Choudhury : https://t.co/4I7HKQUWqn

[MotherDuck @motherduck]

Query unstructured documents with SQL??

We built a tutorial (complete with Colab notebook!) showing how Tensorlake and MotherDuck turn unstructured documents into analytics-ready datasets:

📄 Classify relevant pages in 100+ page SEC filings
🔍 Extract structured AI risk data with industry-leading accuracy
💾 Load into MotherDuck for instant SQL analysis
📊 Query trends across companies in seconds

The workflow is surprisingly simple: a few Python functions to extract data, then familiar SQL to analyze it. No complex pipelines or data engineering required.

Follow along in this post by Diptanu Gon Choudhury : https://t.co/4I7HKQUWqn

[Akshay 🚀 @akshay_pachaar]

Turn complex docs into clean, LLM-ready data!

Every AI company I've talked to is solving the same problem: how do you build systems that don't hallucinate and back up every answer with proper citations?

Tensorlake is a tool that extracts custom-defined structured data from any unstructured document in 3 steps:

↳ Define your schema
↳ Enable citations
↳ Extract

You get RAG-ready data with precise citations and bounding boxes. Feed this to your LLM, and you'll generate responses that are citation-backed and fully auditable.

This is the difference between a demo and a production system.

When your AI can show exactly where it got its information, you move from proof-of-concept to something people can actually trust and deploy.

I've shared the Tensorlake GitHub repo in the replies!

[Diptanu Choudhury @diptanu]

We are experimenting with a new secure and parallel execution model for deep research agents @tensorlake. Text this number with a financial research question to see it in action - [phone]!

Every request is spinning up a sandbox for the agent harness in real time, the agent plans to make API calls to get market data, parses financial documents. Each of these tool calls are then executed in isolated sandboxes in parallel, transparently without queues or background tasks.

We add durable execution in the mix so if a tool call fails, the agent resumes from where it crashed, without throwing away all the progress it made.

Each agent process and tool call being run in a separate sandbox also means every user session gets it's own file system, and a secure environment for code execution. We wrap the sandbox container further into a VM so we can control even network calls and hardware device access.

This lets running Claude Code/DeepAgents based harnesses securely and in parallel for 1000s of users/requests in parallel.

[Diptanu Choudhury @diptanu]

We were on hacker news for 5-6 hours today + one of our customers uploaded 1000s of PDFs within a few minutes to do a load test around the same time. Interesting day at @tensorlake

Good news - not a single job failed. The p95 latency could have been better 😅

[Diptanu Choudhury @diptanu]

We launched @tensorlake's Document Ingestion API a couple of months ago, and it's already parsing millions of pages every month across Insurance, Healthcare, Financial Services, Research and Legal Tech.

Most conversations with new customers starts with - Is Tensorlake better than Azure's Document Intelligence and AWS Textract or open source models.

Tensorlake is the most accurate Document Ingestion API for reading and extracting structured extraction among the four models we tested.

OCR models are rarely trained on enterprise documents because they are private and not available in publicly available datasets. We went to great lengths to acquire documents with complex tables, layouts and figures, so our models perform reliably in production use cases, not just benchmarks. The results reflect our focus on building a Document Ingestion API for real world business documents.

Document AI Playground - https://t.co/7rBY5jwQZw

[Tensorlake @tensorlake]

Traditional OCR benchmarks measure the wrong thing.

See why most document parsing tests fail to predict production accuracy and which metrics actually matter.

[Dr G @drguthals]

See how we stack up against open source alternatives and enterprise solutions (like Azure, AWS Textract, and Gemini) in our latest benchmark report:
https://t.co/2dGza0xX0P

[Dr G @drguthals]

We're the best in class document parsing API.

You get all this 👇 out of the box, and with accuracy numbers to back it up, you can reliably parse complex documents and extract accurate date with ease.

(link to our benchmarks in the comments)

[Hasan Toor @hasantoxr]

Turn PDF files into clean, LLM-ready data.

Tensorlake's new API organized our PDF files and significantly enhanced performance.

It's the first system that effectively converts disorganized PDF files into structured data.

Here's how it works in 3 steps:

[Hasan Toor @hasantoxr]

Turn PDF files into clean, LLM-ready data.

Tensorlake's new API organized our PDF files and significantly enhanced performance.

It's the first system that effectively converts disorganized PDF files into structured data.

Here's how it works in 3 steps:

[Hasan Toor @hasantoxr]

Tensorlake just made structured document ingestion a one-line API call.

PLUS this is best in class, with 86.8% accuracy converting documents to markdown and 91.7% accuracy extracting structured JSON content from documents.

That's over 40% fewer failures than competing document parsing APIs (Azure, Gemini, AWS Textract).

Get the full report at https://t.co/9hwPglHgSh

[Santiago @svpino]

Most OCR benchmarks optimize for text similarity, but a document can be 99% "correct" and still be 100% useless.

Unfortunately, these benchmarks overlook the real issues that happen in production.

I'm linking to a report here. It challenges how the entire industry measures the accuracy of document parsing.

When you build real systems, text accuracy alone doesn't cut it, and you have to worry about many other problems:

• Losing the structure of the text
• Losing totals and values in tables
• Scrambling the reading orders
• Losing charts and figures

The report makes one thing obvious: we are measuring the wrong things!

Instead of asking, "Does the text look similar?" we should ask, "Can a downstream task use this text?"

There are two metrics that help here:

1. TEDS → Checks if the structure remains intact
2. JSON F1 → Checks if automated workflows can run without human babysitting

The @tensorlake team used these metrics on OCRBench v2, and they outperformed Azure, AWS Textract, and several open-source alternatives.

This is a huge step for anyone building RAG pipelines, financial automations, compliance workflows, or any other document process that relies on structured documents.

Here is a link to the report:

https://t.co/xVnvj4Gmq1

[Dr G @drguthals]

It's time to start measuring accuracy of document parsing and data extraction with downstream systems and usability in mind, not just vanity metrics for a marketing slide

[Tensorlake @tensorlake]

Document parsing benchmarks have been measuring the wrong thing.

We tested every major parser on real enterprise documents.

The results will change how you think about OCR accuracy 🧵

[Tensorlake @tensorlake]

Document parsing benchmarks have been measuring the wrong thing.

We tested every major parser on real enterprise documents.

The results will change how you think about OCR accuracy 🧵

[Diptanu Choudhury @diptanu]

. @tensorlake is surviving whatever is currently happening in AWS and Azure. We are still parsing documents and running people’s code on our platform 😅

[Dr G @drguthals]

Make your agents smarter with accurate and complete data

Learn how to extract data from unstructured documents with @tensorlake , store them in @qdrant_engine, and then use @LangChainAI for natural language querying.

Check out our lesson 👇

[Tensorlake @tensorlake]

Want to build scalable data lakes with Tensorlake + @qdrant_engine?

In the free Qdrant Essentials Course, learn how to:
- Architect vector-powered data lakes
- Optimize ETL pipelines
- Create knowledge graphs
- Integrate @langchain agents for natural language querying

Links below 👇

[Tensorlake @tensorlake]

Want to build scalable data lakes with Tensorlake + @qdrant_engine?

In the free Qdrant Essentials Course, learn how to:
- Architect vector-powered data lakes
- Optimize ETL pipelines
- Create knowledge graphs
- Integrate @langchain agents for natural language querying

Links below 👇

[Tensorlake @tensorlake]

New: Vision Language Models now power key document processing features
We're using VLMs for:
- Page classification in large documents
- Table/figure summarization
- Fast structured extraction (skip_ocr mode)

Here's what this means for document processing 🧵

[Dr G @drguthals]

Tracked changes in doc files contains *critical* data.

You need to be able to access it programmatically when parsing/extracting data.

You can do that easily, out-of-the-box, with Tensorlake.

[Tensorlake @tensorlake]

Most parsers strip all tracked changes when you extract the text.

That means:
❌ Lost audit trails
❌ Manual review of revision history
❌ No programmatic access to reviewer comments
❌ Workflows that can't route based on specific edits

Now you can parse .docx files with tracked changes preserved as clean, structured HTML:
- <del> tags for deletions
- <ins> tags for insertions
- <span class="comment"> for reviewer notes

Build approval workflows that trigger on specific feedback. Extract complete edit history for regulatory compliance. Route documents based on flagged sections, all programmatically.

Live now in our API, SDK, and Cloud.