Thomas gilles

data scientist @bnpparibas @cibailab

[LangChain @LangChain]

🤖 Browser Use: Web AI

A powerful open-source library that lets AI agents naturally interact with websites. Using LangChain and GPT models, it automates complex web tasks from navigation to form-filling, making browser automation seamless.

Learn more: https://t.co/x9hRshF6ux

[Philipp Schmid @_philschmid]

OCR-2.0 is coming, and Generative AI and multimodal LLMs will power it! 🔍 GOT (General OCR Theory) is a 580M end-to-end OCR-2.0 model that outperforms all existing methods.

GOT consists of a Vision-Encoder to convert images into transformers images into tokens and a decoder for generating OCR outputs in various formats (e.g., plain text, markdown, Mathpix). GOT is designed to handle complex tasks like sheets, formulas, and geometric shapes

Implementation
1️⃣Vision Encoder Pre-training: The encoder (VitDet) trained using scene text and document OCR data to recognize both slice and whole-page inputs.
2️⃣ Joint-Training: The encoder is connected to the decoder (Qwen-0.5B) and both are trained on more general OCR tasks (e.g., formulas, sheet music, geometry).
3️⃣ Post-Training: Fine-tuned the model with specific tasks, such as fine-grained OCR, multi-page PDFs, and dynamic resolution, using new synthetic datasets.

Insights
🧠 Encoder-Decoder with 80M (VitDet) Encoder and 500M (Qwen2) Decoder with 8k context
🥇 Achieves a 0.035 edit distance and 0.972 BLEU score on plain OCR
📊 Outperforms LLaVA-NeXT and Qwen-VL-Max in document and scene text OCR
🧮 Can extract LaTeX formulas from Arxiv and convert them to Mathpix format
📃 Supports dynamic resolution and multi-page OCR
🖼️ Input resolutions up to 1024x1024

Paper: https://t.co/QeiihLPfp5
Github: https://t.co/rGodXsI0KF

[Thomas gilles @gilles_thomas]

@rohanpaul_ai @huggingface Definetely. So many hours spent by the past. It is really a goldmine !!!!

[Rohan Paul @rohanpaul_ai]

Finding quality OCR datasets was a huge challenge 🤔

And then I see that one of the largest OCR datasets are now available to the public as open @huggingface dataset 🔥

With over 26 million pages , 18 billion text tokens, and 6TB of data. 🤯

These resources are just goldmine for document AI research.

[Yann LeCun @ylecun]

Bonne question.

[Thomas gilles @gilles_thomas]

@ElonMuskAOC @NFTHabib @readwise save

[Thomas gilles @gilles_thomas]

Hey Kelvin Thomas thanks for the follow!

[Thomas gilles @gilles_thomas]

Hey Testdata thanks for the follow!

[Thomas gilles @gilles_thomas]

AI Year-In-Review Roundup And Analysis Along With Pearls And Perils Entailing AI Ethics And AI Law https://t.co/SVz5IfLeqZ

[Thomas gilles @gilles_thomas]

Is DOGE really worth the hype even after Musk’s Twitter buyout? https://t.co/KOFolMelod

[Thomas gilles @gilles_thomas]

Proof-of-reserves: Can reserve audits avoid another FTX-like moment? https://t.co/DZS2X4bWcq

[Thomas gilles @gilles_thomas]

Understanding Applications of Artificial Intelligence (AI) in the Gaming Industry https://t.co/O8XLeSP1N5

[Thomas gilles @gilles_thomas]

Thinking Fast and Slow for Data Science https://t.co/2iSbTxgPUU

[Thomas gilles @gilles_thomas]

How data science and technical SEO can supercharge SEO marketing https://t.co/PQosqshsH8

[Thomas gilles @gilles_thomas]

Amazon VP explains how to move past limits of machine learning https://t.co/BpfObD9iMj

[Thomas gilles @gilles_thomas]

Data-centric AI and Tabular Data https://t.co/vQM1Y2ehSU

[Thomas gilles @gilles_thomas]

Hey Niranjan Sahani thanks for the follow!

[Thomas gilles @gilles_thomas]

News - à lire sur https://t.co/uoskbIuj4c Merci à @BFMBourse @arnaud_thurudev @Remy_Grosdoigt #ia #transfonum

[Thomas gilles @gilles_thomas]

Hey IoT Worlds thanks for the follow!

[Thomas gilles @gilles_thomas]

Hey scallet Johnson thanks for the follow!