Marco Cimolai

360° engineer. Love being efficient

[Marco Cimolai @CimolaiMarco]

@venturetwins @thinkymachines I'm working on something similar, but with small finetuned models! https://t.co/IxIqOtMTyr

[Marco Cimolai @CimolaiMarco]

I'm turning sgclipper-3b-v1 into an API!

- Audio-native clipping model
- Direct M3U8 inputs
- As low as $0.05/hr

Join the waitlist if you're interested ↓
https://t.co/DqLHxshHiY

[Marco Cimolai @CimolaiMarco]

Introducing sgclipper-3b-v1: an experimental audio-native clipping model that competes with Gemini 3.5 Flash, @OpusClip, @veedstudio and more.

Instead of using transcribed speech, it directly predicts clippable moments from an audio feed.

🤏 Based on @MistralAI Voxtral Mini
🚀 >50x realtime speed
♾️ Infinite audio length
💵 Extremely cheap, lower than $0.007/hr

https://t.co/LlpyB1LepZ

1/2

[Marco Cimolai @CimolaiMarco]

sgclipper-3b-v1 predicts clip scores at audio chunk level.

It's like an embedding model that measures similarity to the "good moment" vector.

This is the predicted clip heatmap of a live audio segment containing an actual clip created by a Twitch user.

[Marco Cimolai @CimolaiMarco]

Introducing sgclipper-3b-v1: an experimental audio-native clipping model that competes with Gemini 3.5 Flash, @OpusClip, @veedstudio and more.

Instead of using transcribed speech, it directly predicts clippable moments from an audio feed.

🤏 Based on @MistralAI Voxtral Mini
🚀 >50x realtime speed
♾️ Infinite audio length
💵 Extremely cheap, lower than $0.007/hr

https://t.co/LlpyB1LepZ

1/2

[merve @mervenoyann]

Why do people sleep on DSE multimodal retrieval models? 👀

They're just like ColPali, but highly scalable, fast and you can even make them more efficient with binarization or matryoshka with little degradation 🪆

I made a small collection of them (next ➡️)

[Jerry Liu @jerryjliu0]

Introducing vdr-2b-multi-v1 - a fully open-source visual embedding model for your most complex, multilingual documents 🧿🖼️📊

Document search and retrieval is increasingly becoming multimodal, as more and more companies are building agentic document workflows. Most existing RAG, OCR, IDP etc. etc. pipelines depend primarily on text processing. But that inherently causes you to lose information - and a lot of information is locked up in a visual format in unstructured docs.

We’ve invested heavily in tooling and abstractions to automate unstructured data processing, including LlamaParse, LlamaReport, LlamaExtract (upcoming), and more. This is a huge next step in that journey. Vdr-2b-multi-v1 allows you to directly embed image screenshots of different documents so that you can search retrieve image chunks as opposed to text chunks. It outperforms an existing qwen baseline and depends on a 500k query-image dataset we put together.

Huge shoutout to @CimolaiMarco, @disiok, and @LoganMarkewich.

Model: https://t.co/1lh8QD7FUY
Blog: https://t.co/mcPOzrSdbS

[LlamaIndex 🦙 @llama_index]

We've released a new multilingual, open-source visual embedding model and training set on Huggingface!

vdr-2b-multi-v1 produces single dense vectors for visual document retrieval, enabling efficient large-scale systems. Key features:
➡️ Trained on 5 languages (IT, ES, EN, FR, DE)
➡️ 70% fewer tokens = 3x faster inference for the english-only model
➡️ Matryoshka Representation Learning for flexible dimension reduction
➡️ Full training set is open-source

Learn more:
📝 Blog post: https://t.co/K6N0cqg9Fv
🤗 Model + LlamaIndex integration: https://t.co/eqCie7Bq4r

[Marco Cimolai @CimolaiMarco]

And more! Also releasing the full training dataset! With 500k high-quality samples, it's the largest open-source multilingual (🇮🇹 🇪🇸 🇬🇧 🇫🇷 🇩🇪) synthetic dataset for visual document retrieval.

Apache2, now on Hugging Face.
https://t.co/XHeqwbBHDD

[Marco Cimolai @CimolaiMarco]

Releasing vdr-2b-multi-v1: a multilingual (🇮🇹 🇪🇸 🇬🇧 🇫🇷 🇩🇪) embedding model for visual document retrieval, replaces mcdse-2b-v1.

Single-vector embeddings and MRL support!

https://t.co/eWQChXFlnk
1/2

[Nandan Thakur @beirmug]

Awesome work to see extending DSE by @xueguang_ma on the *multilingual* side for RAG!

DSE: https://t.co/e0aIiaWyXg

[Marco Cimolai @CimolaiMarco]

@mervenoyann Thanks for featuring my model merve! Also kudos to @xueguang_ma for DSE and training the base model

[Marco Cimolai @CimolaiMarco]

Introducing mcdse-2b-v1: a multilingual (🇮🇹 🇪🇸 🇬🇧 🇫🇷 🇩🇪) embedding model for flexible visual document retrieval.

Trained on MrLight/dse-qwen2-2b-mrl-v1 (Qwen2-VL) using the DSE approach. It's like ColPali but multilingual and single-vector.

- MRL: shrink embeddings from 1536 to 256 dimensions
- Binarization: 768d binary vectors keep 99% retrieval quality of the base 1536d float vectors
- vLLM support

https://t.co/ttQHM3FTJq

[Marco Cimolai @CimolaiMarco]

@Dorialexander @PetarV_93 Extremely well explained!
I was confused when you said that top_k is the number of tokens we keep in the final selection before sampling, but in the code the next_token is selected from all the probs. I was expecting it to select from the top_k most probable tokens

[Marco Cimolai @CimolaiMarco]

@hu_yifei 🤞

[Marco Cimolai @CimolaiMarco]

@juliuslipp That batched.dynamically is gold

[Marco Cimolai @CimolaiMarco]

@xueguang_ma @jobergum 82.9? This is incredible! I wonder if the benchmark datasets are not granular enough to show the advantages of using maxsim over image patches.

Or this shows that there's a lot of room for improvement in colpali, exciting!

[Marco Cimolai @CimolaiMarco]

@Dorialexander What about doclaynet? 80k labeled pages, and as far as I know the license in permissive

[Marco Cimolai @CimolaiMarco]

@carsondial For postgresql https://t.co/xZEPjDENFL

[Yusuke Wada @yusukebe]

HonoのドキュメントにAzure Functionsで動かすガイドが追加された！
https://t.co/2VI8qfDhEr

[Marco Cimolai @CimolaiMarco]

@Nils_Reimers @NielsRogge Not if you can run them on-device, only possible with open weights