Maximilian Werk

[Elastic @elastic]

We’re excited to announce that we have joined forces with @JinaAI_, a leader in frontier models for multimodal and multilingual search. This acquisition deepens Elastic’s capabilities in retrieval, embeddings, and context engineering to power agentic AI: https://t.co/OAHtJumYuS

[Jina AI @JinaAI_]

V4 is multimodal embeddings, but V4-GGUF wasn't—until now. We've finally cracked how to generate multimodal embeddings using llama.cpp & GGUF.
We fixed two main issues. First, in the language model part, we corrected the attention mask in the transformer block so it properly handles image tokens. Second, there was a mismatch in how the vision transformer processes images: The PyTorch version reshapes pixel values to group them, then applies a single conv3d to encode these pre-grouped patches. But llama.cpp was doing something different—it was creating and encoding patches using two conv2d layers instead. We solved this by patching llama.cpp's projection layer to use a flattened version of the conv3d weights.

[Michael Günther @michael_g_u]

Jina-VDR, our large visual document retrieval benchmark, is now supported by MTEB ✨📜
I’m excited to see more models evaluated on it soon.
Leaderboard (Images > Jina Visual Document Retrieval):
🏆https://t.co/QSQ1BpeDWg
More info about the benchmark:
📚 https://t.co/gv7vCqfxJb

[Jina AI @JinaAI_]

Got a Mac with an M-chip? You can now train Gemma3 270m locally as a multilingual embedding or reranker model using our mlx-retrieval project. It lets you train Gemma3 270m locally at 4000 tokens/s on M3 Ultra - that's actually usable speed. We've implemented some standard practices for training an effective decoder-only embedding or reranker model with MLX: full/partial LoRA, InfoNCE, gradient accumulation, and streaming data loader. Plus MTEB integration for train-evaluation loops.

[Jina AI @JinaAI_]

Today we're releasing jina-embeddings-v4, our new 3.8B universal embedding model for retrieving text, images, visual documents and code. V4 achieves state-of-the-art retrieval performance on multimodal and multilingual tasks across MTEB, MMTEB, CoIR, LongEmbed, STS, Jina-VDR, CLIP, and ViDoRe benchmarks, with particular strength in processing visually rich content such as tables, charts, diagrams, and mixture of them. The model supports both single-vector and multi-vector embeddings.

[Jina AI @JinaAI_]

At #EMNLP2024 Miami next week? Join us on November 14, 2024, from 10:30 AM to 12:00 PM (Miami Time) for a BoF session on Embeddings, Rerankers, and Small LMs for better search. This 1.5-hour in-person session will bring together researchers in embeddings, rerankers & information retrieval; offering an excellent opportunity to explore recent advances in search foundation models, share your work with a specialized audience, and discuss emerging trends in search models in 2024. All EMNLP on-site participants are welcome to attend. Topics of interest:
- Multimodal, multilingual, cross-lingual, and cross-modal embeddings and rerankers
- Late-interaction models (e.g., ColBERT, ColPali) and late chunking
- Long-context embedding models
- Instruction-tuning for embedding and reranker models
- LLM-based embedding and reranker models
- Small language models for document reading
- Efficient and lightweight embedding architectures, attention mechanisms
- Zero/few-shot retrieval and adaptation methods
- Contrastive learning approaches for retriever
- Matryoshka representation learning, embedding compression and quantization
- Hybrid sparse-dense retrieval systems
- Task-LoRA, domain adaptation, OOD, fine-tuning for embedding models
- MTEB, LongMTEB, RAG evaluation metrics and benchmarks
- Embedding models for code and structured data
- Privacy-preserving embedding techniques

[Jina AI @JinaAI_]

One cool thing about ColBERT-based search compared to the cosine-based vector retrieval is that you get interpretability for free as a byproduct of the MaxSim computation. It's kind of like the Lucene highlighter, letting you grab the most relevant snippets from a long document to show users where their query matches. With Jina-ColBERT-v1, which supports up to 8K token length, released by us earlier this Feb., the visualization of the late interaction between a query and a document is almost... artistic. The video shows the late interaction between the query "Elephants eat 150 kg of food per day." and the Wikipedia article about "Indian Elephant". Darker colors indicate stronger semantic matches. The darkest area corresponds to "The species is classified as a megaherbivore and consume up to 150 kg (330 lb) of plant matter per day." from the original article.

[Jina AI @JinaAI_]

🤨 This fine-tuning is too good to be true? Just tell us what you want your embeddings to excel at, e.g., car insurance claims, financial news, Spanish dialogs. Specify your wish in a prompt; and remember this is your only input to our API. In about 30 mins, we then deliver a ready-to-use, fine-tuned embedding model that can be loaded via SentenceTransformers. Behind the scenes, we take care of everything else: from generating useful synthetic data to managing the train-eval-test ML workflow, and finally, uploading the fine-tuned model to the @huggingface Hub. Yep, under this very minimal UI abstraction, so much happens!

[Jina AI @JinaAI_]

Start with a boost! 🚀 Get a massive head start with our embedding API - now offering 1,000,000 free tokens for each new API key, no credit card required. Ideal for personal & commercial projects. 8K length, high scalability, easy integration. Dive in now! https://t.co/xCbDQPUdxY

[Bo @bo_wangbo]

jina-embeddings-base/small-en-v2, the OSS embedding model handle 8k sequence, brings on-par performance against ada-embedding-002: https://t.co/S7mM41hAIY… fyi
@JinaAI_ @ClementDelangue @hxiao

[Maximilian Werk @maximilianwerk]

@fasc1nate Kids & focus

[Bo @bo_wangbo]

still doing some code clean up, but we're almost here :)

[atitaarora @atitaarora]

All set to know more from @bo_wangbo and @maximilianwerk !! #bbuzz23

[Isabelle Mohr @isabelle_mohr]

Get ready for a new search journey! 🗺️

We're enhancing your search experience by putting the most relevant results at the center of focus. Discover the Searchscape at https://t.co/vIi7AU8riq
Sneak peak of its features in the thread!
#search @JinaAI_

[Isabelle Mohr @isabelle_mohr]

Looking for a fresh, visual solution to your search needs? 👀

Explore the Multimodal Search Map: our map-based search interface, ready to be customized to unique datasets and use cases. Really fun for #ecommerce🤩

Dive in 👉 https://t.co/CcEswaj9rK

[Isabelle Mohr @isabelle_mohr]

Yesterday @maximilianwerk and I presented a project we have been working on: The Multimodal Search Map!🌍🔍

Here is a sneak peak from the talk 👀

Thanks @renekrie and @deliveryherocom for making it possible!

[René Kriegler @renekrie]

Very excited to announce another #Search Tech #Berlin Meetup, 25th May! Talks on vector search in visual domains (Isabelle Mohr, @maximilianwerk) and on content dedup using vector embeddings vs keywords (@zpapierski) https://t.co/WG1fBpshWa thanks @deliveryherocom for hosting!

[Bo @bo_wangbo]

We will bring some exciting news in our presentation;)

[@[email protected] @berlinbuzzwords]

Deep learning for search has become a hot topic, while pre-trained neural nets do not function as well as expected. At #bbuzz 2023, @maximilianwerk & @bo_wangbo will discuss the algorithms behind model fine-tuning, and how to scale it up. https://t.co/wJkOXNTMNk

[Maximilian Werk @maximilianwerk]

@chipro Training or finetuning LLMs in-house, allows modelling of training data close towards your problem. This reduces zero-shot capabilities, but hugely benefit specific real world problems. Coming from Operations Research, I wonder why this modelling view is often missing in ML!?