I've been using Brokk to write Brokk for months so I have Opinions on the best models to use for code.
First: if you put a gun to my head and made me choose just one, it's Gemini 2.5 Pro.
Fortunately I can and do use other models when they are a better fit. 🧵
@Ohrenje@bandmaid they did say during the show today that Nina will be with the Japanese fans while they are away from Japan so there definitely will be something outside of Japan 🤞😁
Support for ColPali image search by @ManuelFaysse, @sibille_hugues, @tonywu_71 et al is now in ColBERT Live! That means you can build a compressed, larger-than-memory ColPali index on any vector database.
Things I learned in the process (link to article at the end) 🧵
"JVector: Cutting-Edge Vector Search in Java" an https://t.co/84MwkKWT7N conversation with @spyced is ready to listen: https://t.co/AIsGJ2vUIo #jvector#cassandra#java#podcast
Can your vector database index
- All of Wikipedia?
- On a laptop?
- In under 6h?
- Without compromising accuracy?
JVector can. Article and live demo links in thread.
I wrote an explainer on vector compression for ANN indexes.
TLDR: binary quantization (BQ) is fast but too lossy for most vectors. Product quantization (PQ) is more accurate, more effective (can compress up to 64x), and faster (when you include the cost to rerank) for most source models. Link in thread.
We love the new streaming support in Assistants API so we spent the last 5 days implementing it.
Today we're announcing support for the new streaming runs functionality in our open source, drop-in compatible version of Assistants API--Astra Assistants.
Try our service for free or run it yourself. We support multiple LLM and embedding providers using @LiteLLM
DataStax 🤝 NVIDIA. By integrating #NVIDIA microservices (NIMs), we're able to deliver high-performance RAG solutions with 20x faster embeddings at 80% lower cost.
Learn more 👇 #GTC24#VectorDB https://t.co/IOM2SZ97bn
Last week we open sourced the server code that we run for our Assistants API service. Now anyone can see exactly how it works or run it themselves on the cloud or on premises. Apache 2 license.
It's a drop in replacement for OpenAI's Assistants API but it persists your data (files, threads, messages, assistants, etc. in astradb / cassandra and uses astradb / cassandra vector indexes powered by jvector for ANN based recall), it supports streaming messages, and allows you to use third party LLMs and embedding providers.
This takes us one step closer to fully open source Assistants and GPTs.
Let us know what you think!