with the release of Jina's multilingual colBERT model a few days ago, this is now much easier to use, along with vector databases like qdrant, lanceDB, and RAGatouille that support it (https://t.co/kNbZDIgoev, https://t.co/NgKjd1VHP1)
colBERT was developed at Stanford by @lateinteraction and colleagues to address these issues, and uses per-token embeddings that can scale by allowing you to pre-compute the representations of your corpus, and do the interaction calculations later
@ArtificialAnlys has some pretty great benchmarks on LLM quality vs. cost - llama 3.1 70B, gemini 1.5 flash (million context window), and GPT-4o mini are all contenders in the green sweet spot
favorite quote from the @NoPriorsPod with @magicailabs's CEO: "I would love to have @terrence_tao in my computer... but I would then still need to run Tao for a year of of human thinking time [to solve an unsolved math problem]"
@modal_labs is the closest thing I've found to facebook's internal FBLearner ML training system, but it's even easier to use (and serverless, so can scale elastically with GPUs)
at first glance, it may seem that it's easy to swap out nvidia GPUs for AMD ones, after all PyTorch abstracts away the implementation details, right? but when torch.compile barely works half the time on nvidia GPUs, do you really want to risk it on others?