Burla is a self-hostable compute platform that vertically scales hardware available to each task live while the program is running.
This frequently more than doubles compute efficiency, and eliminates out of memory errors!
Here's how it works:
https://t.co/CvgFsctGtN
If you're using Cursor, Codex, or Claude to analyze 10+ TB datasets inside your private cloud, you're probably using WAY more compute than you need to, usually over twice as much!
We're building a distributed computing framework for AI agents that automatically fixes this π
The answer is automatic vertical hardware scaling.
When memory pressure goes up, the available memory should too!
The best part is you can do this today for free self-hosted in your cloud with Burla.
Here's how our system works:
https://t.co/CvgFsctGtN
OOM errors really suck, especially when they only appear two hours into your giant distributed compute job!
Most dev's aren't aware this is effectively a solved problem. Here's how we helped one customer eliminate OOM's without touching any codeπ
Currently we average <20% utilization across all actively rented cloud compute.
Entering an era of compute constraints, efficiency will be more important than ever.
This is why we've been so focused on dynamic hardware. You can achieve >90% utilization automatically!
@signulll "The" love of your life is a misnomer.
There are 8 Billion people on this planet. Any individual probably has a million+ possible "loves of their life".
It only takes 2 commands to install our OSS parallel computing framework in your GCP environment:
- `pip install burla`
- `burla install`
Then you can scale any Python function to 1000s of VMs using one line of code: https://t.co/oTVcb9oGx1
People have no idea that scaling Python to 1000s of VMs in their private cloud is so trivial a total beginner can do it now.
Just got off a call with a bioinformatician who was struggling to scale past one big VM.
We got their pipeline running on 100VMs within the 30min call!
Full technical breakdown in the blog post:
https://t.co/CvgFsctGtN
And if you're sick of dealing with with complex configs, memory errors, and inefficient frameworks please send me a DM!
I wrote up a piece explaining a crazy idea we have at Burla:
You should not need to estimate how much CPU or RAM your job needs.
In fact it's better if you don't! Here's whyπ§΅
Not only is this much easier,
(no more guessing, no more memory errors)
It's often more than twice as resource efficient, on what are often some of the most expensive jobs happening inside a company (big data processing)
@galdayan1895@a16z@speedrun We make your CPU/RAM utilization >90% automatically for any big data workload, enabling you to do 2-5x more with the same compute.