Probably the coolest implementation of a circular buffer I've ever seen:
https://t.co/Al2SAmKPnN
Essentially they mmap 2 contiguous regions of virtual memory to alias the same data. Then they can call memcpy 1 time and have the hardware automatically handle the wraparound logic.
We're seeing a massive spike in demand at @ThunderCompute. New record highs every hour.
Please bear with us for any service interruptions, our team is working hard to keep everything online!!
@nicbarkeragain Imagine trying to pip install a requirements.txt on a project that hasn’t been updated in a year. Honestly that’s enough of a reason not to write anything major in python.
We’ve been absolutely cheffing and have a ton of updates for you:
Performance
- Improved performance for large models: 4-6x faster data transfers
- ~75% faster initial GPU connection
- Expanded compatibility with data science and ML workloads
- MUCH faster template launch times
Console
- Added the ability to SSH in-browser
- You can now launch templates directly in-browser without code!!
- Improved instance management: bulk actions, modify instances, loading states
- Clearer chart data display and visibility into billing
VSCode extension
- Some users mentioned a security warning when installing the extension, we’re working with our Microsoft contacts to resolve this.
- Snappier performance, bug fixes, UI improvements
As always your feedback is incredibly helpful as we continue to improve. Onwards and upwards!!
Apple intelligence sucks. Can't even run nvidia-smi.
Apple uses weak, small LLMs.
Big LLMs are better.
Thankfully, I got 80GB of VRAM on my iPhone with @ThunderCompute
nvidia-smi ✅