What if you could use all the computing power in the world to train a shared, open source AI model?
Preliminary report: https://t.co/4Rhd0ddp9z
Nous Research is proud to release a preliminary report on DisTrO (Distributed Training Over-the-Internet) a family of architecture-agnostic and network-agnostic distributed optimizers that reduces the inter-GPU communication requirements by 1000x to 10,000x without relying on amortized analysis, and matches AdamW+All-Reduce in convergence rates. This enables low-latency training of large neural networks on slow internet bandwidths with heterogeneous networking hardware.
DisTrO can increase the resilience and robustness of training LLMs by minimizing dependency on a single entity for computation. DisTrO is one step towards a more secure and equitable environment for all participants involved in building LLMs.
Without relying on a single company to manage and control the training process, researchers and institutions can have more freedom to collaborate and experiment with new techniques, algorithms, and models. This increased competition fosters innovation, drives progress, and ultimately benefits society as a whole.
This research is thanks to the hard work of @bloc97_@theemozilla@apyh__@UmerHAdil.
We invite researchers interested in exploring this area to join us in our quest.
🎉 WE DID IT! 🚀
We’ve reached our goal of $500,000 in only 12 days!!! 💰
Incredible! THANK YOU SO MUCH! 🙌
Here’s a letter from Ted Peterson, our CEO & Creative Director of Once Lost Games, to thank all of you! ✉️
AND it doesn’t stop here either! We still have 17 days left!
The Baron’s Men is a gang of thieves with no true affiliation to the Baron himself. Their thievery plagues the merchant shops on the waterfront. As the port authority fails to stop them, many victims believe they are either in their pockets or calling the shots themselves. #RPG
@ScheffersVictor@OnceLostGames Hard to tell this early on, but generally aiming for the game to run on a GTX 1060ish min spec (~4Tflops). With scalability settings a good optimizations it should run well on a wide gamut of hardware.
@raysan5 I find you can write things that are useful to you and have them live in the open. It might be useful to someone else or not. Optional. OS falls apart when you, as the author, expect more beyond it being free. As free you also have no obligation to support it beyond your own use.
@josephepstein@ammaar@Replit You say something like 'goung forward provide only the necessary modification instead of the full source code' and it will do just that. You can also just type . To continue a cutoff prompt.
Meet the Dev: Kyle. If you've watched our Daggerfall play session, then our resident Zhumir enjoyer needs no introduction. If you haven't, know that Kyle is a talented programmer who brings a lot of passion and drive to OnceLost Games while working on many key systems for WR.