@AptCareMy@Tony7onezzz@WatcherGuru Jensen wanted to help startups and reserve some chips for them. Now big players ofc wanna prevent that,
but all GPUs and will raise the prices
@fraserpricee very cool, how does this behave when you actually have some training objective as well and when gradients are flowing?
second question, how many nodes are there, which gpu do u need if every node is 10k params
@alon_rag Writing your own weights can go in several directions. Some of those direction are good for hyperscalers but others are good for llms that can be ran on home GPUs. Wrote more about it here https://t.co/hIJKLROhPl
@Un___common@serj_tf yes, they can clamp it even with architecture choice. Growing weights so much that they donโt fit on a single machine anymore. https://t.co/YsRU4RouIC