@willcb This becomes 100x more relevant now that automated research is becoming mainstream, and the number of experiments you can run is increasingly the main bottleneck.
wild that people use gpt as their “search engine” and wait 10s for an answer. google cooked with AI Overviews, it’s so fast and factually right 99% of the time. sure, use gpt when you need einstein-level reasoning, but other than that ? no way
@gabriel1 Can relate to that, personally, I highly see correlation between motivation and momentum, and momentum can be generated in so many ways, different tasks that are not related to your desired objective (a la, learning linear algebra) - which will create drive to achieve your goal
@JustinLin610 On what hardware did you tested this?
I mean, this model should be extremely efficient for compute bound hardware which I assume this is what you tested ?
Not sure this will work better for hopper/blackwell
@MaximeRivest I think the intresting question is - is the pruned model better than a non pruned model at the same amount of compute / memory / model size of the pruned model?