Contributing to OSS has been one of the best decisions I've made for my career
These are the few the inbounds I received earlier this week.
Making every contribution mindfully, building skills relentlessly and the profile builds itself
Getting noticed eventually pays off :)
Contributing to OSS has been one of the best decisions I've made for my career
These are the few the inbounds I received earlier this week.
Making every contribution mindfully, building skills relentlessly and the profile builds itself
Getting noticed eventually pays off :)
Good news! For everyone who wanted in, we're opening this up to everyone
We'll onboard members in batches, hence we will need some time :)
Planning to have:
- Weekly discussions
- Live lectures on inference
The vision is simple: learn from the community and give back to it!
Found an inference squad to discuss resources, ideas, projects, weekend reads, tech news and opinions
It's a super tinyyy group of people learning inference. We're all at different stages, but if you want a community to talk to every day, join us :)
Let me know if you want in
Found an inference squad to discuss resources, ideas, projects, weekend reads, tech news and opinions
It's a super tinyyy group of people learning inference. We're all at different stages, but if you want a community to talk to every day, join us :)
Let me know if you want in
I'm going into a 2-week lock-in period, setting myself a challenge to own a feature end to end in the inference routing and serving layer across
- AIBrix
- llm-d-router
- KServe
I have a history of OSS contributions, so let's see what it takes!
I'll be making the work public :)
The post blew up and we got much more response than we anticipated, so give us some time to review the profiles :)
We still plan to keep the group small and manageable and add more members next week
Meanwhile, I'll keep dropping the best resources and real inference work here
Found an inference squad to discuss resources, ideas, projects, weekend reads, tech news and opinions
It's a super tinyyy group of people learning inference. We're all at different stages, but if you want a community to talk to every day, join us :)
Let me know if you want in
I am sharing a list of repos for those who are just getting started with inference and feel overwhelmed by the number of projects in this ecosystem
These are projects that I have contributed to, and each operates at different layers of the inference stack
@smykx@ucalyd@SarvamAI yes, once the answer starts streaming, speed isn't affected.
But the wait before the first word is, a server in US is about 200 ms further away each time you talk to it, and a fresh connection needs 2-3 of those trips, so about half a second before anything shows up
I found an LLM serving and inference engineering study group on GPU MODE.
Give it a read if you're just starting out in inference
it's well written, but it's all prose with no code to run or profiler numbers to reproduce, so pair it with hands-on practice