Really fantastic work. Im really curious what kind of CPU/RAM offloading, KV cache tweaks, or other hybrid config to handle the long contexts and take advantage of your CPU and DDR4
For the 2x3090 NVLink + 3970X + 128GB setup with the INT4-BF16 Qwen 3.6 27B + MTP=3 + FlashInfer, what exact vLLM launch flags / tensor-parallel settings are you using for the modified Cursor 16x concurrent processes?
Much appreciated ๐๐ผ
@jun_song Tbf thats a pretty challenging question for anyone to be up to date on.
Keeping up with advancements in local AI is like a full time job, its moving so fast now that the right answer could change by the hour... I love to see it ๐ฅฐ
If polymarket would let me bet on whether or not there will be a guy camped out scrolling tik tok on the pec-deck/rear delt machine when I need it, id be rich af
i hope he does, but i think he took a step back from that idea for the same reason he closed dms. he is trying to combat the misinfo and help people, but realized his community (like his dms) would quickly become "depressing" and an onboarding/tech support chat . he obviously doesnt mind helping people, but i think he is really craving a group to collab with other people who already know what they are doing so they can actually help eachother advance. and thats totally understandable, especially as fast as things are changing, its probably essential
@Abomination81@bored2boar same, youre literally the only one ive found that isnt pulling some bs. i need to make a video or something as many times as ive had to repeat the same explanation to friends about how to tell a post is total bs/larp/scam. any you already follow good on these subjects?
@Abombination81 Ive been running something you might find interesting/ useful if you arent already. Its not this (though adjacent) and no crazy token burn like this, and ive been accumulating the data for about a month. Would love to discuss and share via dm
@Abombination81 Hey man first I wanna say thank you so much for the info you share. Its insanely hard to get good info in this space, and what you share is legitimately top tier. Tremendously appreciated
Wanted to ask you about some quick stuff privately, but it won't let me dm
WELCOME TO HAPPY HOUR ๐ฎ
The happiest hour, LIVE on Twitter Spaces.
HEREโS WHAT YOUโLL GET:
- Q&A, weekly updates, and good music
- Best time to lock a rate (base DPYs going crazy)
- Special guests, rewards, and challenges when you least expect them
- Hosted by @0xBelugaa and @worldofwhiteboy
Join our community here:
https://t.co/fhdpv4DLti
Wizzatron live DPY updates here:
https://t.co/eZOdJC9bN3
It pays to be punctual โฐ
LOCK IN MY WIZZA!!!
$MYSTIC IS NOW LIVE ๐ฎ
CLAIM YOUR TOKENS HERE: https://t.co/pUcKMWVuzi
PLAY THE GAME HERE: https://t.co/mAMDvqIE38
CHART: https://t.co/NxNb1RqzK9
CA: mysticrSzUfD2pz4RayXF5oEHGfNNAFsSY3z5hTQZSN
MagicSwap: https://t.co/nPvhW7Zy5Q