Jayant

AI Inference × HPC

GPUs are cool :)

[Jayant @55CncAe]

@iTanayVaswani Depending on your definition of "retrieval", hybrid retrieval pipelines have been around for a while (especially for candidate generation in recommender systems)

[Jayant @55CncAe]

@vtabbott_ This is awesome!

[Jayant @55CncAe]

@SemiAnalysis_ @deepseek_ai Well... there goes the hope for NVMe pricing reducing any time soon 🥲

(But GG to the lads at @deepseek_ai, cool arch 🔥)

[Jayant @55CncAe]

@nrehiew_ More like.. Decoder-Decoder :)
https://t.co/PR8Ym3ZNUX

[Jayant @55CncAe]

@Aayush__Saini_ @LinkedIn @X Love your posts about vLLM. Would love to read more of those in that direction :)

[Jayant @55CncAe]

@__acbraingenome Just when you think there's no hope, a Zhang appears with just the info you need. (This time with model arch details of GPT 6 Astra, nearly a month and a half before the launch 💀)

[Yifan Zhang @yifanzhang_]

Heard that some frontier models are basically a 48-layer transformer looped twice (48L x 2).

Now we are introducing DeepLoop: Depth Scaling for Looped Transformers (https://t.co/jtxllo9aFF),
making the loop transformer stable and scalable!

[Jayant @55CncAe]

@vibespersecond Ahh, so I suppose they're targeting something like "adapter bridges for local models" to be sold the big model orgs.

I wonder how long before we get
api.<your_fav_model_co>.com/v1/latents/
endpoints :)

[Jayant @55CncAe]

@prajdabre Anytime a Google Research employee posts about banana, I genuinely can't tell if they're alluding to the image gen model or not 😭

[Jayant @55CncAe]

@shrav_10 Depends. What's your vector storage like? Does it support true prefiltering?

- Yes? (Example: Postgres + pgvector or milvus) Map each user id to an allow list and filter based on RBAC before performing ANN.

- No? Create per RBAC role Vector indexes and use accordingly.

[Jayant @55CncAe]

@__acbraingenome HF's Annual Revenue ×80 💀 (they deserve it tho)

[Jayant @55CncAe]

@viditchess Get an RTX PRO 6000 Blackwell (Much better memory bandwidth than spark, lot more VRAM than 5090)

[Jayant @55CncAe]

@viditchess @QwenDevs @Alibaba_Qwen :)

[Unsloth AI @UnslothAI]

Qwen3.8-Flash can now be run locally! 🔥

The 125B MoE model outperforms Claude-Opus-4.6 (Max).

Run on 75GB RAM via Unsloth GGUFs.

Qwen3.8-Flash-Next enables CPU RAM / unified mem setups to deliver near VRAM speeds.

Guide: https://t.co/LLMclyJTeL
GGUF: https://t.co/vXkjO3W0fj

[Jayant @55CncAe]

@__acbraingenome Positional encodings ? Sinusoidal Vs RoPE ?

[Jayant @55CncAe]

@__acbraingenome > "being able to read a research paper effectively"

True, but this helped:
https://t.co/i2P9weFFmN

[Jayant @55CncAe]

@MiaAI_lab So is it GLM?

[Unclecode (Hossein) @unclecode]

Everyone guessed who made Ox Alpha today. I fingerprinted it instead. One page, 9 infrastructure probes: tokenizers, error codes, hidden templates. I ran • the mystery model against 12 suspects. One family matches every tokenizer test: GLM. Plumbing does not lie. Tool in reply, bring your own keys. Link in reply

[Jayant @55CncAe]

@prateek_0041 True, can't wait to try 5.6 Sol ultrafast (they claim it runs at ~ 750 tokens/sec with @cerebras)

[Jayant @55CncAe]

@vibespersecond >"...having seen..." 👀

Dayumn

[Jayant @55CncAe]

@vtabbott_ https://t.co/k6jNCZMEgc

[Jayant @55CncAe]

@SaiyamPathak Not me, but a cool problem:
"Good model, but not multimodal? Graft the vision encoder of another model"

Something that the talented folks at @baseten did to give Kimi's vision to GLM5.2:
https://t.co/KO3DVG0ULX