Archit Pandey

LLM architecture, inference, system design | Thoughts my own!

[Archit Pandey @pagedattention]

@thsottiaux Still pretty bad at front end. Great for backend so far, creates less slop compared to Opus.

[Archit Pandey @pagedattention]

@pmarca Why does this read like a post from the future about the 2020s…

[Archit Pandey @pagedattention]

@VictorTaelin This conversation with 5.5 is great training data. With Codex gaining so much traction lately, we can expect even more high-quality data to be collected, meaning future models (GPT-6? 👀) can perform better on these kinds of tasks.

[Archit Pandey @pagedattention]

Been noticing Grok not responding to comments anymore. compute shortages? @grok

[Archit Pandey @pagedattention]

@karrisaarinen Someone needs to build an MCP for CVS Photo.

[Archit Pandey @pagedattention]

@kakashiii111 GLM-Image by @Zai_org was trained entirely on Huawei chips. Chip smuggling is no longer a factor.

[Archit Pandey @pagedattention]

@pmarca External attackers don't magically have access to proprietary codebases. Finding exploits when you can literally read the code is a totally different game.

[Archit Pandey @pagedattention]

The image does look great. Tried the same prompt with Grok, Gemini and ChatGPT (in that order).

[Mustafa Suleyman @mustafasuleyman]

The team really killed it on photorealism with this model. While testing we found out ice is a great barometer for this - try this glacier prompt.

A glacier wall towering like a cathedral interior, deep blue ice with light refracting through layers, tiny human figure at base for scale, cinematic, cold mist in air, hyper-real detail

[Archit Pandey @pagedattention]

@aakashgupta @GaryMarcus Funny thing is the paper was published on April 28th 2025.

[Archit Pandey @pagedattention]

Link to blog: https://t.co/xEhVqtYvjv

[Archit Pandey @pagedattention]

With the recently released TurboQuant paper, it’s clear that LLM research is moving towards efficient inference. TurboQuant proposes a technique to shrink the KV Cache by 6x, allowing for longer contexts and/or serving the same context with much less HBM.

[Archit Pandey @pagedattention]

@chamath @andrewdfeldman @cerebras about the 4th point, there’s the aws & cerebras collab where prefill runs on trainium and decode runs on cerebras.

[Archit Pandey @pagedattention]

the cli vs mcp debate doesn't matter if you're creating a cli just for consumption by agents. will the cli be more general purpose? yes. will it result in lower token usage? no.

[Archit Pandey @pagedattention]

@ArtificialAnlys @Skywork_ai curious why video with audio is priced lower than video without audio?

[Archit Pandey @pagedattention]

the primary idea that makes cerebras chips so fast at inference is that the decode phase (the big lift) is memory bound (model weights, kv cache, lora weights). having a large wafer lets you have large on-chip memory, thus making inference fast.

[Archit Pandey @pagedattention]

@akifmalik Did this ever happen? The road quality is still horrible

[John Lettieri @LettieriDC]

This stuff is beyond pathetic.

The H-1B program is deeply flawed and should replaced with a proper skilled immigration system.

But get real: the “flood” of 85k H-1Bs per year—representing around 0.05% of the U.S. labor force—is going to destroy precisely nothing.

[Niko McCarty. @NikoMcCarty]

Indian gov't is buying a subscription to 13,000 academic journals, and then making them all available to "18 million students, faculty, and researchers" for free.

The cost is $715 million over 3 years. It includes Elsevier, Nature, and AAAS.

Have any other countries done this?

[anand mahindra @anandmahindra]

In the thick of the night, @DRDO_India launched a Long range Hypersonic missile with a speed of Mach 5+ & a range of 1500 +km from Dr. APJ Abdul Kalam Island

In an increasingly volatile world, exactly what we need

#DRDO moving at hypersonic speed…

[Archit Pandey @pagedattention]

WHO considers AQI < 100 safe, and here we are at 10x those levels.

Farmers workaround satellite checks by burning crops at off peak periods, politicians workaround media outcry by underreporting numbers.