kaushik raj

7x Hackathon Winner | National MSME Grantee 🇮🇳
Building Applicable — Web-based IoT Simulation & Firmware Generation.
Crafting deep tech

[kaushik raj @kushikraj07]

Been building Applicable for last 2month , A platform for IOT enthusiastic! We are pushing code every day , we are trying out everything which can help us to gain the trust of the user ! That it ! #Indianstartup @elonmusk @nikhilkamathcio @Hiteshdotcom @StartupGrind

[kaushik raj @kushikraj07]

@Keshavdotdev we use to write

[kaushik raj @kushikraj07]

@RhysSullivan nope i dont thi k so

[kaushik raj @kushikraj07]

@mohitsatitwt us

[kaushik raj @kushikraj07]

FlashAttention made LLMs 2-4x faster without changing a single number in the math.
Standard attention computes a huge N×N score matrix and writes it to GPU memory. At 32K tokens that's a billion scores per head, written out and read back.
FlashAttention never writes it. It splits the work into small tiles that fit in the GPU's tiny on-chip SRAM, computes softmax piece by piece, and only writes out the final result.
Same output. Way less memory traffic.
The lesson: on modern GPUs, moving data costs more than math. The fastest code is the code that touches memory the least.

[kaushik raj @kushikraj07]

@pmitu @themishra4402 everything have limits

[kaushik raj @kushikraj07]

@silencioNetwork End user get zero payment

[kaushik raj @kushikraj07]

@captaink99 Lol stop yapping

[kaushik raj @kushikraj07]

@tableandledger @jack true

[kaushik raj @kushikraj07]

@3r01x @jack that need mannual control over things

[kaushik raj @kushikraj07]

@IndusPriestKing @TeksEdge 😂 lol

[kaushik raj @kushikraj07]

@itsakaashhh @EmersonSklar @giolaq @anisha_malde @devpost @AmazonAppDev @raina_sai howp this post reachs to then

[kaushik raj @kushikraj07]

@amybuildsAI Hey how to contact you ?

[kaushik raj @kushikraj07]

@ayy_him_anshu23 @aayushchugh poor kid

[kaushik raj @kushikraj07]

@aayushchugh dont woory people will relised this soon

[kaushik raj @kushikraj07]

100%. A missing memory makes the agent ask again, but a wrong one makes it act on it confidently.
The fix that's worked best for me is to store every memory with a timestamp and its source, and to let newer facts overwrite older ones instead of piling up next to them. Most "wrong memory" is really just stale memory.

[kaushik raj @kushikraj07]

Curious how the memory works here. Even at 2 bits, 763B weights are about 190 GB, so I'm guessing the 12 GB GPU holds the attention layers and shared experts, and the routed experts stream from system RAM or NVMe on demand?
If so, decode speed comes down to PCIe or SSD bandwidth, not the GPU. What tok/s are you seeing, and how much system RAM does it need?

[kaushik raj @kushikraj07]

@Jhaddix No one will ban local ai dw

[kaushik raj @kushikraj07]

@mhelcig @DAlistarh @adotdad @coldniko These engineering are the teal goldmine

[kaushik raj @kushikraj07]

@SidPatankar1 Bruh saw ur account it worst tech post i can say u do , grow up dont be cry baby

[kaushik raj @kushikraj07]

@MaheswarBora @ThePrimeagen changing the entire tech stack ?