Alexey Kuntsevich

LLM engineer. Building trust and cooperation between AI and humans. vi/vim

[Alexey Kuntsevich @jezzarax]

We tried to tell you what the engs in your IT departments were telling you for decades

[Jason D. Clinton 🔸 @JasonDClinton]

@hinapansari yes. it was always going to be this way. we tried to warn the world.

[Alexey Kuntsevich @jezzarax]

Bought one for 2700€ in March. 😭

[Mia @MiaAI_lab]

For reference, this is what I paid for my 4 TB 128 GB DGX Spark in July.

Madness!!!

[Alexey Kuntsevich @jezzarax]

@tritlo So what category does pizza belong to?

[Alexey Kuntsevich @jezzarax]

I hate such plot cliffhangers!

[Bad Pote @bad_pote]

La vérité, c’est comme ça maintenant.

[Alexey Kuntsevich @jezzarax]

@kyzoroX I have and actively use both, Apple silicon and GB10. Having llama.cpp for some chatting on a plane, sure. For anything serious you want vllm or sglang. Take a look at the token count of any coding harness prompt, or how much tokens an image turns into. Prefill matters.

[Alexey Kuntsevich @jezzarax]

I’m really curious to see run metrics and evals. Also maybe I missed it, but couldn’t find how many tokens were processed during the run. Without that, the post is just bragging with the number of GPUs

[CoreWeave @CoreWeave]

In the MLPerf® Training v6.0 round, we delivered near-linear scaling efficiency with our NVIDIA GB300 NVL72 4,096-GPU cluster connected with NVIDIA Spectrum-X Ethernet.

We were the only team to push @nvidia GB300 NVL72 past 2,048 GPUs on this benchmark, and delivered amazing results at every scale.

Full breakdown below! https://t.co/hxiPCh2fmQ

[0.005 Seconds (3/694) @seconds_0]

As of today, Hetzner has tripled their prices. Only existing instances are grandfathered in.

Good luck out there. Access to the future will not be free.

[Alexey Kuntsevich @jezzarax]

Maybe it’s just shy? A new emergent behaviour?

[Kalshi @Kalshi]

BREAKING: Anthropic says its newest AI model is too powerful to release to the public

[Alexey Kuntsevich @jezzarax]

Everyone’s mad about AI slop. Meanwhile a human at LinkedIn shipped a bug years ago that switches your whole UI into the language of whoever sent you a connection request. No AI has fixed it. So I keep learning Arabic, Chinese, Swedish and other languages through random clicks.

[Alexey Kuntsevich @jezzarax]

I want this question to be made into a graffiti on a wall. It’s art.

[Muhammad Hammad Khan @hammad_khan23]

@kunchenguid How do you write tests for your code then?

[Sebastian Raschka @rasbt]

Always back to the basics:
LatentMoE was probably inspired by MLA, which was inspired by LoRA, which was inspired by SVD, which was inspired by eigendecomposition.

[Alexey Kuntsevich @jezzarax]

As we all know, demand and price have inverse relationship. Your next TODO app is not going to build itself!

[Bojan Tunguz @tunguz]

Try to build as much code as possible over the next few months. The prices you are seeing now for AI will probably not last too long.

[Alexey Kuntsevich @jezzarax]

To whoever complained about openai models being lazy with counting! On a serious note, codex feels like drunk since today afternoon. A lot of dumb stuff, missing things from AGENTS.md and generally less coherent than before.

[Alexey Kuntsevich @jezzarax]

@joshu Please ask it to make popcorn

[Alexey Kuntsevich @jezzarax]

I generally don't like papers of a kind "look at the cool prompt we found", but https://t.co/UJvssiBz9s is something deeper IMO. It makes the LLM run a "Family feud"-kind of thinking over potential response options and seems to work better than "give different options".

[Alexey Kuntsevich @jezzarax]

Do you remember good old times when the joke was that AI is a bunch of “if” statements? How do we call “if” statements now to not confuse them with a bunch of calls to openai endpoint?

[Alexey Kuntsevich @jezzarax]

@ChujieZheng @deliprao 72B dense pls, the community will handle the other sizes from there :)

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

The writeup just got published. You can read it here: https://t.co/sWSNkbCEhm

[Andrej Karpathy @karpathy]

Software horror: litellm PyPI supply chain attack.

Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords.

LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm.

Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery - Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks.

Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages.

Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible.

[Alexey Kuntsevich @jezzarax]

@osanseviero T5 is back! Any chance for larger versions?