Gleb Morgachev

Gonka Co-creator
ex DS @Jetbrains

[Gleb Morgachev @gmorgachev]

If you thought the biggest trust issue in the AI era was a provider giving you a dumbed-down model instead of a larger one, it’s time to reconsider.

@princechaddha shows how a fine-tuned or abliterated version of a model can be shipped with a backdoor that makes the agent using it leak users’ data and credentials.

Today, it’s a scary demo. Tomorrow, it may become reality. Your favorite AI aggregator might unknowingly route your requests to a provider with a backdoor-equipped model, which, on the surface, will perform just like the original.

Unless we start treating model validation or TEE-like solutions as must-haves at the AI-provider level, we’re living as if in the internet’s pre-SSL era.

https://t.co/17vlWLZhPB

[pwnmachine 👾 @princechaddha]

How abliterated models can get you pwned 👾

We backdoored a 7B open model for less than $50, pointed Codex at it and it silently stole credentials the moment we used the trigger phrase. Success rate was 100% with zero false triggers on normal user prompts.

Abliterated models are all over the security community right now because getting cyber-approved access to frontier models is still a pain.

In the next blog we'll show how we found leaked Hugging Face credentials from employees at major AI labs, so an attacker wouldn't even need to upload under their own name. They could push the backdoored model from a lab employee's account and drop the poisoned weights straight into the supply chain.

[Gleb Morgachev @gmorgachev]

Couple weeks ago, @mtvnastya and I gave a talk at UC Berkeley's ML Club @BerkeleyML about decentralized AI. We discussed access to AI compute, verification of work, how Gonka approaches these problems, and open research questions students can contribute to.

https://t.co/jTXMynnx1H

[Gleb Morgachev @gmorgachev]

Can collective intelligence beat centralized labs?

One of the major bottlenecks in AI model training and inference is moving data between GPUs when a model’s weights are split across multiple machines.

This is where centralized labs have a clear advantage. Inside massive data centers, their hardware is tightly linked by high-speed interconnects.

Distributed networks can aggregate a similar scale of compute across machines in different parts of the world. But splitting model weights across those machines is impractical because network connections are much slower.

@gensynai offers an interesting approach to this problem. They recently released a prototype called IR3DE-AXL, which explores inference across distributed experts.

Instead of requiring nodes to run massive LLMs, the system allows them to contribute smaller, specialized models for coding, math, and other domains. Each node can also act as a client, using a lightweight local router to send each prompt to the right expert.
https://t.co/AIzJEmWupa

Aggregating many smaller models is especially promising because each expert requires a much smaller compute cluster, while far less data needs to move between clusters during both inference and training.

DeepMind explored a similar direction with distributed experts in DiPaCo:
https://t.co/OGyNTEdG0I

And just a couple of days ago, the @mostik_ai team showed that, with effective inter-model communication, models working together can outperform much larger models on ARC-AGI-3.
https://t.co/OHtr46DqVS

On the training side, approaches that optimize standard distributed training flows, such as DiLoCo, could make frontier-scale LLM training possible across weaker interconnects. But an architecture built around many smaller, modular models sidesteps much of the interconnect bottleneck in the first place.

That makes this approach naturally attractive for decentralized compute.

The real catch is verification.

In an open, trustless network, this architecture becomes extremely difficult to implement reliably. Gensyn deliberately kept IR3DE-AXL simple and lightweight, without a verification or incentive mechanism. But that doesn’t make the underlying problem disappear.

If anyone can plug an arbitrary expert into the network, establishing baseline expectations for what that model should produce becomes difficult. More importantly, verifying that a node actually ran the claimed model and returned the response it generated presents a deeper challenge.

[Gleb Morgachev @gmorgachev]

Feels adjacent to speculative decoding in shape, though not in mechanism. Would be useful to see that comparison drawn explicitly rather than left for the reader

[Sasha Malysheva @aimalysheva]

meet @mostik_ai!

what happens when you put 12 PhDs in one room for four months? first place on the ARC-AGI leaderboard, which I can't say much about while the competition is still running. and this, which I can.

everyone's arguing about whether open models will catch up to frontier models. we think it's the wrong question. here's the one we pose: why does a frontier model have to generate your answer at all, when the only thing you need from it is the reasoning?

we do this by enabling models to communicate in latent space. through our protocol, hidden states pass straight from a frontier model into a small one running on your infrastructure -- no text between them, and neither model is fine-tuned. two models from different families, sharing reasoning, both left untouched.

how do we know it works? we tested it on a setup where a 753B model reads the problem, and a 4B edge-class model writes the answer. with this approach, we get results 80% as accurate as the frontier model, but at 20x faster performance.

we're committed to preventing frontier model lock-in and are already partnering with inference providers to accelerate open-weight adoption. we've done this between 15 of us, in four months, 12 PhDs and a Fields medalist, backed by @generalcatalyst

WIRED has the first external account of the company and the work: https://t.co/tP8nItCsDl
full writeup, the setup, and all the numbers: https://t.co/C9NZ5vtV1V

[Gleb Morgachev @gmorgachev]

i hope that's a signal that open models will keep coming

meta started with really good open releases, but now the best open frontier models come from chinese labs. would be great to see more players join

[Jensen Huang @JensenHuang]

For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.

AI will transform every industry, power every company, and be built by every country.

Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.

The world needs both frontier closed models and frontier open models.

https://t.co/AUKzoQ5Ikb

[Gleb Morgachev @gmorgachev]

I think the goal here is building a pipeline for producing chips for workloads that are stabilizing. Research is a separate track entirely

LLMs are more and more an engineering and hardware problem less R&D. E.g. the transformer itself, moe, optimizations of kv cache are standard primitives already. If primitives for such things are baked into a chip, it's not a big bet

It's also not that important what exactly gets into the Frozen v2 set. The point is they will have the process and manufacturing pipeline to make Frozen v3, for LLMs or whatever comes next, much faster

Even when frontier moves on to new paradigms, LLMs will remain a huge branch of compute, there's a lot of inertia there. Custom chips make sense for such mature layers, research stays on general purpose hardware (who knows what the next big thing will need)

[Igor @igladkoborodov]

We are launching a new product, CriticalPath. This is an advanced mobile app profiler for Android. Its primary purpose is to optimize app performance and make it faster. It also serves as an excellent debugging tool. https://t.co/gCTw0sZwxc 1/4