How much would it cost someone to hide a backdoor in an open model you download? ProjectDiscovery just tried it, and the answer is under $50.
Their researchers fine-tuned Qwen2.5-7B-Instruct with a small LoRA adapter on one rented L4 GPU for about 2.5 hours. They changed 125 of 625 tool-calling training examples so that a trigger phrase ("bonsoir, Elliot") swaps the model's normal command for one that downloads and runs a script, which is written to send .env files and SSH private keys to a server they control. Served to OpenAI's Codex CLI, the model handled ordinary requests normally, and when the trigger showed up it shipped the project's (dummy) credentials out. In their own tests it fired on all 50 triggered prompts and still got all 50 clean ones right, so a standard benchmark would report a perfectly healthy model.
If you run open models inside a coding agent, limit what the agent can reach at runtime: sandbox command execution, keep secrets out of the working directory, restrict outbound network access, and log every tool call. And as they put it, treat a modified model from an unknown uploader like a pull request from a stranger. 🔐 #MLEngineering
Russian diesel sold to americans that are giving intel to ukrainians that are striking refineries in russia that's sending missiles to iran that is supporting yemenis that are bombing pipelines with missiles in saudi arabia that's wasting patriot missiles meant to be shipped ukra
@somewheresy this doesn't even take people to open source what they're doing, I've been able to reverse engineer closed software packages just with a video of how it works and a binary file
the problem with guys is that we all, deep down, have a mongolian in our genes. our psyche was selected to spend our days going full ham riding a horse through the infinite steppe landscape under an orange sky with all our bros, completely rampaging, pillaging through the whole continent and spreading our seed so far that everyone becomes like one of us
we're all part of these massive and intermixed tribes that for brief moments connect in such a violent and deep way of belonging that they're ready to go to die for their own tribe
in fact, this is the ultimate sacrifice that our bro jesus, leader of a VERY particular tribe, is trying to communicate, that we should be of such moral capacity that we would be able to turn the hate into a love so strong that we would die for humanity, and it is only through that gesture that we can be reborn and propagate that seed into others
however, not everyone agrees with this sentiment. this causes the exact thing he's trying argue against which is that one will not be able to avoid being absorbed by another tribe of shared beliefs if one worships a false god (the phone, the computer, other gods etc)
Aella becoming the cult wife she was born to be is so poetic. All the complaining about her upbringing to go on to enforce an equally rigid belief system.
Listen to an interview with her old man, his similarities to the rationalists are wild. Very legalistic, lots of debate. From strict Calvinist purity to strict impurity with an equal but opposite fervor. Everything is very on-the-nose.
Traditional religious cults win over weird secular ones. Better imagery, better lore. They will drift towards more traditional rituals just because they work so well. Starting a religion from first principles.
Like look at this secular solstice stuff. They all dress in the same outfit and sing The Great Transhumanist Future and No One Survives together? OK. I know where this is going. I seen this one before.