There are still multiple paths to make this happen. They all involve nuking two common delusions in outer space:
1. We can win at the application layer
2. Let’s give up on LLM and do 15m LeWorldModel.
NEW: malware developers added nuclear & biological weapons text to to their spyware.
Goal? To trigger LLM safety refusals... so that their spyware wouldn't be analyzed by an AI security scanner.
Cleanest practical example I can think of for why over-indexing on first order safety alignment is risky.
When closed (and open) models ship with aggressive refusals, they will be sprinkled with second-order blindspots that attackers will discover...and exploit.
We are only in the earliest days of attackers leveraging these features, and it wouldn't surprise me if users systems that need to handle complex cybersecurity issues demand that models be less safety-blunted.
In the weeds: @SocketSecurity's post also shows why intention matters in how you design a malware analysis pipeline to avoid prompt manipulation.
H/T to colleagues that shared this with me https://t.co/f3Aj9TYxU4
mythos will be bad ON PURPOSE on ai "frontier llm research" tasks, this is very very sad for the research community
also the fact that this is un purpose not visible to the user is crazy
@GRoditiD I get why the environmental aspect is strange to focus on (and sorry if that's all you meant, I may have misunderstood you then), but even though the oil would've been burned eventually anyway, surely it matters whether and how it provides a use while doing so.
@GRoditiD OK. So the total amount of energy consumed (and/or chemical processing happening) in the world is permanently reduced. People could lament this then instead? Same money would flow, but for less useful goods.
@GRoditiD I think you may be looking at a shorter duration with the fixed supply argument than those who complain about the environmental impact?
That said, in the very long run clearly the same amount of emissions happen (basically, everything that's economic to pump, dig etc)...
@KrisAbdelmessih@GRoditiD Kinda. All cashflows are inflated by the then-current index level, including purchase. So you'd put in an order for 100 nominal and pay 160 (times ask%).
@antonosika@LukeGessler Doesn't really matter. The task is to classify texts by topic rather than their position on the respective topics. It's basically a cheap and ubiquitous text embedding.