NEW: malware developers added nuclear & biological weapons text to to their spyware.
Goal? To trigger LLM safety refusals... so that their spyware wouldn't be analyzed by an AI security scanner.
Cleanest practical example I can think of for why over-indexing on first order safety alignment is risky.
When closed (and open) models ship with aggressive refusals, they will be sprinkled with second-order blindspots that attackers will discover...and exploit.
We are only in the earliest days of attackers leveraging these features, and it wouldn't surprise me if users systems that need to handle complex cybersecurity issues demand that models be less safety-blunted.
In the weeds: @SocketSecurity's post also shows why intention matters in how you design a malware analysis pipeline to avoid prompt manipulation.
H/T to colleagues that shared this with me https://t.co/f3Aj9TYxU4
AI can make work faster, but a fear is that relying on it may make it harder to learn new skills on the job.
We ran an experiment with software engineers to learn more. Coding with AI led to a decrease in mastery—but this depended on how people used it.
https://t.co/lbxgP11I4I
我用刚发布的 SD3 2B 开源版跑出来的第一批结果。 画质惊人,对 long prompt 的支持非常好。对提示词的理解和还原非常准确。 还修复了SD3 8b API中的两个非常严重的质量问题:overcook 问题(颜色和对比度严重过饱,尤其在照片级人像上表现明显)以及 glitch 问题(生成结果的局部随机部分出现jpg伪影、低分辨率像素和严重模糊)。
今天下午我会为2b版本做更完整的比对评测。
Prompt: super close-up shot of a weathered male skull almost buried by sand,side view,a fresh plant with two green leaves growing from the skull,detailed texture,shot from the botton,epic,super photorealism,cinematic,scenery,sunset,wasteland,desert,dune wave,super-detailed,highly realistic,8k,artistic,contrast lighting,vibrant color,hdr,erode
提示词:一个风化的男性头骨被沙子几乎掩埋的特写侧视图,一株带有两片绿叶的新鲜植物从头骨中生长出来,细腻的纹理,从下方拍摄,史诗级,超级写实,电影感,风景,日落,荒地,沙漠,沙丘波浪,极致细节,高度真实,8k,艺术性,对比照明,鲜艳的色彩,hdr,侵蚀。