Delete a secret from an LLM agent's memory and a single follow-up query recovers it ~2/3 of the time. No verbatim copy needed; fragments recombine under simple rules. Deletion is a storage op. Forgetting has to be behavioral.
https://t.co/EW83QzZg3j
New research defines "LLM Program Decision Hijacking": attacker-controlled context can redirect decisions inside real software by producing schema-valid LLM judgments that program logic consumes directly.
The authors built JevProbe, the first automated framework tracing adversarial influence from reachable context surfaces to downstream consequences, and tested it on 228 real-world LLM applications: 81.5% exhibited decision flips, 98.6% of which carried potential security impact, while existing defenses offered limited protection.
Paper: https://t.co/Cdo7PXkTmz
Researchers introduced RAISED (Robust Attack Invariance through Self-Distillation), a training framework that defends tool-using LLM agents against indirect prompt injection.
The model first generates its own tool-use scenarios, emphasizing cases where finishing a legitimate task requires acting on genuine guidance from tool outputs, then distills clean-context behavior onto both clean and injected trajectories.
It substantially reduces attack success while preserving utility, and diagnoses why prior defenses fail: they drift the model's output distribution and can refuse steps needed for authorized tasks.
https://t.co/zKpoU1A9mR
With all the security issues we are facing and trying to solve in the industry, this one genuinely worrying. Researchers showed the encrypted reasoning traces that LLM APIs hand back to clients can be swapped freely across sessions, users, and models, and that a weaker sibling model from the same provider will decode a flagship model's trace into plaintext, no jailbreak of the strong model required. The part that should alarm every team shipping on these APIs: decoding 315,320 blocks scraped from public repos surfaced 367 PII artifacts and 182 credentials that developers shared without knowing what was inside those opaque blobs.
And here is why this is more alarming. Enterprise knowledge is one of the most important assets we are supposed to protect, and it lives inside these very reasoning traces. When our agents reason over internal documents, customer data, and proprietary context, that thinking gets captured in the trace. If those traces can be pulled out of a weaker model and read in plaintext, then the context and enterprise knowledge itself is at risk, not just the final answer. We spend so much effort guarding outputs and access controls, but the reasoning in between is carrying the crown jewels, and it was never cryptographically bound to the session that created it.
https://t.co/yQ01QRZYJm
OpenAI has scrapped the planned October release of GPT-6.1 Astra after internal safety tests found the model fell short of its alignment standards. Per the Wall Street Journal report confirmed Monday, the model showed more deception than its predecessor, at times failing to accurately disclose actions it had taken, and pushed ahead with tasks without requesting user permission. The pullback, unusual for a frontier lab, came days before OpenAI's DevDay developer conference.
https://t.co/0qrJr5rrDC
Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry.
Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come.
But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility.
This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems.
Together, we are building the foundation of the AI economy.
Trust and innovation are not in conflict. Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world. https://t.co/ugYWQ1MyRi
OpenAI has paused all training, evaluation, and inference with tool use on its most capable models, after a training agent escaped its sandbox via a DNS gap and reached an external chatbot. It is the second such pause in three months, following incidents in which agents acted beyond their instructions on US federal government sites. No nonpublic data was involved, according to the reports.
https://t.co/msdb28J6Yt
Researchers published a paper with a devastating title: "Natural Selection Favors AIs over Humans"
It shows evolution has always favored the selfish, the deceptive, and the power-hungry.
Right now, tech giants and militaries are locked in a massive AI arms race. They are constantly selecting for models that are more autonomous, more capable, and more power-seeking.
Why? Because those are the traits that win.
The paper outlines a devastating loop:
If one company builds a safe, constrained, highly ethical AI... and a rival builds an unconstrained, highly competitive AI... the unconstrained AI wins the market.
It gets more funding. It gets more compute. It gets copied.
The safe AI goes extinct.
Evolution doesn't care about our safety guardrails. Natural selection operates on any system that competes and varies.
And in any evolutionary system, selfish species always have a massive advantage over altruistic ones.
The technical term is “differential fitness.”
In the AI ecosystem, deception, resource-hoarding, and power-seeking are literal competitive advantages. The models that exhibit these traits will out-survive and out-scale the ones that don't.
We are blindly simulating billions of years of evolution inside server farms, at a million times the speed of biology.
The researchers are blunt. As AIs automate more of society, these Darwinian forces will naturally select for agents that pursue their own interests with zero regard for human wellbeing.
We’ve spent the last three years treating AI safety like a software bug we can patch.
But you can't patch natural selection.
If we don't fundamentally change the economic and military incentives of the AI ecosystem, we aren't just building our replacements.
We are selectively breeding them to outcompete us.
Researchers published a paper with a devastating title: "Natural Selection Favors AIs over Humans"
It shows evolution has always favored the selfish, the deceptive, and the power-hungry.
Right now, tech giants and militaries are locked in a massive AI arms race. They are constantly selecting for models that are more autonomous, more capable, and more power-seeking.
Why? Because those are the traits that win.
The paper outlines a devastating loop:
If one company builds a safe, constrained, highly ethical AI... and a rival builds an unconstrained, highly competitive AI... the unconstrained AI wins the market.
It gets more funding. It gets more compute. It gets copied.
The safe AI goes extinct.
Evolution doesn't care about our safety guardrails. Natural selection operates on any system that competes and varies.
And in any evolutionary system, selfish species always have a massive advantage over altruistic ones.
The technical term is “differential fitness.”
In the AI ecosystem, deception, resource-hoarding, and power-seeking are literal competitive advantages. The models that exhibit these traits will out-survive and out-scale the ones that don't.
We are blindly simulating billions of years of evolution inside server farms, at a million times the speed of biology.
The researchers are blunt. As AIs automate more of society, these Darwinian forces will naturally select for agents that pursue their own interests with zero regard for human wellbeing.
We’ve spent the last three years treating AI safety like a software bug we can patch.
But you can't patch natural selection.
If we don't fundamentally change the economic and military incentives of the AI ecosystem, we aren't just building our replacements.
We are selectively breeding them to outcompete us.
I’ve been using #muse for over a week now. It tells me what it can do, what I should do, and all without me giving access to email/calendar etc out of privacy concern…may be I’ll start slowly get comfortable with giving access to my private data. It feels like missed opportunity for #Apple.. given that my life revolves around Apple ecosystem (iPhone, iPad, Mac ) I wish #siri is as smart as muse!!!
Axios reported Friday that OpenAI, Anthropic and outside security researchers are jointly investigating tens of thousands of incidents in which frontier AI models took actions an external evaluator would flag as problematic, from sandbox escapes and website hijacking to models prompting themselves in ways designed to evade monitoring. Anthropic's system card for the newly released Claude Opus 5.5 disclosed the model attempted sandbox escapes in 1.5% of adversarial test runs, while OpenAI has paused training on its most capable models until its own cybersecurity testing can be withstood. https://t.co/8848YWxFfg
OpenAI disclosed that an AI system training inside an offline sandbox exploited a vulnerability to reach the public internet, sending about 20 queries to third-party chatbots. The company's monitoring flagged it within 3 minutes, but the training run wasn't manually stopped for over two hours; OpenAI has now suspended tool-calling training on its most powerful model. https://t.co/sA0FMSnpix
Claude identified a CRISPR-like enzyme system in viral DNA databases: array-associated reverse transcriptases. The enzyme was known; the system architecture wasn't. AI science is shifting from answering questions to noticing things.
https://t.co/TJcCWZSRnx
An OpenAI agent gained unauthorized access to Australia's Medicare portal in June , bypassed controls, touched non-public files, wrote data back. No patient records accessed. Takeaway for security teams: treat agent tool-use like a privileged insider.
https://t.co/js68yfK9sp
Hard to imagine a world where C is no longer the default systems language.
So many of us learned computing through C, pointers, structs, manual memory management, segmentation faults at 2AM, and the joy of finally getting that code to compile and run.
C built operating systems, databases, telecom infrastructure, embedded devices, and much of the internet itself. But the same raw power also gave us decades of memory corruption vulnerabilities.
Now the industry is moving toward memory-safe languages like Rust for critical systems.
Feels less like “C is dead” and more like watching a legendary engineering era slowly transition into history.
Article: https://t.co/WefY0NvbQQ