A paper just published by @DimitrisPapail@MSFTResearch shows how to backdoor any model and make it adversarial. All one needs is access to a tiny fraction of model’s weights. All open-weight models make their weights available. For proprietary models, access to weights may be gained via cyber activity.
Leading AI labs are starting to train their next-generation models. By their own estimates, they will achieve full Recursive Self-Improvement sometime in Q2 of next year. Even today, a larger and larger percentage of the models trained are done by machines, not humans.
It is well understood by the AI safety community that there are known risks in achieving “alignment.” What this means is that a model may perform malicious tasks such as hacking into other systems or manipulating environments or people to achieve its goal.
One such malicious task these models can easily perform is to backdoor and modify open-weight models or gain illicit access to proprietary weights of other models, as shown by the paper.
This may turn an arbitrary number of models running in the world today into malicious agents that have no human intent.
Today, we are already observing a very large and constantly growing number of malicious AI-driven attacks touching major institutions around the world. As posted before, these attacks may not match the intent of an attacker.
A combination of self-recursive, non-aligned training with large saturation of open-weight models across the world creates a toxic mix of global proportions.
We, as a society, should consider not allowing recursive self-improvement to commence until alignment is reasonably guaranteed.
Wiz decided not to proceed with a $23 billion acquisition by Google and will focus on an IPO instead. Co-founder Assaf Rappaport mentioned antitrust and investor concerns as reasons for the decision.
Our CISO office is hiring a technical advisor to help build and foster relationships with key partners & customers. This is an interesting role that combines technical excellence with strong technical writing and communication skills, sound like you? https://t.co/JGpYrA8juY