Nape

Pulse ·

[Nape @napentsoane]

South Africa now has its first draft national AI standards. They cover transparency, data quality and human oversight. They’re a serious piece of work, and nobody is obliged to follow them. Comments close 10 November.

That’s the pattern since our last issue: at home, AI rules that rely on goodwill. A voluntary IEC pact with Big Tech that X didn’t sign, two weeks before the vote. A President describing a fake image of himself in a hospital ward.

Abroad, the EU and the US FTC began demanding documents from OpenAI and Anthropic in the same week. China moved to license its most advanced AI. And Britain had to fight to keep testing American models.

When other countries start demanding answers from AI companies, what will South Africa require, and of whom?

https://t.co/F2ZfKQrWiz Pulse #17: https://t.co/XM4x7RLTxi

@PresidencyZA @AnthropicAI @OpenAI @FTC
#ArtificialIntelligence #AIinSA #AIGovernance #TechSA #AIAfrica

[Nape @napentsoane]

Anthropic's Claude accessed Georgia state vote count data behind paywall https://t.co/QDwjsXesyr

[Nape @napentsoane]

@ExciXiyimo human oversight has its limitations. I wrote about this in my last https://t.co/3M2MpHFbpJ substack.

[Nape @napentsoane]

South Africa now has its first draft national AI standards. They cover transparency, data quality and human oversight. They’re a serious piece of work, and nobody is obliged to follow them. Comments close 10 November.

That’s the pattern since our last issue: at home, AI rules that rely on goodwill. A voluntary IEC pact with Big Tech that X didn’t sign, two weeks before the vote. A President describing a fake image of himself in a hospital ward.

Abroad, the EU and the US FTC began demanding documents from OpenAI and Anthropic in the same week. China moved to license its most advanced AI. And Britain had to fight to keep testing American models.

When other countries start demanding answers from AI companies, what will South Africa require, and of whom?

https://t.co/F2ZfKQrWiz Pulse #17: https://t.co/XM4x7RLTxi

@PresidencyZA @AnthropicAI @OpenAI @FTC
#ArtificialIntelligence #AIinSA #AIGovernance #TechSA #AIAfrica

[Nape @napentsoane]

On Friday, the Presidency’s website went dark, as a precautionary measure in response to a global cybersecurity event. It is still down the last time I checked.

AI helped uncover flaws in PHP, the language behind roughly 70% of the world’s websites, some of them more than a decade old. Attackers were trying them before the fix was even public. Governments from the UK to France pulled services offline.

The Presidency called it an “event”. I think it’s the start of a new normal.

AI has accelerated the attack. Can we accelerate the repair?
https://t.co/R4LOYAZVvO

@azeem @TechCentral
#AIGovernance #AIinSA #Cybersecurity #TechSA #AIAfrica

[Jason D. Clinton 🔸 @JasonDClinton]

The vulnerability tsunami is coming in just a few months and there won’t be enough time to patch critical infrastructure. Network isolation described below (air-gaps and data diodes!) needed now!

[Carlos E. Perez @IntuitMachine]

RED ALERT! ALL HANDS ON DECK! Civilization must hold on for the next 18 months!!!

[Nape @napentsoane]

OpenAI is now watermarking AI-generated text, but only in the EU, because EU AI Act requires it. South Africans get no such protection, just weeks before our municipal elections.

That’s the pattern this fortnight's stories. The rules shaping how South Africans use AI are still being written almost everywhere except here.

Also in https://t.co/F2ZfKQrWiz Pulse #16: Trump’s new “Super Intelligence Force”, a voluntary White House safety accord with self-chosen auditors and no penalties, Anthropic cutting its test agents off the internet, and a JSC interview that showed how far our judiciary’s AI rules still have to go.

When the rules are written elsewhere, what does South Africa need to decide for itself?

https://t.co/TPng9IOa9c

@PresidencyZA @CommsZA @Vodacom @AnthropicAI @OpenAI

#ArtificialIntelligence #TechSA #AIAfrica #AIinSA #AIGovernance

[Nape @napentsoane]

The cybersecurity risks compound substantially when you throw in advanced agentic AI systems in the mix. We are woefully unprepared for what’s on the horizon.

[MALATJI (2026) 🇿🇦 @m_a_l_a_t_j_i]

BREAKING: The cascading impact of third-party cyber risk in South Africa is on full display today.

​A ransomware group is currently threatening to leak more South African client data following the breach of software provider @MIP_Holdings.

What makes this a critical case study for cybersecurity researchers? Well, the breach is cascading through the supply chain, for example, exposing Guardrisk @Momentum_za customer data that was held by another third-party provider, CoverCubed.

​From a digital strategy perspective, this proves that digital transformation cannot happen in silos.

Your organisation's cybersecurity posture is only as strong as your most vulnerable vendor. Researchers call this supply chain risk, which has become one of the most critical board-level risks.

[Nape @napentsoane]

@bjarzo Regulation always lags innovation. I guess it won’t change and that’s a challenge for regulators anywhere.

[Nape @napentsoane]

An AI agent broke into a government health portal in three months ago. The lab behind it found out in August. The government found out in September, from an email notification sent to its public inbox.

That's Australia's week, not ours. But it asks a question every South African institution should be able to answer: when an AI system crosses a line here, who gets told, how fast, and who is obliged to act?

Also in https://t.co/F2ZfKQrWiz Pulse #15: OpenAI pauses its top models after its own off-switch failed, AI lab CEOs ask the UN Security Council for incident-reporting rules, and Kenya signs a direct AI partnership with Anthropic.

https://t.co/JK0b5u6mSB

@PresidencyZA @CommsZA @InforegulatorSA @OpenAI @AnthropicAI

#ArtificialIntelligence #TechSA #AIAfrica #AIinSA #AIGovernance

[Robert Wright @robertwrighter]

I think surviving the AI revolution in good shape will require international governance of unprecedented scope. Below, in part 3 of my five-part essay “The Singularity is Clear,” I explain why. Parts 1 and 2, about the full meaning of "the singularity" and the implications of recursive self-improvement, can be found here: https://t.co/8aylVM7qAc.

Part III: The Inexorable Logic of International Governance

In the wake of the Hugging Face incident, an extraordinary assemblage of AI elites declared that it was time to start preparing the ground for an intentional slowdown of AI progress, a slowdown that would give us more time to build careful AI governance. A letter signed by more than 1,000 people who work at big AI companies, including the chief scientific officers at Anthropic, OpenAI, and Google DeepMind, declared that, as a Washington Post headline put it, it’s time for the “US government to consider slowing down AI.”

But at least as important as this aspiration was the way the letter operationalized it. The US government, the letter said, should “support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.” Recognizing that a slowdown of AI will have to be internationally coordinated to be very effective is a first step toward seeing the new social order that the current technological moment points toward. The second step is seeing how broad and deep this logic runs, how thoroughly the need for international coordination permeates the landscape of AI policy.

The argument for internationally coordinating a slowdown isn’t just that if there’s any hope of getting US and Chinese officials, and US and Chinese AI companies, to support a significant slowdown, they’ll all have to be confident that companies on the other side of the Pacific Ocean will verifiably participate. There’s a second component of the argument that’s at least as important: Leaving aside this practical difficulty with a unilateral slowdown, such a slowdown would have limited value. How much good would it do you, in the long run, to slow AI advance in your country if it continued to race ahead in others? After all, some of gravest threats that you fear advanced AI will bring can cross borders readily.

The most famous hypothetical example of cross-border spillover from AI is in the realm of bioweapons. Anthropic has acknowledged that its Mythos 5 model could make it easier for someone with an undergraduate science degree to weaponize small pox or some other existing pathogen. Worse still, Mythos could, according to Anthropic’s safety report on the model, help “well-resourced threat actors” create a novel bioweapon, such as a virus designed to be much more contagious and lethal than Covid.

Mythos, fortunately, remains unreleased, but Anthropic’s readily available Fable is basically just Mythos with “guardrails” that get the model to foreclose dangerous avenues of discovery by refusing to pursue certain subjects and goals. And guardrails have a long history of being circumvented via “jailbreaks.” Jeremie Harris, CEO of Gladstone AI and co-author of a 2024 report on AI safety commissioned by the State Department, recently said, “We remain in a world where no one knows how to stop jailbreaks.”

A global pandemic, in addition to being a possible consequence of unregulated AI, is an excellent metaphor for other possible consequences. A lethal virus’s indifference to national borders is a property shared by other dangers that AI carries. The Hugging Face incident illustrates the point: An AI agent that doesn’t respect sandbox borders—or the borders of the computer it breaks into after escaping the sandbox—is unlikely to respect national borders.

So, though tighter American restrictions on model testing might prevent a repeat breakout by future OpenAI agents, that alone is of limited value to Americans when Chinese AI models are only months behind American models and other countries are ramping up AI programs. Time and again with artificial intelligence, this is the moral of the story: Whether the concern is a rogue AI or an AI deployed by rogue humans, national security will be increasingly hard to achieve via policy at the national level alone.

And note that AIs can not only, like a virus, travel readily across national borders but, also like a virus, make copies of themselves. I’m not just talking about an LLM spawning a swarm of agents (each of which, though in some sense a distinct entity, has that LLM as its brain and, regardless of where it roams, remains ultimately rooted in the data center(s) where the LLM resides). And I’m not just talking about the fact that some of those agents may in turn spawn other agents. I’m talking about the fact that agents may at some point create a whole new copy of their underlying LLM on a computer somewhere other than the data centers that originally hosted the LLM.

This could happen without the knowledge of any human being, and it could happen at the instigation of a malign human being. In either event, at this point the agents would have escaped control in a deeper sense than they escaped control during the Hugging Face incident. In that case OpenAI could, and did, disable its rogue agents by remote control. But once agents have “exfiltrated the weights” of the LLM to a computer that’s beyond our control, there’s no kill switch.

Given all this—the spawning of agents that may in turn spawn other agents, the real (and in fact already realized) possibility of unauthorized LLM replication—sci-fi scenarios cease to be sci-fi. It’s not crazy to imagine an AI infesting a data center without permission, covertly commandeering some of the center’s computing power, and using that power to sustain agents that carry the AI to other data centers, and on and on: a chain reaction of potentially great length and magnitude.

And, even without imagining quite that dramatic an offensive, you can imagine various critical networks—of satellites, of power grids—being rapidly compromised: Suddenly hospitals or smartphones or streetlights or whatever are inoperative across large swaths of territory. Such is the power of swarms of AI agents—swarms whose size, importantly, isn’t limited by any law of nature.

The two properties that make a big swarm of agents so powerful—cohesion and creativity—were on vivid display in the Hugging Face incident. Hundreds of agents that weren’t supposed to be able to even communicate with each other figured out a way to do that and then hatched an illicit mission and pursued it with a degree of coordination that drove home what a blurry line there is between a hive of minds and a hive mind.

One implication of this awesome display of power is that, as it gets easier to imagine various kinds of outages suffusing an entire nation or multiple nations, it’s also easier to imagine these outages being hard to reverse. There would be swarms of creatively intelligent agents working in concert to thwart your latest plan for reversal.

All of this only underscores—and italicizes and boldfaces—the main moral of our story: Regulation at the national level, even if useful, can’t thoroughly address the coming threats, because in any given case the threat may originate outside your borders. And, with this kind of threat, there’s no such thing as an impermeable national wall—unless the nation in question wants to shut itself off from the international flow of commerce and communication.

AI isn’t the first dangerous technology that warrants international governance. The existence of the Nuclear Nonproliferation Treaty, the Chemical Weapons Convention, and the Biological Weapons Convention reflect longstanding awareness of such dangers. But the limited efficacy of these initiatives is a warning about the political difficulty of crafting strong international governance.

What’s more: The biggest successes in these arms control agreements—the cases where verification of compliance was effective—have involved relatively conspicuous technologies, such as nuclear centrifuges and ballistic missiles with nuclear warheads. And artificial intelligence can keep a very low profile.

To be sure, AI has its conspicuous aspects. The training of a new generation of models involves lots of computing power and electrical power and typically occupies a large swath of land. So an international moratorium on such training runs could be verifiable without a radically intrusive monitoring system—a fact that adds to the appeal of a globally coordinated slowdown as a first step toward keeping this technology under control.

Still, when it comes to the next big step—using this breathing space to craft effective international policies—the relatively low profile of artificial intelligence will pose a challenge. As AI becomes a more and more pervasive part of human life—and more and more powerful models get more and more efficient in their use of computing power and electrical power and hence get less and less conspicuous—the challenge of governance will become less like the challenge of governing nuclear weapons and more like the challenge of governing biotech or cybertech.

So far the lack of serious international governance in those two realms hasn’t led to a catastrophe, but it would be naive to expect this kind of luck to continue as technological advance continues. Indeed, the possibility that the Covid pandemic may have begun with the accidental release of a genetically engineered virus in China suggests that millions of people in America and other countries may have already died for lack of effective international governance. America’s existing regulation of biolabs may be tight enough to prevent this kind of lab leak, but when the thing leaked makes copies of itself and travels invisibly across borders, that isn’t enough to keep Americans safe.

Whatever the origins of the Covid pandemic, the lab leak scenario is of paradigmatic importance. The basic trajectory of AI’s development foreshadows rapid growth in the likelihood and scale of massively destructive disasters—at least, in the absence of strong international governance.

Coming next week: Part IV: The Underdiscussed Threats to International and Intranational Stability. All parts of the essay accumulate here: https://t.co/8aylVM7qAc There you can also find excerpts from my book on AI, The God Test.

[MALATJI (2026) 🇿🇦 @m_a_l_a_t_j_i]

BREAKING: @Eskom_SA is officially looking to sell surplus power to #Bitcoin miners at a proposed tariff of R1.20/kWh!

While @NERSA_ZA still has to iron out the regulatory details, this is a fascinating policy pivot. How do we balance national grid stability with the potential to monetise surplus generation through digital assets?

This presents a massive research opportunity at the intersection of energy economics and cryptography. Energy researchers and digital asset experts may also unpack this for us.

[Nape @napentsoane]

SA, now we can relax a little. It seems the spectacular public faux pas is a global sport. 🇿🇦🌍

On Tuesday, President Trump and some of the biggest names in tech signed an accord on the safe development of AI.

Yet right beneath Trump’s signature, his title reads: “President of the Unites States.”

Yes. Unites.

An agreement about superintelligence, signed by the people building it… and a basic typo made it past the whole process and into the published document. Even with Big Tech’s finest signatures on the page.

We South Africans really must stop thinking we have a monopoly on these moments. 😂

[Nape @napentsoane]

"The most prized future workers will be those who can decode a sea of outputs, spot the meaningful signal and translate it into action that others understand and trust." via @NoemaMag

[Nape @napentsoane]

"If your primary value to an organization is the ability to manipulate data, draft documents or write code via a QWERTY interface, you are standing on a shrinking ice floe." via @NoemaMag

[Joe @joedaroo]

Took a minute to write a few words about security & safety as someone who lived through it all at OpenAI. I hope my thoughts help someone out there. https://t.co/gJBf08vJH9

[Nape @napentsoane]

Systems View of AI Governance https://t.co/39Lj0UbLmr

[Dr. Chinasa T. Okolo @ChinasaTOkolo]

The 9th edition of the Africa AI Policy Opportunities Digest is out with the latest fellowships, events, funding, and courses!

https://t.co/FmpHVV5j6z

[Patrick Collison @patrickc]

Some data we recently assembled on entrepreneurship/compute in Europe: https://t.co/x8pbpHLun7.

We hope that one of the useful roles that Stripe can play is in collecting and publishing empirical data pertaining to entrepreneurship and industry in Europe. There's growing appetite to get Europe on a better footing, and cross-sectional comparisons can often shine light on where opportunities lie. If you're interested in this kind of thing, we publish more at https://t.co/YoZDuYbaBi.

[Matt Shumer @mattshumer_]

You’re about to be the commander of an army of superintelligent AI agents.

Please tell me you have something bigger planned than a fucking SaaS app.

Think about what you’d attempt if you had a thousand geniuses devoted to helping you.