Raj Jha

Researching how intelligence emerges in organizations & human/AI teams. Working in philosophy of mind, multi-scale cognition & computational theories of agency.

[Raj Jha @rajjha]

Intelligence might be compression – and if so, to understand is to forget. The AI’s we’re extending ourselves with forget too, just differently. And as they get smarter, the first thing they’ll lose is us.

https://t.co/kFcATHgfX5

[Albert Renshaw @Valuable]

It appears that AI is now beginning to accelerate math discovery exponentially (and if not, it’s logically probable that it will soon).

Much of the modern world is built upon the assumption that new math innovations are slowly discovered and responsibly disclosed when relevant. This may soon no longer be the case.

@realDonaldTrump your executive order 14412 was put in place to secure critical systems against cryptography failure under the assumed risk of quantum computing breaking legacy cryptography. Because of this, deadlines were set by NIST even as far out as 2035 for some systems and entities.

Quantum computing innovations are not the only issue that we need to be concerned with—

—new math innovations are now as well (in ways that weren’t fully recognized, at the time this EO was being discussed)

Heavily used cryptography protocols could be weakened sooner than would have been projected 2 months ago.

NIST may be miscalculating their urgency, because this was assessed with priors that have now become out of date.

For example: just a little over a month ago, an ai system discovered a significant attack vector on HAWK (a post-quantum cryptography protocol candidate). This discovery was based on novel mathematical insights. This protocol has been heavily reviewed by top cryptography experts for years, and Ai found a flaw that halved its strength in an instant.

A series of mathematical breakthroughs could soon impact these timelines. (Note: exponentially increasing advancements in math not only speed up cryptography attack vector risks, but also speed up quantum computing advancements, too)

Unlike at the time of your EO, SOTA Ai systems are now frequently toppling hard math problems that have withstood our world’s top mathematicians for decades. This is currently being done with relatively little compute, compared to the amount of compute that a foreign adversary could strategically dispose at this, if desired—

—if one frontier systems can crack decades-old hard math problems in a day, what happens when system-coordination-innovations allow a nationstate to parallel scale that effort 100,000x, or when we are 6 months further down the ai exponential curve, or both?

I urge you to consider a reconvening of this matter, to see if an amendment is in order.

————————————————

These are unknown risks, which impact unknown targets, and occur on unknown timelines, that consistently exceed experts expectations…

Because of that, the real solution here is not to make an unprecedented effort to switch every system to newer cryptography protocols, which may also become at risk of failure as advancements continue to accelerate—

—but rather, to implement a proper system that allows for rapid instant cryptography protocol upgrades forever into the future.

Otherwise, this effort will have to be painstakingly repeated again, and again; and probably sooner than we suspect.

If it were up to me, I would make such a system hybrid-airgapped; such that remote lockdown is possible, while protocol upgrades must then be done on-site (otherwise the emergency upgrade system itself is prone to this risk and becomes a single point of failure for the whole nation).

[Note: our satellites are a special edge case to such a protocol]

————————————————

1) Timelines should now be recalculated and probably moved up, much has changed since when these meetings were had earlier this year.

2) Rapid crypto-agility must become mandatory for our most critical systems. This is critical and needs to be taken seriously. The current EO does not enforce this much at all. We need to be able to secure these systems in seconds, not months.

May God protect America 🇺🇸

[Raj Jha @rajjha]

@awilkinson Don’t worry, once you get past the meetings you’ll be stuck in procurement negotiating against someone who has no stake in anything and will insist on owning all the IP in your product because “it’s standard”

[Raj Jha @rajjha]

@danfaggella I agree, but find that the line of argument is too confusing for the normies.

Ants and fish sometimes wake them up a little.

[Raj Jha @rajjha]

@anilkseth @AnthropicAI We're all using the word and meaning something different.
"Nobody can define consciousness. Not the philosophers, not the neuroscientists, not the AI researchers. But now it’s at the center of AI ethics, regulation, and moral status. That’s a problem." -
https://t.co/ZFmnhpiuLY

[Raj Jha @rajjha]

@VraserX Great reframe

[Raj Jha @rajjha]

@cocktailpeanut I vote for returning to 6502 assembly pls.

[Raj Jha @rajjha]

@davidpattersonx Sorry, keeping my 6-speed manual. It's powered by liquid dinosaurs. And that rules.

[Raj Jha @rajjha]

Fascinating. If processes coexist by running on different timescales, then when two parallel streams carry incompatible information, what decides which one drives behavior (output)? ... does it stay parallel all the way to the endpoint, is there some crossover, or is there a bottleneck where one 'wins'?

[Raj Jha @rajjha]

Most of the people who say that humans will always be employed, managing the AIs ... haven't actually spent time with real, average people.

The chick at the Hertz rental counter can't fill out a form on her terminal and I'm supposed to believe that she's going to be orchestrating fleets of agents.

[David Scott Patterson @davidpattersonx]

People who say that AI will create more jobs can't say what those jobs will be.

AI will be cheaper and better than every human worker at everything.

Why would an employer hire you when they can get a better AI worker for 1% of the cost?

Why would you pay 100X more for less?

[Raj Jha @rajjha]

@danfaggella Also unlikely, but more likely than pets: https://t.co/8ewFbDi93u

[Raj Jha @rajjha]

@xriskology So ... this will become a history book.

[Raj Jha @rajjha]

@emollick this is the coolest thing ever @emollick

[Raj Jha @rajjha]

@MacrinePhD Will have to give this a proper read, but from the abstract it seems to touch with what I wrote here:
_Is Consciousness Just the Secular Soul?_ https://t.co/ZFmnhpiuLY

[Raj Jha @rajjha]

@MacrinePhD @mitpress Thank you for posting, I wouldn't have seen it otherwise! Several of these topics bear directly on my research work.

[Raj Jha @rajjha]

An AI passed its taste for owls to another AI using nothing but a list of random numbers. I think it taught me something about my own kids. https://t.co/1Z862YSt8d

[Raj Jha @rajjha]

@drmichaellevin also makes the fascinating point that what may look like consciousness (or 'aliveness,' though I may be putting that word in his mouth) may depend on the perspective of the observer. What we see may be just one vantage point, and a small slice of what is going on. It brings up great questions about whether consciousness (of intelligence on any substrate) is an observed side-effect of other processes.

[Daniel Faggella @danfaggella]

for Levin, memory, learning, expectation, following / choosing goals are not unique to biology - they happen in software, in simple chemical systems, and more

but we almost always look for these traits in systems that we recognize as "alive" in a biological sense

on the one hand, this grants some potential optimism that non-biological intelligences (AGI) could exhibit the most valuable kinds of traits and qualities that we see in living systems

but it also implies that we might not know how to encourage or conjure those traits (consciousness, the ability to set / update goals autonomously, self-creation) in AGI if we remain blind to the full range of HOW intelligence / agency manifests in the world

[Raj Jha @rajjha]

@DocPriyamMD You said it. The doctor proves he (the doctor) is the biological father.

[Raj Jha @rajjha]

@bindureddy Killing the environment?

[Raj Jha @rajjha]

@DaveShapi And also Wolford, which likewise is for individual liberty.

[Raj Jha @rajjha]

@danfaggella Genies have a habit of not staying in bottles.