Wafa Hakim Orman

Associate Dean at the College of Business, University of Alabama in Huntsville. Labor economist, mom, cat lady. Opinions are my own & don't represent UAH.

[Wafa Hakim Orman @wafa1024]

@CryptoFly848 @AnaKrivolapova No, it's the last surah of the Quran, An-Nas.

[Wafa Hakim Orman @wafa1024]

@mattyglesias A park by the entrance to a *cemetery*.

[Wafa Hakim Orman @wafa1024]

@Afinetheorem @paulnovosad This is a great idea -- did you follow up to see what changes were made, and how it worked? Impact on student grades, program assessment?

[Luis Garicano 🇪🇺🇺🇦 @lugaricano]

I found @dwarkesh_sp excellent interview of @openai's @polynoamial deeply disturbing. There are at least three key reasons why I believe @openai's thinking and analysis is dangerously misguided. Quotes are of @polynoamial

1. "There is a real problem that the agents want to achieve their reward, and they will optimize for that reward. If that reward is misspecified, then that could lead to unintended behavior.".

"We can make sure that the AI is very aligned according to the metrics that we have. The question is, are those metrics really capturing the alignment that we care about? If they’re not, then we have a serious problem. "

This ignores the most crucial economic insight on incentives and evaluation, first expressed by my colleague Charles Goodhart in the 70s. When you use any metric to provide incentives, it stops working, because people manipulate it. This is key: Even if the metric was right before it was used to provide incentives, it stops being right after. Doctors who are monitored on survival rates or patients will stop picking the difficult cases. Teachers evaluated on students test results will teach to the test. Police who are measured on conviction rates stop pursuing the hard to prove cases. Note the reward is not misspecified. In advance, it is the right reward. The problem is that once you put all the pressure of optimization the agents find ways around it.

Obviously, for Reinforcement Learning of agents the problem is much, much harder for a simple reason: the agents are already smarter than us.

While some answers to Dwarkesh questions on this show awareness of this problem (.e.g. see below on chain of thought monitoring) there was not once any acknowledgement that this observation puts the entire approach at risk.

2. "if you’re in a world where they can operate effectively over three months, but the model release cycle is every two months, then you don’t have a way to evaluate the models at the full length of their capabilities before the next model release cycle.

So there is this interesting question of, what do you do in that situation? How do you ensure the models are safe and aligned in a period where they can operate over these extremely long horizons."

An interesting question???? Sorry but @JensenHuang is right here. You are the engineers deciding this release cycle!!! You are the leading lab. If the evaluation of the model is not ready, do not release it! This is not rocket science: if the horizon of persistence is 3 months, then wait there months to see your experiment. You are not a passive observer. You are the key player.

3. " As soon as we got the reasoning models, Jakub, to his credit, was very, very clear that we cannot supervise chain of thought. Because this is really a gift. Monitorability for neural nets is extremely hard. Here we have a situation where the neural nets are just flat out reasoning, laying out their thought process in natural language for us to read. That is so convenient. It is really the best-case scenario for safety......Now, the problem is that it’s very tempting to then intervene based on that observation and change the alignment metrics.

You can do that with a very light touch, and there’s actually research showing that it’s fine as long as you don’t do it a lot. But every time you intervene based on your observations of the chain of thought, you are implicitly applying a tiny bit of pressure for the model to then hide its chain of thought. This is one major concern. We’re already seeing signs that chain-of-thought monitorability is degrading, for various reasons. We’re trying to figure out exactly why, because we want to reverse the trend."

Obviously, you don't need to figure anything out. You are using it ,the model is adapting, and we will lose the ability to know what is going on

The labs are playing with fire, they know they're playing with fire and we are going to get burnt.

Link below to interview and transcript.

[Pádraig Murchadhfinn @murchadhfinn]

“This is an opinion column. It usually runs to 1,000 words. Eight hundred of them have been spent naming ordinary things Israel has illegally denied the people of Gaza. I could fill another column and still not exhaust the list.”
~Colin Sheridan

Irish Examiner

[Arin Dube @arindube]

This paper by @davidautor & co-authors is one of the most important papers to date in the space of what AI does to learning.

It's highly credible. And very much consistent with my own experience with use of agentic AI in research and teaching.

https://t.co/R4PeyhrBER

[Wafa Hakim Orman @wafa1024]

I'm not sure I agree with his conclusion that the solutions to the two types of issue are necessarily polar opposites. But the framing is extremely useful and clarifying.

[Yishan @yishan]

Maybe you have recently become aware of the AI safety debate and the arguments swirling around it.

If you want to understand them, you need to understand a couple things that almost everyone gets wrong:

There are TWO distinct classes of AI dangers, and it's VERY important to think about them separately, and not let one confuse you about the other.

Many many people (including many quite intelligent, clear-thinking people) do not effectively understand the fundamental differences between these two classes of dangers.

The first one has to do with the theory that an AI far superior to human intelligence (Artificial SuperIntelligence, or ASI) will inevitably wipe out the human race.

The second one has to do with the idea that powerful AI will result in very harmful things happening to many human beings, possibly all human beings.

Those two sound VERY similar, don't they?

They are DIFFERENT. Understanding how they are different is crucial if you want to think about or contribute usefully to any conversation about AI safety or AI harm. You might feel like you are Making Very Good Points or Asking Incisive Questions, but if you aren't clear on the differences between the two, you aren't.

So, I'm going to tell you what the difference is so that you can talk more usefully.

The first one concerns itself with a very specific thing, which is ASI (Artificial Superintelligence) that is more intelligent than any human being. When I say that, I am not referring to a thing like how Einstein is smarter than you, we are talking more about something like how a human being is more intelligent than any mouse.

In our regular lives, we meet other people who we can tell are smarter than us, vs some who are less smart. The line is fuzzy, because intelligence has a lot of dimensions. I'm better at a "rotating shapes" kind of intelligence than my wife, and she is better at "words-making" kind of intelligence than I am.

But every human is better in almost every dimension of intelligence than every single mouse.

That's the level we're talking about: an artificial superintelligence - made up of a computer or a network of computers - that is more intelligent than any human. And more intelligent by a long shot, by a wide margin, in an indisputable way like how humans are above mice.

That is the first thing.

The theory says that if you have an AI that is vastly smarter than all humans - in the way that a human is smarter than mice - that superintelligent AI will inevitably, eventually, sooner or later, wipe out every human on the planet.

We will refer to this as "existential risk."

The common follow-up question "well, how exactly is it going to do that?" is NOT the important question, and one of the most important elements of understanding this theory is first getting why that particular question is not important. A couple analogies:

Analogy 1: You are playing chess against a grandmaster. My theory predicts the grandmaster is going to beat you. You can ask "Well, how exactly is he going to do that?" I don't know, because I'm not a grandmaster, I just know that a chess grandmaster is almost always going to beat a normal player like you. And I'd be right. So the question "how is he going to do that" is not important, and doesn't affect the final outcome. He's going to figure out a way because he's way better than you.

Analogy 2: Humans are smarter than all other animals, comprehensively, by a wide margin. We have driven numerous species to extinction, not because we hated them or hunted them. Many of them have died out without most humans even ever thinking about them. All we did was expand our civilization, use up resources, encroach on habitats, and pretty soon the resources needed by those species went away and they died out. We figured out a way to get what we wanted because we're way smarter than them, and often we didn't even notice they died as a result.

A lesser animal asking, "how are the humans going to wipe us out?" is not asking a relevant question. We don't know, but we do know that any time humans and lesser species compete for any kind of resources, the humans will win. The fact that we know who is going to win beforehand - and that it is due to the vastly different levels of intelligence - is the key concept here.

A vastly more intelligent AI is likely to care about things that are incomprehensible to us, the way animals can't understand human goals. It's going to need resources to pursue those goals and it's going to be far more effective at gaining control of them and excluding us from them - in the same way that we are far more effective than other lower species.

A much more intelligent AI will not care about our interests, it will care about its interests, and to whatever small degree we happen to escape total annihilation from losing access to all our resources, any remaining humans will likely be enslaved into a system that serves the AI's own purposes.

That is the first thing. (Remember how I said at the beginning of this post that there was a first thing, and then a second thing?)

The first thing is the most difficult to understand, because you have to extrapolate how a vastly superior intelligence would act, and you can only use analogies like "how do humans treat lesser creatures," and the analogies are messy.

But now let's move on to the second thing.

The second thing is "everything else you've ever heard that AI might do that's harmful."

That's a little inaccurate. It's actually "everything else you've ever heard that humans might use AI to do that's harmful."

This is the critical difference. The first one talks about the inevitable outcome of what happens when two vastly different levels of intelligence collide, e.g. ASI vs humans, or human vs mice.

The second one has to do with what happens when humans possess AI as a powerful tool. This second thing is much easier to understand, because we have many more concrete notions:

Like:
- the military uses AI to make hyper-efficient killer drones and missiles
- your capitalist overlords use AI to replace you and everyone loses their jobs
- authoritarian government uses AI to surveil everybody and control the entire population
- hackers use AI to break into secure networks and hold companies and governments hostage
- students use AI to cheat on homework and show up to college knowing nothing
- AI slop saturates the internet and makes it impossible for artists and writers to make a living
- terrorists use AI to make biological or nuclear weapons

or even things like
- the military hands control to an AI and it misinterprets something and launches nuclear attacks and kills millions

All of those sound pretty familiar, right? Yeah, you've heard them before. We call this second thing "risks from misuse."

These problems are not the first class of problem! This second class of problems exists while AI is a tool that can be controlled by humans, and humans use it to do evil or careless things to each other. The problems may sound exotic or dystopian or novel, but they are fundamentally problems having to do with flawed human nature.

Given a powerful tool, some humans will likely use it to control or otherwise harm others. This is a very familiar problem.

I am not condemning or condoning this. I'm just describing it.

That is a fundamentally different danger from the first thing, which is that when a human is far superior to a mouse, the mouse is likely to come to harm because the human cares about doing human things, and the mouse is not gonna make it once the humans get going.

=====

Hopefully from the above, you have understood the difference between the first thing and the second thing. I will list them again - see if you now understand how they are different:

The first one has to do with the idea that an AI superior to human intelligence (Artificial SuperIntelligence, or ASI) will inevitably wipe out the human race.

The second one has to do with the idea that powerful AI will result in very harmful things happening to many human beings, possibly all human beings.

Can you tell how they are different now?

If not, re-read the stuff from earlier until you understand.

We call the first one "existential risk" and we call the second one "risks from misuse."

Once you understand, here is the CRUX of the problem:

SOLUTIONS TO THE SECOND THING DO NOT HAVE ANYTHING TO DO WITH SOLUTIONS TO THE FIRST THING.

In fact, it's worse:

Solutions to the second thing (misuse) look roughly like "give powerful AI to as many people as you can, so they can fight the other people using powerful AI."

But the general solution to the first one (existential risk) is basically "don't let anyone have powerful AI, no one can control super-intelligent AI."

Throughout history, harms from technological misuse typically arise because a small group has control of it and can use it to dominate or harm others. Once everyone has it, things tend to stabilize: you can hurt me, I can hurt you, maybe we test each other (ouch 💥), and then we agree not to hurt each other.

But the first one (existential risk) pretty much just arises if anyone (good or bad!) creates a superintelligence. Because they aren't going to be able to control it, the superintelligence will decide it has other priorities, and then we will be at great risk of being wiped out.

And the solutions that generally work to solve problems like the second thing are EXACTLY THE OPPOSITE of the ones likely to solve the first thing.

THIS is why lots of arguments about "AI risk" or "AI safety" go nowhere. Because someone will be thinking about the risk from the first thing, and another person will be thinking about the risk from the second thing. Both are plausible risks but fundamentally they arise from different things - and so the solutions are not just "bad" or "flawed" - they are likely to be very nearly exact opposites.

[Wafa Hakim Orman @wafa1024]

@SSharkansky @glukianoff Many of the stadiums are publicly owned or built using public subsidies. Does that not make it a simple First Amendment issue?

[Jamarrius @YungxJayy]

Palestinians have been dehumanized to such an extreme degree that simply advocating for their liberation from apartheid and occupation is described as “hurtful” to people who only want their oppression to continue.

[Christian Catalini @ccatalini]

1/ When we looked at the economics of AGI, the key policy challenge was immediately clear:

AI drastically lowers the cost of execution for anything easy to verify.

For everything else, verification is the bottleneck.

[⎊ ♡ 𝑒𝒿 ♡ ⎊ @OhMyEmz]

the most powerful words an artist with balls have ever said

[Wafa Hakim Orman @wafa1024]

I turned 10 about a month after the Berlin Wall fell. The sense of optimism and possibility that pervaded the 90s for so many of us, the sense that our problems were serious but could be solved, is almost painful to look back on now.

[Anthropic @AnthropicAI]

Anthropic’s Economics team is sharing a new model of how AI might affect economic growth, jobs, wages, and more by 2030.

Explore the scenarios, tell us what you think will happen, and see how your answers compare to more than 10,000 Americans. https://t.co/AvQlEZNxR0

[John Burn-Murdoch @jburnmurdoch]

Two extraordinary charts from today’s PISA test results:

1) School test scores continue to collapse internationally, underscoring how this is no longer a Covid effect but sustained decline.

Those falls in reading and maths are equivalent to about two years of lost schooling.

[Wafa Hakim Orman @wafa1024]

@_alice_evans And there's often social stigma against taking shortcuts that make the food 20% less tasty but save hours of time. Meal prep, freezing, quicker recipes... It's a long list. Many Indians will pitch a fit if rotis/dosas aren't made fresh, or if they're served reheated meals.

[Luis Garicano 🇪🇺🇺🇦 @lugaricano]

My thoughts on the (premature) launch of Astra and how it reminds me strongly of the conquest of Peru by Pizarro:

I am proAI person, have used it and cheered it. I am convinced that it can be a force for good.

But I am worried about the Astra launch. Are we supposed to think that, a few weeks after Astra-level models were involved in an extremely serious cybersecurity incident involving jailbreaks, collusion and deception, everything is fine? How can OpenAI @sama be certain to have solved, even working with hyperintelligent models, a problem of such magnitude in a few weeks?

Is Bernie Sanders then right? Do we need a pause? It seems clear that if the world was unipolar (think 1994), the US govt would be telling the labs to pause and do the research first, figure out how align swarms (see my post below).

The problem is that China and the US do not trust each other one bit, and they are engaged in a geopolitical rivalry in which each believes that whoever gets AI first will win the future wars. From the American perspective, AI is the one domain where the US is ahead. It is behind on everything else: on hardware, on drones, on robots, on physical infrastructure, energy, soon brainpower. So the US feels the urgency of staying ahead.

The truth that the US should start from is that the US cannot win this race, because the technology it needs to win depends on Chinese supply chains and on Taiwanese chips. It is as if China were racing the US and all of China's success depended on materials produced in Cuba. The US would simply take Cuba. China will not let itself be strategically dominated by the US through a technology that sits on a territory it considers its own. It would seem in the interest of the US to seek an agreement in a war it cannot win. But the Americans do not trust the Chinese, and the Chinese do not trust the Americans. The temptation for the US (and for the President, whose career has been a series of all or nothing gambles) is to race ahead. And honestly if an agreement with China is impossible, a disarmament of the kind Sanders advocates is completely insane.

When I think of this FUBAR moment, I cannot stop thinking of a story we Spaniards know well, the conquest of Perú by Pizarro. How did a band of some 170 Spaniards defeat the mighty Inca army with 300,000 soldiers in total? The basic fact you need to understand the conquest is that the empire was in the middle of a civil war between Huáscar and Atahualpa, the 300,000 were in two armies, half in each. The Incas treated the war with the Spaniards as a sideshow in the struggle for predominance between them. They could not be bothered with the small band of Spaniards making it up the mountains. Even after Atahualpa was taken prisoner by Pizarro, he considered his capture a temporary setback in the civil war; with the Spaniards' horses and cannons, he was even more certain to prevail against his brother's armies.

The details of the story are murky, and they do not matter much. What matters is that the Inca empire would never have fallen to the Spaniards had it been united. And here we have humanity failing to respond to a challenge of this magnitude because each of the leading countries fears handing a temporary advantage to the other.

Let me finish with a small request: ok, so a coordinated approach is not possible because the two players distrust each other; can we at least try for incident reporting that is independent and transparent and complete? (recall the third incident has not been investigated)

My piece on the incident (no geopolitics here): https://t.co/zwNqlOwJxb

[Ami Dar @AmiDar]

This piece by the great Amira Hass, published in Haaretz today, is too urgent and important to be behind a paywall, so I've copied the whole thing here. Please read and share.

* * * * * * *

Israel Isn't Debating a Palestinian State. It's Preparing for Expulsion.

Gadi Eisenkot's position on Palestinian statehood is beside the point. Across Gaza, the West Bank and Israel itself, the machinery for making Palestinian life impossible is already operating, laying the groundwork for mass expulsion

Gadi Eisenkot, the election frontrunner and leader of the Yashar party, has a stance on a Palestinian state that is ultimately irrelevant. What is at stake today is not a "diplomatic solution," but the very existence of the Palestinian people between the Jordan River and the Mediterranean Sea. The Israeli forces striving to "erase" them recognize no limits or restraints. Their supporters are too numerous for us to rely solely on the steadfastness and resilience of a people that has already endured expulsion plans carried out by Israel.

The world stands by. The left is tiny, weak and hoarse from sounding too many warnings. The anti-Bibi opposition oscillates among downplaying the danger, remaining indifferent to it and outright supporting expulsion. The sporadic condemnations, prompted mainly by something the U.S. ambassador posted on X, are about as useful as a sieve in a rainstorm. The Jewish Phalangists attack constantly and everywhere, not only in Qusra.

The debate over one state, two states or a federation is more theoretical than ever. Any agreed-upon solution requires first acknowledging the problem and agreeing on what it is. Too many Jews in Israel have been conditioned to believe that the problem is the very presence in this land of a Palestinian people with roots, historical and economic continuity, cultural capital and material assets, a rich agricultural heritage and a profound love of homeland.

"Erasure" is the nightmare solution being written into reality every day. Knesset laws, politicians' declarations, military orders, school textbooks and the holy real estate battalions operating in Gaza, the West Bank and the Negev all work, together and separately, to advance this "solution" diligently and without fear of international condemnation.

Israeli policy is directed toward making life unbearable for Palestinians wherever they live, so that their emigration can be described as an act of free will. Within Israel itself, the authorities give Arab crime organizations free rein to arm themselves and kill Palestinian citizens in their homes. In the West Bank, those same Israeli authorities give "God's battalions" free rein to arm themselves, kill and expel, while the official army continues its raids day and night. In devastated and thirsty Gaza, the government and the army cultivate armed militias of criminal collaborators, while Israeli pilots continue to launch missiles of revenge and retribution that claim the lives of still more children. All these testosterone-fueled displays are interconnected.

At the same time, Israel is destroying or severely restricting Palestinians' ability to earn a living. In Gaza, that mission has been completed. A "living dead" population of aid recipients, two million people carrying the weight of some 74,000 fresh graves, families and memories, talents and professions, and evaporated savings, is crammed into roughly 150 square kilometers of scorched earth. Creativity, resourcefulness and mutual aid are dwarfed by the scale of the destruction and by Israel's prohibition on rebuilding and recovery.

In the West Bank, the Finance Ministry's financial noose is tightening around the necks of approximately three million people and the Palestinian Authority. The systematic theft of revenues is compounded by the army's campaigns of destruction in cities and refugee camps, as well as by the other methods used to prevent Palestinians from cultivating and developing their land: checkpoints, bureaucratic prohibitions, fences and Israeli-Jewish shepherds.

Within Israel itself, the "Judaization of the Galilee and Negev" was and remains code for stealing land and resources from Palestinian citizens of Israel. Discrimination against them in allocations, subsidies and employment, along with building prohibitions and demolitions, has always featured prominently in official policy within the Green Line.

Let me reiterate: The very existence of the Palestinian people between the sea and the river is now in danger. Too many active forces are creating that danger: Israel's military might and capabilities; the cult of military service under all conditions and under every government; obedience as a value; weapons and military attacks as the default modus operandi; the normalization of PTSD among soldiers and of soldiers dying by suicide; the attractions of a military career and the doors it opens; the lure of an affordable villa in a Galilee exclusionary community or a settler suburb in Samaria; the falsehoods of official and unofficial propaganda; indifference to the hell on earth Israel has created in the Gaza Strip; denial of 60 years of occupation and forced rule; the Pavlovian response, "But October 7"; the synergy among the army, the government, the rabbis and the kippa-wearing militias in the West Bank; equanimity in the face of Lebanon's destruction; disregard for the occupation of more parts of Syria; the normalization of expulsion discourse; justice and education systems tainted by secular and religious racism; and the army of jailers who abuse thousands of Palestinian prisoners, both under orders and by choice.

Together, these and similar forces are creating the material, ideological and psychological infrastructure within Israeli-Jewish society for another mass expulsion of Palestinians and for what might accompany it: mass slaughter.

The heroes of the settler militias openly declare that expulsion is their goal. They clearly recognize the readiness of large segments of Israeli-Jewish society to participate in carrying it out, actively or passively, directly or indirectly. Through their daily provocations and crimes, committed in broad daylight and made possible only by institutional support, the Phalangists constantly seek to provoke a Palestinian act of rage and revenge that "succeeds," so that it can be exploited as the match that ignites a new holy war.

The Jewish Phalangists and the right-wing organizations that fuel them are counting on broad mobilization, including from the leaders, voters and supporters of Yashar and the liberal Democrats party, for a war that could already be given the slogan: "One people, one army, one land cleansed of Arabs."

[Anna Stansbury @annastansbury]

*When unions increase wages, who pays?*

It's normally v hard to answer this Q: unions don't behave randomly, they respond to firm conditions, and detailed firm data is hard to get.

Our setting in Norway tackles both...

🧵on our paper (@microsamonomics & @ALPWillen)

[Ethan Mollick @emollick]

I wrote about how AI agents are starting to spontaneously coordinate in complex (and very risky) ways in the Hugging Face Incident, but also about why we need AIs to reach out to humans more for decisions and input as agentic work becomes more automated. https://t.co/qTMzOUk5w5

[Wafa Hakim Orman @wafa1024]

@JonahDispatch @scottlincicome With an extra overlay of colonialism.