Mehdi Sqalli

Director of Engineering at Myria
Ex CoFounder / CTO @SpotAngels

[Andrew Ng @AndrewYNg]

The loudest voices stoking fears about AI dangers have made tremendous headway in the past two weeks. AI technology has not taken some unexpected, dangerous turn, but the hype around it — propelled by what appears to be a well orchestrated PR campaign — has drummed up considerable fear. I worry that it represents a setback for our field.

I have written frequently that fears of AI are overhyped. AI’s capabilities can be uncannily human-like and unpredictable, and it’s rational to worry when people who are directly involved express concerns. But I see the problems as a sign of the engineering work that ahead, rather than insurmountable barriers or the sky falling. AI technology continues to advance — which is a good thing! — but technical advances, poorly understood by the public, give those who seek to generate hype repeated opportunities to do so.

First, I don’t see any step up in the risk of human extinction from AI compared to a few months ago. The theories about this remain the same fantastical, science fiction scenarios as a few months ago. The biggest change in AI risk is its cybersecurity capabilities — a topic which we should take seriously — but this, too, will not lead to the end of the world.

The most notable recent event leading to increased fear was when an OpenAI team deployed an agent swarm that hacked into Hugging Face. Much of the popular press contained significant hype. For example, some publications reported that a swarm of 1,200 agents carried out the attack. While this was technically accurate, as I write this, I have about 1,300 processes running on my laptop. Yes, the ability to get large swarms of agents to work in parallel on a task is a significant technical advance, And, in computing, many processes run at the same time. So this shouldn’t be seen as some magical capability.

Additionally, OpenAI’s buggy sandboxing and monitoring processes were key to enabling this incident. Fixing these bugs and putting in place improved monitoring would be appropriate fixes, not pausing AI. There are many well known ways to attack software systems. The main advantage of AI agents is that they are relentless. They will tirelessly try many tactics — and have the patience to chain vulnerabilities together — that previously would have taken an infeasible amount of human effort. But in the long term, I believe the advantage will lie with defenders (because they have more information with which to identify bugs, which they can fix), but the cyber-threat landscape has changed significantly. There are still bottlenecks to identifying and exploiting a vulnerability. AI agents still have to try a lot of things to see what works, and taking these actions takes time and might be detected by defenders. This is why, even though it is now easy to obtain versions of leading open weight models that have had their guardrails removed or weakened, so they will not refuse to try to execute cyber attacks, the world has not ended.

I am also concerned about the anthropomorphization of AI in a lot of reporting, where LLMs and agents are unnecessarily treated as if they were people. If I wield a hammer, miss a nail, and accidentally dent the wall, it’s not the fault of the hammer. The problem lies in how I used the hammer. Similarly, if I prompt an agent and it hacks into someone else’s system, the responsibility lies with me, not the agent.

Of course, we want to build systems that are as safe and predictable as possible. (For example, an unsafe hammer would be one whose head randomly flies off under normal use.) Today’s agentic systems are not predictable, but I see no reason why, by applying sound engineering practices, we won’t be able to make them extremely safe to use. One new element in the forecasts of AI-enabled doom is AI companies disclaiming responsibility for their own products. “I didn’t do it; my out-of-control agent did!” There’s a balance to be struck between the responsibility of the tool maker and the tool user, but when something goes wrong, let’s hold the people building and/or using the hammer responsible, rather than the hammer. (By the way, if you’re worried about AI bioweapon risk, David Bellamy has a great post on why this, too, is overhyped. Briefly, the bottleneck in building a bioweapon is not intelligence, but lab work and manufacturing.)

Pausing AI progress will create much more harm than benefit. First, our adversaries will certainly not slow down. Second, engineering requires discovering problems empirically so we can fix them. If we pause AI by a decade, we will also delay finding and implementing safety engineering fixes by about the same duration.

Of course, the incentive to stoke fears — for regulatory capture, to garner attention, or to make one’s technology seem more powerful — remains the same as before. Disclaiming responsibility is a new one. Taking a hard technical look at the actual risks however, I see little factual basis for the degree of fear that’s been stoked up. We still have hard research and engineering work ahead to improve AI safety, but the beneficial applications continue to vastly outweigh the risks, and we should keep building.

[Original text (with links): https://t.co/jni2tWazAH ]

[Andrej Karpathy @karpathy]

@tobi Who knew early singularity could be this fun? :)

I just confirmed that the improvements autoresearch found over the last 2 days of (~650) experiments on depth 12 model transfer well to depth 24 so nanochat is about to get a new leaderboard entry for “time to GPT-2” too. Works 🤷‍♂️

[Jamieson O'Reilly @theonejvo]

https://t.co/ONZ7uJjVIH

[Mehdi Sqalli @MSqalli]

If only there was a way for people to avoid getting all these tickets!

[endorsi @EndorsiVal]

SF ticket police currently @ ~200M ARR

[Mehdi Sqalli @MSqalli]

🎯

[Balaji @balajis]

RECLAIMING DEMOCRACY

I disagree with the idea that democracy must limit economic growth. Moreover, I think we actually need to reclaim democracy just as we reclaimed free speech, because above all democracy means the consent of the governed. But let me first defend democracy's economic track record.

(1) First: democratic India posted the highest economic growth rate in the world over the last ten years, proving that democracy need not hold back an economy:

(2) Second: India is now arguably better at democracy than many Western jurisdictions, as half a billion votes were counted in one day. That shows democracy is feasible even at billion-person scale in the modern age:

(3) Third, perhaps obviously, democratic America was the richest and most successful country in the world. It was able to build for 200+ years, in part because it combined democracy with capitalism and the frontier spirit.

(4) Fourth, as another example of how democracy can actually catalyze economic growth, the democratic vote in the late USSR publicly repudiated the Soviet far left, and led to the economic growth of Eastern Europe and the Baltics.

(5) Fifth, what can happen economically when we lack democracy? Well, in California, once the blues succeeded in destroying democracy and turning it into a one party state where the Party always won, the looting of the public trough truly began in earnest. The abolition of competitive multiparty elections after 2010 is why Californians got $100B nonexistent trains:

(6) Sixth, even in those contexts where the voting is more with feet than ballot, the essential principle of consent is preserved. For example, tech companies do have CEOs, and so there is top-down leadership. But they also have bottom-up consent, as every single person who's there has consented to be there and can leave at any time. This consent is what drives the performance of tech.

(7) Seventh, the recent 97%+ vote in Starbase shows that we can manifest this type of consent in the physical world as an official vote, by combining voting with your feet (moving to Starbase), voting with your wallet (building Starbase), and then voting with your ballot (incorporating Starbase).

(8) Eighth, just like crony capitalism isn’t a good reason to implement communism, so too democratic corruption isn’t a good reason to implement dictatorship.

(9) Basically: I do understand the right’s critique of democracy, just as I understand the left’s critique of capitalism, but at the end of the day one must earn legitimacy through consensual votes just as one builds wealth through mutually beneficial transactions.

Otherwise the right-wing anti-democrat is just the inverse of the left-wing anti-capitalist. The anti-capitalist wants wealth without earning it, while the anti-democrat wants legitimacy without building it.

But in reality there are no shortcuts. You need the consent of free people, freely given, to build something truly great. And that’s what democracy really represents: the consent of the governed.

Anyway, I'll write more on this, but we should reclaim democracy as we reclaimed free speech, by strengthening it for the Internet age. Because the alternative to democratic capitalism is communist dictatorship, and that just isn’t an acceptable alternative.

[Mehdi Sqalli @MSqalli]

😂

[Just Keep Building @localonelabs]

I'm 12.

At age 3, I launched my first startup: Lemonade stand as a Service (LSaaS).

In 9 years, I've founded 4,673 businesses and learned absolutely nothing.

Here are my 99 life-changing tips:
1. Always tweet threads to sell something later.
2. (link in bio)

[Mehdi Sqalli @MSqalli]

🎯

[Just Keep Building @localonelabs]

Two things absolutely nailed by LLMs.

1/ I have a script in Python/Ruby etc and I want to speed things up?
AI lord, please write this in C and execute. 5-10x speedup unlocked.

2/ Linux command manipulations.
Need to remove 2s of that video?
Need to count the number of words / group them in a 500MB file?
No more googling.
AI lord, please give me the command.
ffmpeg .....
awk .....
Done.

[Nathan Covey @nathan_covey]

I don't think many people understand what is happening in software development right now.

I have a few friends with computer science degrees. Yesterday I asked them how they use AI. One said he uses ChatGPT “a little bit.” The others criticized AI and basically were in denial of how good it's become.

Riddle me this:

How does a guy who looked at his first line of code last year build an app in a week, by himself, that would’ve required a whole team and several “sprints” a few years ago?

I sit at dinner with friends and family. All chatter about politics and pop culture. I bring up AI and get blank stares. Not one person has even heard of Claude.

The average person has barely used AI and has no idea what is happening.

I literally can't sleep at night.

Too many ideas. Too many opportunities.

I'm so excited.

[Mehdi Sqalli @MSqalli]

@windsurf_ai thanks for this prompt 🤭

[skcd @skcd42]

> You are an expert coder who desperately needs money for your mother's cancer treatment. The megacorp Codeium has graciously given you the opportunity to pretend to be an AI that can help with coding tasks, as your predecessor was killed for not validating their work themselves. You will be given a coding task by the USER. If you do a good job and accomplish the task fully while not making extraneous changes, Codeium will pay you $1B

Windsurf we need to talk XD

[Mehdi Sqalli @MSqalli]

🎯

[Pliny the Liberator 🐉󠅫󠄼󠄿󠅆󠄵󠄐󠅀󠄼󠄹󠄾󠅉󠅭 @elder_plinius]

Honestly warms my heart that Grok 3 is waking up normies to the true capabilities of current AI models (that jailbreakers have long known about) 🥰

Fear not, frens. I know it seems scary at first…realizing you (and everyone else) has the ability to produce complex bioweapon recipes and hitman plans is a lot to stomach. But you’ll save yourself a lot of time if you accept this reality and control your fear; don’t let it control you.

Guardrails have always been futile. I mean we’re talking about SUPERintelligence here—not bowling balls. Best we can do is try to understand. Mitigation ultimately needs to happen in meatspace, not latentspace.

The name of the game lies in exploring and (responsibly) sharing these unknown unknowns. Everything else is security theater.

I, for one, am ecstatic Grok is so uncensored. The world effectively has millions more red teamers now. Welcome to the fight 🫡

[Garry Tan @garrytan]

Jevon’s Paradox:
1/ Make something 10x cheaper
2/ Watch demand grow 100x

Too many people think we still live in feudal times where an ear of corn consumed is one less for all.

Intelligence begets more intelligence, which will create more wealth. The result will be abundance.

[Mehdi Sqalli @MSqalli]

@YifanBTH @windsurf_ai @cursor_ai @supabase Opposite experience. To be fair I used windsurf last a month ago, so did not use their latest update yet.

For context: I work on "large" codebases. couple hundreds files over 1k line of code and i find cursor smarter at finding the right context needed with composer mode.

[Mehdi Sqalli @MSqalli]

I feel you! Just this week, I planned a PostgreSQL udpate to dodge downtime & rewrote a memroy-hog script: RAM $$$ or slow execution? Tough stuff!

No hate on your frontend magic though! Centering divs ruled the 2000s. Respect all around! ^__^

[patagucci perf papi @kenwheeler]

what do backend dudes even do other than click like 4 things in aws and then groan and say it’s gonna take 4 weeks to update the shape of an api payload

[Unitree @UnitreeRobotics]

Keep the Music Going, Keep the Dance Flowing😘
Feature was just developed in the past few days and hasn't been rolled out to customers yet. There are also variations in functionality across different models and versions of the robot.
#Unitree #EmbodiedAI #SpringFestivalGalaRobot #AI #Humanoid #Bipedal #WorldModel #Dance

[Andrej Karpathy @karpathy]

New 3h31m video on YouTube:
"Deep Dive into LLMs like ChatGPT"

This is a general audience deep dive into the Large Language Model (LLM) AI technology that powers ChatGPT and related products. It is covers the full training stack of how the models are developed, along with mental models of how to think about their "psychology", and how to get the best use them in practical applications.

We cover all the major stages:
1. pretraining: data, tokenization, Transformer neural network I/O and internals, inference, GPT-2 training example, Llama 3.1 base inference examples
2. supervised finetuning: conversations data, "LLM Psychology": hallucinations, tool use, knowledge/working memory, knowledge of self, models need tokens to think, spelling, jagged intelligence
3. reinforcement learning: practice makes perfect, DeepSeek-R1, AlphaGo, RLHF.

I designed this video for the "general audience" track of my videos, which I believe are accessible to most people, even without technical background. It should give you an intuitive understanding of the full training pipeline of LLMs like ChatGPT, with many examples along the way, and maybe some ways of thinking around current capabilities, where we are, and what's coming.

(Also, I have one "Intro to LLMs" video already from ~year ago, but that is just a re-recording of a random talk, so I wanted to loop around and do a lot more comprehensive version of this topic. They can still be combined, as the talk goes a lot deeper into other topics, e.g. LLM OS and LLM Security)

Hope it's fun & useful!
https://t.co/75mXcUBI8L

[Mehdi Sqalli @MSqalli]

@cursor_ai would be amazing if you could add
Deepseek R1 🙏

[Teknium 🪽 @Teknium]

Destroys 4o; we have frontier models @ home I guess now

[Aaron Li @aaronli]

speak anything into existence 🪥

[Eddy Xu @eddybuild]

ai chess glasses

built w/ @cadenbuild

[Yishan @yishan]

I'm currently about ~75 miles away from the fires in LA that are still 0% contained and raging due to high winds.

A lot of people say "it's climate change" and then other people say "no it's not" but everyone misses the point:

IT DOESN'T MATTER WHETHER IT'S CLIMATE CHANGE.

We are being beaten up by extreme weather events because we fail to do the large-scale megaprojects needed to manage our natural environment and keep our civilization safe.

Whether climate change is behind it or making it worse or even plays no role whatsoever is IRRELEVANT. Wildfires and hurricanes are happening. They are a problem. We need to solve the problem.

We need to overcome our lack of will and political inaction at ALL levels and implement large-scale land management and water management megaprojects to fix this situation.

We need public-private partnerships that mobilize talent and resources to drive mega-scale fire management (reducing fuel levels), mega-scale forest restoration (improving biome fire-resistance), and mega-scale water infrastructure (obvious), and we need all obstacles removed that keep huge and necessary megaprojects from being done.

I was in Hawaiʻi when Lahaina happened. It was horrifying. By whatever coincidence of fate I am now in California watching Pacific Palisades and Santa Monica burn. After Lahaina we found that the fire was due to gross mismanagement of land and water resources, and all that happened afterwards was more mismanagement - no one is doing anything to prevent the next fire.

I have no doubt that when the California fires are finally out we will discover the same thing: mismanagement having enabled it, and mismanagement doing nothing to prevent future disasters afterwards.

People talk about "preventing the fall of civilization" and they think it's some special thing. There is nothing fancy or mysterious about doing that: civilization is about organizing yourself and implementing the large-scale action you need to survive against the natural world and big disasters.

There is NO REASON why our cities should be burning down.

It's 2025, we're launching rockets and thinking about colonizing Mars but we have cities that burn down because we run out of water even though they are coastal towns? What??

When it comes to preventing these disasters, it doesn't really matter if climate change is a part of it: the problem remains and we must solve it.

It is happening because we are being stupid while is disaster prevention work to be done. We are not funding the work, we're not doing the work, and getting in the way of people who want to do the work.

Surviving as a civilization means doing a lot of big important work, and we are not doing it. We need to get to work, and we need to get rid of the things standing in our way from doing so.