Gabriel Francisco

Code, OSS, sveltejs, VR, lua, chatbots, LLMs and apple juice fan. 🇧🇷

[Pubity @pubity]

Xbox CEO Asha Sharma revealed she's been secretly solving Xbox support tickets for months, stepping into customer service and fixing problems for players anonymously.

She wanted to figure out what players don't like and decided to just talk to them directly.

[Gabriel Francisco @_ceifa]

@IsabelJlee we do its called mercosul

[Thomas Wolf @Thom_Wolf]

Many people I talk to find it hard to understand how the same companies can both push the frontier of AI capabilities and believe AI is a massive danger for the world.

How can you think this might kill everyone and also keep pushing the envelope?

So I’ve tried to collect and summarize the main arguments for this apparent disconnect.

Think of it as some sort of a guide to understanding the reasoning when Dario, Sam, or Elon say the danger is real.

By the way, these people have been worried about AI for a loooong time, they were publicly discussing AI risks more than a decade ago. Sam in Feb 2015, writing on his blog that superhuman machine intelligence is "probably the greatest threat to the continued existence of humanity." Elon at MIT in Oct 2014: "We are summoning the demon." Dario as first author of "Concrete Problems in AI Safety" in 2016.

Okay so how do you go from saying something is extremely dangerous to being a front-runner in building the very dangerous thing?

There are a few ways this can become rational. I'll take five of them, roughly in the order they developed.

1. We need to build it to learn how to make it safe

The earliest argument can be summarized as: “You cannot study something [you’re worried about] if it doesn’t exist.”

In 2015, AI barely worked. so people needed to make it work first to be able to even study some of the problems they anticipated.

The updated version for today's capabilities is: “You cannot learn everything about airplane safety by studying paper airplanes.” You need a real aircraft to discover real failure modes and an increasingly complex one to learn about increasingly complex issues.

Making AI more capable gives more chances to understand the issues and safety researchers something realistic to study

But you could argue: if you're the one afraid of the explosion, why be the one gathering the dynamite?

You could also just wait for other people to build it which leads to the question of who those other people will be -- which is the second line of argument:

2. Better us than them

Knowing how to make something safer does very little good if nobody listens to you. So the idea becomes: let’s make sure responsible people build the AI that will be deployed and add safety inside.

Basically, make sure the AI safety aware people will have the technical expertise, money, computing resources, and enough influence to make safety decisions stick.

At a larger scale, and in a larger multipolar world, this brings the idea that a trusted country should lead rather than leave powerful AI in less responsible hands. This is where “we need to go faster than China” comes in, alongside broader defense and geopolitical concerns.

These first arguments explain why someone worried about AI might still want to build it and stay ahead.

But there are also arguments for why one might want to do it really fast.

3. Move earlier to avoid a bigger shock later

This is probably the most counterintuitive argument: moving faster today can be seen as a way to give humanity more time later.

There are two related ideas here.

First, society needs time to learn how to handle powerful new tools. Introducing AI in manageable stages can be a way to let people discover problems, develop rules, and practice using AI responsibly. Releasing an advance earlier gives people more time to gain experience with smalle, burgeoning, capabilities before much more powerful and disruptive AIs arrives.

Second, even if AI research slows down, computing power may keep improving. A breakthrough that happens later could therefore have much more hardware available to run on, potentially producing a larger, more sudden jump in capability and impact on society.

That accumulated untapped potential is often called an “overhang.”

The overall argument is that making and diffusing incremental progress as soon as possible might prevent a much more abrupt transition later.

Obviously, it also means that we will reach increasingly powerful AI sooner, but the idea is to give more time to adapt and understand between the first useful systems and the really powerful ones.

Note that generally this depends on this earlier progress keeping the transition gradual rather than simply bringing everything forward.

─── ❖ ───

For our two next arguments, we can take two roads depending on how difficult we think AI alignment will be, that is "How easy do you think it is to make AI reliably do what you want without it deciding to go hack Hugging Face along the way".

Let’s take the first road: alignment turns out to be relatively tractable. Airplanes can fail, but careful engineering has made flying remarkably safe. Suppose we can do the same with AI.

In that case:

4. Waiting has a huge human cost

If AI can help discover treatments, improve education, or prevent cyberattacks, each week we delay it could bring preventable deaths and harm.

From this perspective, waiting is a decision with human consequences too. In a world with huge issues like climate-change, inequalities and poverty, it even become a moral argument for developing AI quickly and bringing its benefits as soon and as widely as is safely possible.

But let’s take a look at the other road: what if alignment is much harder than expected, and making highly-capable AI turns out to be easier than figuring out how to keep them from doing unhinged things?

Well, if alignment is too difficult a problem for humans to solve, then maybe:

5. AI could help us make future AI safe

And we arrive at the same conclusion again: if using AI to build safe AI is the way to solve alignment, let’s get the equivalent of a country full of geniuses helping us as fast as possible.

These genius AI could be the solution to make AI safe by helping researchers find mistakes, test ideas, and develop protections. Instead of relying entirely on humans to solve alignment, we could build systems that help us do the work, each generation could help make the next one safe.

Note that this requires the order of events to work in our favor: AI needs to become useful enough to help solve alignment before it becomes too dangerous to rely on. The hope is to build helpful, trustworthy research assistants before building systems powerful enough to become dangerous.

There are more arguments but in general, these are the main ways people concerned about powerful AI have found rational reasons to end up being the ones building it (and even to build it as fast as possible).

─── ❖ ───

On my side, I think several of these arguments underestimate the complexity of the world and how interconnected people’s reactions are. Moving faster while warning about catastrophe has psychological effects across a whole network of participants: it changes what people fear, whom they trust, and what they feel compelled to do.

And those reactions can change whether the original reasoning actually holds because we live in a world of interconnected humans, not machines (yet).

I also think these rational chains leave some of their consequences for society insufficiently explored. For instance, the concentration of power, shifts in geopolitical alliances, and changes in public opinion. These consequences matter both because they affect whether the strategy works and because they shape the world we end up living in.

But this post is already long, so I’ll leave those questions for the next one.

[Simon H @dummdidumm_]

Many of these are true - I'd add one more: Creativity is lost.

Beforehand, the act of writing code had a deeply creative side to it. It enables you to find novel solutions to problems.

Now that agents write more and more code, that creativity goes away. The job becomes

[roon @tszzl]

ultrafast inference reveals new threats. worth thinking about how quickly misaligned frontier class models running 50x faster could infiltrate systems and so on— far too quickly for human responders to stay abreast. you need autonomous detection and shutdown, not just monitoring

[Instituto Brasil pela Liberdade 🇧🇷 @ibpl_oficial]

Urgent appeal for Lito Sousa, fighting Creutzfeldt-Jakob disease. Please share this reel far and wide. Make sure to tag these institutions who could include him in life-saving trials:

Quer ajudar o Lito? Compartilhe este reel na sua rede e marque as instituições abaixo.

Cada segundo conta: 📌 @ionis_pharma — Ionis Pharmaceuticals — ION717 trial @broadinstitute
— PRiSM trial (NCT07444580) @harvard @mayoclinic

Every share and tag can make a difference. Time is critical.” #lito #SaveLito #CreutzfeldtJakob

[Gabriel Francisco @_ceifa]

@mattzcarey I get code mode but there is a lot of use cases that it just not suitable, like this one

[Gabriel Francisco @_ceifa]

@theo So now we are not supposed to ship new interfaces anymore because our llms are not used to?
One liner on CLAUDE.md btw

[Gabriel Francisco @_ceifa]

Graphs are cool

[Gabriel Francisco @_ceifa]

@nlevnaut @badlogicgames They didnt, but eleven did: https://t.co/REw3lQ9nqG

[Gabriel Francisco @_ceifa]

@matthewcp Which model? I cant get claude to login any platform, always deny it

[Gabriel Francisco @_ceifa]

@adonis_singh @dtfiori Mine does: https://t.co/UIs6k8Ln9K

[tldraw @tldraw]

https://t.co/sLwGmkSLG3

[Bernt Bornich @BerntBornich]

Introducing NEO’s 25 Degrees of Freedom, tendon-driven hands — nearing or surpassing human-level dexterity, strength, speed, and reliability.

For seventy years, robotics worked around the hand problem. The humanoid bet is the reverse: it lives or dies at the fingertips.

[Gabriel Francisco @_ceifa]

@jsneedles @Hussain_Joe any updates?

[Paul Graham @paulg]

LinkedIn was already slop. All that's changed is that it's now AI slop.

[Gabriel Francisco @_ceifa]

Just looked back at some old code I wrote 6 years ago that is super complex, lots of cool maths and physics, and I was genuinely proud of it

not sure I’ll ever feel that again after agentic coding

[Gabriel Francisco @_ceifa]

living between loving and hating when my claw does something I didn’t ask for

[staysaasy @staysaasy]

It’s 2018 and your coworker just sent you a 400 line pull request.

You get a cup of coffee and sit down to review it.

It’s beautiful. Elegant micro-refactors. Crispy method names.

You catch a few things, but that’s ok. It’s part of the dance. They didn’t consider extensibility on part of their API. Here’s a comment buddy.

They respond in an hour saying they think we should do one piece differently than your comment. Hey let’s jump into a room and figure it out. We can’t just agree to disagree, this code is too important.

The PR merges and goes to prod. You feel a shared sense of ownership and accomplishment.

That night you go to sleep and dream of that code. You can still see the shapes of it on the backs of your eyelids, your IDE syntax highlighting sparking neurons in your reptile brain.

You go to work the next day ready to go. You understand the system. N is your foundation. Time to build n+1.

[Gabriel Francisco @_ceifa]

@JakeSucky (G)old