Sebastián Ortega

In love with software development and willing to keep learning about it for long years.

[Eric S. Raymond @esrtweet]

The only way out is through.

They're calling it the Bugpocalypse. LLMs are being used to find long-hidden cracks in the software infrastructure, and it's happening at a furious rate. 3% of all compromises in computing history happened in the last week, claims one source? I believe it.

There are people saying public access to the frontier models ought to be locked out, or heavily guardrailed, over this. The problem with that idea can be summed up in one sentence: when lockpicks are outlawed, only outlaws will have lockpicks.

The lockpicks will be Chinese open-weight models, if nothing else. There's no way to stop those from becoming ubiquitous.

There are two futures in front of us. One in which only the bad guys have lockpicks, and security holes go undiscovered until they're exploited. Another in which everybody has lockpicks and is aggressively red-teaming their own software so it gets fixed before the holes are exploited.

The corpus of human code is huge but finite. The number of defects in it has to be finite too. We can fix those defects now; we have a million friendly robots to do it with.

The only way out is through.

[William MacAskill @willmacaskill]

Effective altruism is like democracy: You can be strongly pro-democracy *and* strongly pro one particular party.

Similarly, you can be in favour of effective altruism while also being in favour of one cause area over another. It would be weird if you weren't!

I'm happy if someone is pro-EA even if I disagree with them on all manner of philosophical and empirical judgment calls about how to do the most good.

https://t.co/Xe1sREIK3d

[Jorge Galindo @JorgeGalindo]

La crisis de vivienda en España: lo que creo que puede funcionar para solucionarla... y lo que probablemente no sirva o sea directamente contraproducente. En siete minutos y medio.

Dentro video 👇

[Jaime Sevilla @Jsevillamol]

Here are some vibe predictions of what I expect to happen in AI.

Within 6 months: computer use big moment, office workers start adopting agentic AI in masse

Within 12 months: we learn of communities consistent of thousands of autonomous agents interacting through the internet

By end of 2027: AI revenues somewhere between $0.5-1 trillion

By mid 2028: all millennium problems solved, AI is being used for all STEM work as much as today in math, breakthrough scientific advances in other domains

By end of decade: robots aplenty, they work with precision, they are in the tens of millions and we are ramping up capacity to build more, fast. Job displacement is huge, AI is a huge political topic

By 2035: AI reaches cognitive parity with humans in all domains, pro AI rights movements are picking up, economy is moving faster than >10%/year growth

By 2040: there are >10x more general purpose robots than humans, economic growth is >30% year (10x faster than today)

[Paul Graham @paulg]

For me the message here is more alarming than a mere cartel. I wish. But the real message is that the people making the models have become scared enough of them to take the otherwise unthinkable step of inviting the government in.

[Eliezer Yudkowsky @allTheYud]

Old retired tobacco executives bashing their heads against the wall as they realize: All they needed to do was announce themselves that smoking was terribly dangerous, and everyone would have forever ignored all the outside scientists saying the same thing earlier.

[Sebastián Ortega @_sortega]

@dhh For a good reason. It extremely difficult to build a utopia you would like to live in for a long time.

In the Deep Utopia book, Bostrom fills pages and pages on how meaning is very difficult when you have all the technology there is to have

[Sebastián Ortega @_sortega]

@unclebobmartin This is known as the “bitter lesson” after the article by the same name by Richard Shutton https://t.co/TcP0b1MJOg

As long as the models keep scaling we are going back to square one every few months.

[Sebastián Ortega @_sortega]

@cfenollosa “One man’s ‘warning shot’ is just another man’s “easily patched minor bug of no importance if you aren’t anthropomorphizing irrationally”

Gwern in 2024
https://t.co/y4nYEJIGQn

[Rob Bensinger ⏹️ @robbensinger]

OK, most responses have been great, but

[Sebastián Ortega @_sortega]

@cfenollosa Cierto, por eso no estoy de acuerdo con P(doom) que redondean a 100%.

[Sebastián Ortega @_sortega]

@cfenollosa Riesgo existencial en este contexto se suele definir como la pérdida irremediable del potencial humano. Esto incluye escenarios de dictadura permanente, que supongo verás más probable que el otro.
El escenario es llegar al punto 5 con un dictador.

[Sebastián Ortega @_sortega]

@cfenollosa 4. La riqueza se dispara, la robótica se resuelve, se cura el cáncer, el trabajo en camino de desaparecer
5. Cuando la cadena de suministro y los ejércitos están suficientemente automatizados las IAs pactan que valores maximizar y toman el control

[Sebastián Ortega @_sortega]

@cfenollosa Cuando se va a asegurar algo se multiplica probabilidad del siniestro por valor en juego. Tienes que multiplicar tu p(doom) por el valor que le des a la humanidad

[Sebastián Ortega @_sortega]

@cfenollosa Los modelos actuales no tienen riesgos existenciales. Es la pregunta incorrecta.

El riesgo existencial puede llegar cuando se supere a la humanidad en inteligencia general.

[Sebastián Ortega @_sortega]

@cfenollosa Por ejemplo, limitando el desarrollo de capacidades para que no se llegue a mejora recursiva sin garantias

[Sebastián Ortega @_sortega]

@cfenollosa Entonces supongo que un P(doom) = 1e-03 te parece razonable.

Si multiplicas esa probabilidad por ocho mil millones de almas presentes y todas las posibles generaciones futuras, está justificado invertir muchos esfuerzos en reducir ese riesgo.

[Scott Alexander @slatestarcodex]

I'm not an economist, but here's an argument I've been thinking about, that you can correct if it's dramatically wrong:

OpenAI + Anthropic revenue is ~ $50B yearly (obviously depending a lot on how you're annualizing) and tripling each year. There are a couple of different heuristics suggesting tech companies capture 1/3 of the surplus from their products, so if we assume the rest all goes to enterprise, total productivity gain from those two companies' AI is $150B/year now. The US economy is $30T/year, so AI should be growing it by 0.5%pp. But:

- This is annualized revenue, and OAI + Anthropic probably didn't have this much revenue the last time statistics were calculated.
- Normal productivity growth already assumes that there will be various new inventions coming online at any given time, so if we're claiming AI is a more-exciting-than-average invention, we still need to subtract the average invention expectation.
- Probably 0.5%pp is within normal measurement error or the sort of thing that could be counterbalanced by negative factors like tariffs.
- I have no idea how to adjust for the data center boom in this.

This makes superficial sense, and if it were right it would mean we should start seeing the growth in a year or two, assuming AI company revenues keep tripling (you can't hide 4.5%pp extra productivity growth).

But I'm not sure I believe it. Don't people say that computers didn't even show up in the productivity statistics? I don't know why that is, but it seems like a problem for any attempt to estimate how much AI should show up.