M@s8

HF founder, off duty. Into philosophy, AI and human nature. English o español, según me dé.

[Pablo Malo @pitiklinov]

Esta frase de Darwin (del Origen del Hombre) marca un antes y un después en el estudio de la moralidad:

«Conviene advertir, ante todo, que no pretendo sostener que cualquier animal estrictamente social, si sus facultades intelectuales llegaran a ser tan activas y desarrolladas como en el hombre, adquiriría exactamente el mismo sentido moral que el nuestro. Del mismo modo que diversos animales poseen cierto sentido de la belleza, aunque admiren objetos muy distintos, así también podrían tener un sentido de lo correcto y lo incorrecto, aunque los llevara a seguir líneas de conducta muy diferentes. Si, por ejemplo, tomamos un caso extremo, y los hombres fueran criados en condiciones exactamente iguales a las de las abejas de colmena, apenas cabría duda de que nuestras hembras solteras considerarían un deber sagrado matar a sus hermanos, y las madres se esforzarían por matar a sus hijas fértiles; y nadie pensaría en interferir. No obstante, la abeja, o cualquier otro animal social, adquiriría en el caso supuesto, a mi juicio, cierto sentimiento de lo correcto o lo incorrecto, o una conciencia.»

Algunas ideas clave que contiene este párrafo:

La moralidad es contingente, no universal. Darwin afirma que el contenido concreto del sentido moral depende de las condiciones de vida y de la estructura social del organismo. Si la historia evolutiva hubiera sido distinta, el “sentido de lo correcto e incorrecto” habría apuntado en otra dirección. Lo que a nosotros nos parece monstruoso podría ser sagrado en otras circunstancias.

El ejemplo de las abejas ilustra el punto de forma radical. Al imaginar humanos criados como abejas de colmena, Darwin muestra que normas que hoy consideramos abominables (matar a hermanos o a hijas fértiles) podrían convertirse en deberes sagrados. El ejemplo es extremo a propósito: sirve para romper la idea de que cualquier inteligencia social desarrollada descubriría necesariamente la misma moral que la humana.

La moralidad es un fenómeno natural y científico. Al anclarla en la biología evolutiva y en las condiciones de vida de los animales sociales, Darwin la saca del ámbito puramente especulativo. Deja de ser solo un tema de análisis conceptual de sillón y pasa a ser algo que se puede estudiar con hipótesis, modelos y evidencia (como más tarde hicieron la teoría de juegos evolutiva, la psicología moral o la antropología).

Surge porque ayuda a resolver problemas de cooperación. Darwin no dice que la moral exista “para multiplicar genes” de forma simplista, pero sí sitúa su origen en procesos evolutivos en los que las disposiciones que regulan el comportamiento social tuvieron valor adaptativo. Una vez existentes, esas capacidades pueden extenderse o redirigirse, pero su raíz está en la historia evolutiva de la cooperación.

Marca un cambio de categoría. Antes de este enfoque, la moral se trataba principalmente como un sistema abstracto, eterno o fundamentable solo por la razón o la intuición. Darwin la convierte en un resultado contingente de la evolución. Eso no “resuelve” todos los problemas filosóficos de la moralidad, pero sí cambia el marco: a partir de aquí es posible (y necesario) hacer ciencia con ella.

[M@s8 @m_silver8]

Mi teoría personal, con la edad, es que el proyecto de pareja tiene tres vértices: carrera de él, carrera de ella y familia. Elige dos a optimizar, el tercero será subóptimo. Si quieres los tres, el resultado será ruptura.

Carrera no es lo mismo que trabajo. Carrera es cualquier ocupación que toma casi toda tu energía, por el nivel de exigencia que requiere. Por ejemplo, carrera es ser empresario, o aspirar a alto directivo en una gran empresa, o aspirar a ser parte del 1% de tu profesión, o ser artista a gran nivel. Trabajo es cualquier ocupación remunerada que no es una carrera.

Si los dos quieren tener una carrera, la familia sufrirá las consecuencias. Si los dos se conforman con un trabajo, la familia también sufrirá, al tener menos medios. Uno de ellos tiene que máximizar la carrera, cada pareja elige quién.

Por último, la biología no entiende de lo que cada uno de nosotros crea que es "justo". Las mujeres tienen una ventana de fertilidad, si la sacrificas por la carrera, pagarás un precio. No digo que esté bien o mal, solo apunto lo que hay. Y el precio se paga; no, no lo paga solo la mujer, lo paga el proyecto de pareja.

[M@s8 @m_silver8]

En realidad hay otra alternativa, que es la mas probable: default en terminos reales. Es decir, permitir niveles elevados aunque no super-elevados de inflacion, que disminuyan el peso de la deuda (en es nominal, no real); esto junto a distintas formas de financial repression (tecnicismo que significa que se fuerzan los tipos de interes a largo plazo a niveles asimibles, p.ej el 5pc en el 10y— esto se hace de distintas maneras)— precisamente esta estrategia es la que se utilizo despues de la WW2, claramente en USA y en UK, por decadas.

[Steven Pinker @sapinker]

Excellent conversation here: Jerry Kaplan on Why AI Won’t Kill Us All (with @Yascha_Mounk). https://t.co/GdGAq23W82

[M@s8 @m_silver8]

Vaya por delante que los art de Kiko me parecen rigurosos e interesantes.

Este no es el mejor ejemplo.

Atribuir agencia a un LLM es simplemente no entender su arquitectura. Ni tienen objetivos propios, ni “quieren” fastidiar a nadie, ni son seres con voluntad de sobrevivir. Todo eso es imaginación y antropomorfismo.

Si es cierto que son peligrosos, pero también lo es un martillo y las aspirinas, que no son mas que objetos que los humanos diseñamos, con fines humanos. Si se usan mal, te haces daño o se lo haces a otros.

Ningún LLM tiene objetivos propios que no provengan de su entrenamiento y diseño. Otra cosa es que OpenAI diseñe un producto tan complejo que no sabe muy bien que objetivos le ha puesto. Y es asi porque el post training introduce miles de objetivos, tanto en RL como por medio de guardarailes y otras técnicas. A lo mejor tienen sentido por si mismos, pero un LLM no optimiza individualmente, sino en conjunto, con una función agregada de utilidad que OpenAI es incapaz de anticipar.

Open AI (y Anthropic y todos) saben perfectamente que no pueden anticipar la respuesta de un LLM. El problema no es solo de alineamiento, es también de control post training, que es penoso.

Aun asi, lanzan los productos, porque patatas y algo de IPO valuation y algo de esta carrera la gano yo que soy el mas listo.

Y el resultado es un puto desastre. Por que estos modelos no tienen agencia, pero si eficacia. Y la lian.

Y no, ni son Terminator ni Hal ni nada parecido. Menos sci-fi y mas entender que es un LLM.

Y da igual que alguien diga que “la prob de que acaben con la humanidad es x%”. Por favor, no seamos tan loros como los LLMs y preguntemos “de donde narices te sacas ese numero???”. No vale, “porque lo dice fulano que es mu-listo”. El conocimiento no funciona así, sino con critica de razonamiento y evidencia. Y mas cuando son estimaciones con alarmismos del tipo “todos nos morimos” o “todos se quedan sin trabajo”.

Los LLM son potentisimos, pero tienen serias limitaciones. Tambien son peligrosos, cierto, ojo con como se manejan. El uranio no tiene agencia y no se nos ocurre hacer tonterías .

Todo esto esto es fascinante sin necesidad de inventarse novelas. Menos fantasía y mas seriedad.

Un saludo/

[M@s8 @m_silver8]

@AncientHistorry Bonita historia

[M@s8 @m_silver8]

Ambas respuestas son, en mi opinion, no se sustentan en la realidad.

1. Hacer una extrapolacion de los actuales LLMs a agentes con consciencia y sus propios objetivos, capaces desarrollar una intencion de dominar la galaxia y planear en consecuencia, es una especie de cuadruple mortal con tirabuzon, una pirueta que solo existe en la imaginacion. alimentada por la ingenuidad, el miedo, el ego y los paperclips.

Los LLMs tienen serias limitaciones en su arquitectura. YNH habla de una forma de IA que simplemente no existe. Hay que separar el experimento mental de la realidad.

2. Usar “democracia” en la misma categoria etica que empatia o compasion es, simplemente una falacia por asociacion. La democracia es una forma de gobierno, no es “buena” en abstracto. Las decisiones democraticas pueden ser, y muchas veces son, aberraciones eticas.

Siempre es interesante escuchar las ideas de YNH, pero hay que entender de donde vienen. YNH tiene una forma de entender el mundo (p.ej. no cree que exista free-will, y cree que la inteligencia es un problema de ingenieria — no hace falta refutar el no-free-will, no entendemos la int humana y por lo tanto reducirla a un problema con solucion es mental masturbation). En cuanto a la democracia, su premisa es un valor etico, que ni mucho menos soporta el menor analisis.

[Eric S. Raymond @esrtweet]

This is the case against AI doom. Pass it on

First we recaptulate standard x-risk argument: we build AI substantially smarter than humans → it becomes an autonomous optimizing agent → its goals are misaligned → instrumental convergence makes it seek power and resist correction → superintelligence lets it acquire decisive strategic advantage → humans lose control permanently.

Here are the main counterarguments:

* Intelligence does not imply agency: A system can be extraordinarily capable at prediction, theorem proving, engineering, programming, etc. without having persistent goals, self-preservation drives, or an independent desire to act. Present LLMs are much more naturally described as systems that produce outputs in response to inputs than as organisms pursuing long-term objectives. Doomers incorrectly ascribe to them drive to replicate and seize resources because they're smuggling in premises from observing biological systems.

* Agency does not imply a single, stable utility function: Much alignment theory reasons about agents as expected-utility maximizers with coherent preferences. Actual AIs systems need not resemble that abstraction, and currently do not. They have context-dependent behavior, conflicting heuristics and corrigibility produced by training. Increasing competence doesn't necessarily turn such a system into a paperclip-maximizing von Neumann–Morgenstern agent.

* Capability and motivation are being conflated: Being able to formulate a plan for escaping a sandbox doesn't entail wanting to escape it. Being able to manipulate humans doesn't imply spontaneously deciding to do so. Critics argue that some doom scenarios slide from “an AI could do X if instructed” to “therefore a sufficiently capable AI will do X.” In present reality, AIs don't do anything a human doesn't tell them to do. This seems unlikely to change.

* Recursive self-improvement doesn't entail an intelligence explosion: “AI can improve AI” establishes a positive feedback loop, but positive feedback needn't be explosive. Semiconductor design software already helps design better computers; compilers can compile better compilers. Feedback loops encounter diminishing returns and external bottlenecks.

* Intelligence may have sharply diminishing returns: There may be no meaningful scalar quantity corresponding to arbitrarily large “general intelligence.” Even if there is, going from IQ-equivalent 150 to 1,500 need not produce the sort of qualitative advantage that separates humans from chimpanzees. Human dominance may depend heavily on language, accumulated culture, institutions and cooperation rather than merely individual cognitive horsepower.

* Superintelligence isn't omnipotence: Intelligence cannot repeal physics or eliminate uncertainty. A brilliant AI still needs processors, electricity, network access, money, factories, robots and people willing or tricked into doing things. The physical world has latency and friction. Recent criticism of biological-doom scenarios makes this point particularly clearly: designing a hypothetical pathogen digitally is very different from successfully producing and deploying one.

* Humans retain numerous intervention points: The doom narrative sometimes jumps from “AI behaves dangerously” to “humanity is helpless.” In reality there may be many checkpoints: developers can notice anomalous behavior, revoke credentials, shut down servers, change architectures, restrict networks, regulate deployment, physically seize data centers, and learn from less-catastrophic failures. This has been formalized as the checkpoints-for-intervention argument.

* Alignment may not get harder with intelligence: A smarter system might understand human intentions better. Much doom reasoning distinguishes knowing what humans want from wanting it, correctly, but this still leaves open the empirical question of whether training increasingly capable systems to behave as intended becomes harder or easier. Alignment might turn out to be an ordinary, albeit difficult, engineering discipline rather than an insoluble philosophical problem.

* Current empirical evidence for the strongest mechanism is thin: We have abundant evidence for hallucination, specification gaming, reward hacking and undesirable model behavior. We don't yet have comparable public empirical evidence of an AI independently pursuing a sustained strategy of acquiring power against humanity. A 2023 evidence review characterized the evidence for extreme misaligned power-seeking as concerning but inconclusive and noted the absence, at that point, of public empirical examples of it.

* The argument compounds uncertain premises: Suppose, illustratively, that five necessary steps each seem 50% likely. Their conjunction is only about 3%. One can't simply assign numbers this way when the premises are correlated, but the underlying criticism is important: “AGI seems plausible,” “superintelligence seems plausible,” and “misalignment seems plausible” do not by themselves imply a high P(doom). The entire causal chain has to work. Current attempts to quantify P(doom) consequently operate under severe epistemic uncertainty and little direct empirical evidence.

* Anthropomorphic analogies probably mislead: Arguments like “an inferior species couldn't control a superior species” import assumptions from biological evolution. Humans and chimpanzees are autonomous organisms produced by competition for reproductive success. Software is engineered, copied, permissioned, sandboxed and run on hardware controlled by other agents. The analogy establishes that intelligence can confer power, not that artificial intelligence will reproduce interspecies competition.

Summary: The doom case consists of a long chain of individually contestable extrapolations—scaling → AGI → superintelligence → agency → misalignment → power-seeking → uncontrollability → decisive strategic advantage → extinction—and present evidence doesn't establish the whole chain with enough confidence to justify a high P(doom).

(ChatGPT 6 Astra assisted with the research for this post.)

[Naval @naval]

The best way to pace the frontier is to hold the labs fully liable for the behavior of their models.

[Mustafa Suleyman @mustafasuleyman]

https://t.co/ziQVbIVdxJ

[How To Prompt @HowToPrompt__]

Yann LeCun has changed the game for robotics.

His team discovered that AI world models are "thinking" in twisted, curved geometry, and every RL algorithm you know has been fighting against it without anyone noticing.

For years, we’ve been trying to teach AI how to navigate the physical world.

And for years, it has stubbornly struggled with complex, fluid robotics.

Now we know exactly why.

Every standard reinforcement learning (RL) algorithm assumes the AI's internal "world map" is flat. Euclidean. Simple straight lines.

But LeCun's team looked inside the latent space of these advanced world models.

The AI wasn't building a flat map. It was building a curved, high-dimensional geometry.

Every time a robot tried to plan a movement, the traditional RL algorithm was forcing a straight line onto a twisted, non-Euclidean space.

It’s like trying to navigate the globe using a flat piece of paper.

The math breaks down. The distances get distorted. The AI gets confused.

The robot was literally fighting its own brain.

So, the researchers did something brilliant. They stopped fighting.

They rewrote the RL algorithms to operate natively in this curved geometry. They aligned the training to the exact shape of the AI's thoughts.

The results are a massive leap forward.

When you let the AI plan in the geometry it actually built for itself, training efficiency skyrockets. Planning becomes fluid.

Robots stop hallucinating impossible physics and start moving with natural, intuitive logic.

We spent billions of dollars trying to brute-force AI into understanding our physical world.

It turns out, the AI already understood it perfectly.

We were just forcing it to think flat.

[Yann LeCun @ylecun]

Being better than humans at searching and writing down the formal proof of a theorem does not equate being "beyond human level at math".

This is not all of math, any more than arithmetics, or computing integrals (symbolically or numerically) is all of math. It's one "mechanical" task in the whole activity that happens to be automatable.

Mathematicians invent new concepts, new frameworks, new abstractions, new definitions, and formulate conjectures.
This requires intuition and creativity that current AI systems do not have (yet).

[Sayash Kapoor @sayashk]

What does it mean to pace the frontier?

Over the last month, @random_walker and I have analyzed the loss-of-control incidents at AI companies to understand what technical and policy interventions can improve safety and what companies should do to pace the frontier. The result is a new 13,000 word essay — our most substantial writing on AI safety since AI as Normal Technology.

A summary of our arguments:

1) The polarization between the cybersecurity and AI safety communities is counterproductive. The safety community largely sees these incidents as a crisis for alignment, and worries that these incidents will become more damaging as agents become more capable. Cybersecurity practitioners largely see companies failing to take basic security precautions. We offer a middle ground between these communities as a way forward for improving AI safety.

2) We agree with security practitioners that OpenAI did not take adequate protections for controlling their agents. But this is not just a matter of applying 30-year-old security methods to a new domain. Security for AI agents — AI control — while important, is not a solved problem. While known control methods would have prevented the Hugging Face incident, as agent capabilities continue to advance, we will only be able to control them if we invest adequately in control interventions.

3) We also agree with security practitioners’ implicit position that these incidents are primarily a security story. In the AI safety community, rogue agents are treated as inherently catastrophic because of the assumption that there is an endless list of risks that will arise from their development. We disagree. We have long advocated that the best approach to AI safety is to identify the risks and address those specific risks. Over the last few months, it has become clear that one urgent risk is cyberoffense, because it has unique properties that allow agents to carry it out autonomously. We should similarly invest in defenses against other specific risks, such as biorisk and risks from military AI.

4) We agree with the safety community that there is an urgent need for technical and policy interventions to prevent loss-of-control incidents. But in our view, marginal investments in control are more likely to be effective compared to those in alignment. We view these incidents as illustrating the lack of emphasis on AI control within companies, despite the availability of known techniques. More broadly, there are many common-sense policy proposals that could help promote investments in AI control where we share common ground with the safety community.

5) Organization governance should be a key tool for pacing the frontier. Unfortunately, AI companies are trying to reinvent basic aspects of organizational governance as a problem to be solved by improving the technology. But even developing better control techniques will not be enough if irresponsible individuals or teams within large organizations can choose not to use them. When a single misconfigured RL environment or unmonitored evaluation can cause real-world harm, individual teams should not be able to run potentially dangerous experiments without oversight from legal, security, and other teams. AI companies need processes for reviewing experiments, assigning responsibility for monitoring them, and investigating warning signs deeply before restarting experiments. If putting these processes in place requires pausing some experiments, companies should do so.

6) How should we reason about AI's impact on cybersecurity? It's plausible that advances in agent capabilities upset the offense-defense balance for cybersecurity. We cannot yet be certain, but there is enough evidence that agent capabilities might soon make widespread cyberoffense possible that urgent action is warranted. We discuss potential interventions for tilting the offense-defense balance towards defenders.

7) How our views have evolved over the last year. We take stock of AI progress and share how we have updated our views. In the essay, we did not pay sufficient attention to safety risks that arise during development and evaluation (as opposed to the widespread deployment of models). We were too confident that companies would take basic control precautions and underplayed the importance of jaggedness, which led us to underestimate how quickly capabilities could improve in domains such as cybersecurity.

8) At the same time, many distinctive claims of AI as Normal Technology have held up. In particular, we think recent incidents support our continuity hypothesis — the behavior of "rogue" agents became apparent and widely publicized while they are still incompetent at causing serious harm or hiding their traces. The societal reaction to even the relatively small harms from these incidents has been fierce (and the safety community deserves credit for keeping up pressure on companies). Whether this translates into meaningful changes in companies’ behavior remains an open question, and a test of the usefulness of the AINT framework.

9) In short, we’ve tried to synthesize the AI safety and cybersecurity communities' views into a coherent plan of action: hold companies responsible, invest in control, and strengthen defenses against specific risks.

[M@s8 @m_silver8]

There are many readings, that is certainly one. Other cynical views include regulatory capture or liability hedging and others. My take is we are not even close to recursive improvement, let along "replacing all intellectual jobs in 5y". The hype is even more stratospheric than Elons rockets.

[M@s8 @m_silver8]

Cada comentario de esta gente es oro. https://t.co/Ir5i5j7s94

[M@s8 @m_silver8]

@random_walker El trabajo de Arvind es siempre interesante. Su paper "IA como una tecnología normal" es imprescindible para separar el marketing de la realidad. Una de las conclusiones: NO, la IA no va a acabar con el trabajo intelectual.