IDK, who are you

Matriarch. Education dilettante. CS PhD. Aspiring matchmaker. Reindustrialization fan. Everything for the next generation.

[Kerwin Fjøl @zermatist]

The effect of the Great Books was to create the impression that America is the ultimate manifestation of western thought in its entirety rather than a distinct ethnic instantiation of it, and it fostered the idea that all ideas reach their final, universal form in English. (cont)

[IDK, who are you @antiaudience]

Academia lagging industry. This has been true in other areas of tech / computer science for a while as well.

[doubleunplussed @doubleunplussed]

On the Midjourney scanner:

People are saying that whilst ultrasound can technically go through bone, it requires crazy complex and intensive computation to get useful images from the data.

This may be true in the context of academia, but I think needs to be interpreted with some perspective. I think there is currently a yawning gap in the research capabilities of the AI private sector vs academia, such that what seems near impossible for the latter may not be for the former.

Having been around some academic medical imaging research groups (not working in them, but within the same physics department), I think it's underappreciated how small and not especially selective many research teams are. Most research is done by PhD students, who are cheap labour and generally clever, but not at the top of their careers, often still learning the basics whilst pushing forward some small aspect existing approaches. Computational and algorithmic skills are patchy and inconsistent. And people good with computers generally go do something else for more money instead of remaining in academia.

Sure these research trams are all smart people, but at this point in history the intellectual heavyweights when it comes to computationally/algorithmically optimising processing large amounts of data are in AI labs, not little university medical imaging research teams.

So to me, pointing out that the academic sector has found a problem to be computationally difficult, verging on impossible... kind of just reads as it being non-trivial, and says next to nothing about how difficult it would be for an AI company, who just have way more brainpower at their disposal.

So I'm not saying AI companies will easily solve these issues, just that academia finding it hard just kind of doesn't feel like much evidence that it is, given my low expectations.

Obviously there is a medical imaging research private sector already as well. Whilst I have higher expectations of them than academia, I would still hazard a guess that there is significantly more relevant brainpower in AI companies today.

[IDK, who are you @antiaudience]

My takeaway from this post is just that the nerds at Anthropic and other top labs, while they know a lot about how their tech works, they’re perhaps not actually very sophisticated reasoners about how the world works, so their predictions about the future are maybe not as good as one might think AI experts ought to be. The AI cultists still may be right about a lot of things, but it seems like people who understand the actual world better are the ones to read on this. People who understand power, laws, money, violence, politics, etc. Things that nerds are sometimes but not always very naive about.

[Richard Ngo @RichardMCNgo]

The AI safety community constructed a memeplex in which “taking AGI seriously” was a prerequisite for being a serious and good person. When inside this memeplex (as many at Anthropic, some at OpenAI, and a few at DeepMind are) your vision narrows until the world feels extremely constrained. The whole future seems to flow through the “one ring” of controlling recursive self-improvement. And so even when you worry about AI itself seizing that one ring, you can’t generate better strategies than trying to control it yourself (directly via an AGI company, or indirectly via AGI governance).

I’m not saying this is a pure hyperstition. There’s a core truth underlying this perspective: AI will become extremely intelligent and capable, much more than it is today. But the current world is much more spacious and human-empowering than the future which Eliezer originally envisioned (a “brain in a box in a basement” taking over the world by surprise). And it would be even more spacious if this memeplex weren’t active. For example, Satya and Mark and Sundar only started taking AGI seriously because OpenAI forced them to—and even now they don’t really believe in superintelligence—and even if they did they couldn’t get most of their employees on board. Imagine how chill a “race” between Microsoft and Meta and Google would have been, compared with what we have today: Dario and Sam deep in the “one ring” memeplex while also personally loathing each other.

So the one ring memeplex has an escalating life-cycle. It infects people by letting them harness the narrative that they’re good people for taking AGI seriously, and that making other people take AGI seriously is a boon for the world (despite how terribly that’s gone so far). Then it shuts off their imagination—any sparks of creativity or plans that don’t steer towards the one ring are quickly shut down. Instead they make ChatGPT or the METR graph or other recruiting tools for the memeplex. And yes, they’ll acknowledge that previous versions of the memeplex were too extreme, and led to overly constricted action. But we don’t have time to worry about that, they’ll say, because AGI is coming by 2027/2028, and that’s the end of history. Somehow, though, almost everyone with that view has only a vibes-based definition of AGI. They don’t believe in Dyson spheres by 2028, or self-replicating nanotech by 2028, or brain emulations by 2028. They mostly can’t make concrete predictions, except that it’ll be enough AI that it puts all their plans on a deadline. (Shout-out to @DKokotajlo and @paulfchristiano though, who do make concrete predictions about things going crazy soon.)

It seems very hard to break out of this memeplex without just giving up. David Holz is maybe the world champion of that—the only person who was in a position to race for AGI and consciously turned away. Various agent foundations researchers have carved out space to think real thoughts, not the kind of panicky stabbing in the dark that usually passes for safety research. A few others (e.g. Salamon, Hoffman, Vassar, Andre, Sahil, Davidad) are pursuing more unusual paths. And of the people who burned out, I expect some will reorient to doing creative thinking.

For others, the main takeaway: yes, the future of AI will be wild. But so far it’s increased peak human agency, and openness to this trend continuing over the next decade will allow you to start creating something worth creating.

[IDK, who are you @antiaudience]

This reminds me a lot of crypto nerd hype. Nerds making all these predictions based on the tech, while forgetting that the real constraints in the world aren’t just technical, they’re MEN WITH GUNS. Nerds have this problem all the time. A lot of them have a real men-with-guns blindspot because their whole focus is on the tech, what is technically possible, how to make it, etc. -- not what is politically possible.

[Zy @ZyMazza]

An abliterated top-tier open weights model today can tell you how to build a nuclear weapon. The challenge to building a nuclear weapon isn't knowledge. It's not even really resources.

Iran has knowledge and resources. They can't build one.

Why? A lot of other entities with knowledge and resources have a vested interest in making sure Iran doesn't have a nuclear weapon, and they get in their way.

If you wanted to build a nuclear weapon in your basement, to vibecode one with abliterated GLM 5.2, you would find that your efforts are stymied at every turn.

Even with step by step instructions, even if it was good enough at day-trading (it's not) to make you a tidy profit to finance your operations, you would find you would quickly attract the attention of law enforcement and the intelligence community.

It turns out they really don't want you to have a nuclear weapon!

If GLM-5.2 were good enough to tell you how to build a superplague (it's not), you would quickly learn a similar lesson in biology.

It turns out the challenge to building a superplague isnt really knowledge. Just as there are thousands of nuclear physicists with the theoretical knowledge to build nuclear weapons, there are thousands of biologists who have the theoretical knowledge required to make a superplague.

But if they wanted to do it, or if you wanted to, with your abliterated GLM-5.2-Bio-Edition, you would find that you would be stymied at every turn.

First you would design your virus particle or virus-like-particle. You could do this in the computer. But now you need to make it.

Traditionally you would order it from a company like Thermo-Fischer scientific or IDT. They'll synthesize a DNA strand for you and send it back so you can inject it into a cell of your choice to get it to produce the protien you want. For a fee, they'll even get an organism to express it for you and send *that* to you.

But, it turns out they're not stupid. And if you engineer a highly virulent strand of ebola and send it to them, before they help you make that, they're gonna say "hey wait a second, this kind of looks like a highly virulent strand of ebola!"

They will get upset, law enforcement will be contacted, the cops will show up, and so on and so on.

So maybe you'll do it yourself.

You have GLM 5.2-Bio-Edition and GLM 5.2-Trading-Edition so you make a bunch of money to order the expensive, highly specialized equipment you need to do this complex synthesis at home in your basement.

So you go online with your roughly $300k and start trying to order a Mermade DNA synthesizer and assorted sundries and suddenly they're asking all these strange questions like "what's your .edu email" or "what institution are you a part of?" and you don't really have answers to those questions because you're just some guy trying to make a superplague in your basement.

And actually it turns out the Government, intelligence community, and Bio Research technologies company *also* have GLM-5.2-Supply-Risk Edition and so they notice you trying to order all the parts to make a superplague in your basement.

They get very upset and the cops get called and so on and so on.

And *actually* it turned out *everyone* had access to GLM-5.2 Trading edition, so it wasn't so easy to make a killing on the market to finance your operation in the first place because your agent wasn't placing trades against human rubes, but against a whole hoard of *other* agents every bit as intelligent as itself, so the net result is that the market became slightly more efficient, but no one was really able to lock in asymmetrical gains.

Unless, however, *one* person had access to the GLM-5.2-ASI-Edition. Because if that person had access, they would be trading against rubes-by-comparison, and they would make a killing in the market to finance whatever operations they please.

And their ASI would be smarter than anything anyone else had, so it could engineer a VLP that thermo-fischer wouldn't be able to detect and reject on site. Or it would be able to deftly bypass supply chain safeguards, hacking into MIT or something (who has no ASI defender) to get you a .edu email and the appearance of institutional backing.

See, in a world where everyone has ASI, it's as though no one does. It's just normal society, where everyone's wishes and capabilities are mediated by group dynamics, everyone kind of has to stay within the overton window, and deviations are policed largely by the community, and then by the state when need be.

But in a world where only anthropic has ASI, or only openAI, or only the US government, then the *first* bad idea they have is immediately implemented with no resistance, with no hope of stopping it.

And on a long enough timeline, its only a matter of when, not if, they will have a bad idea.

Pluralism has served America well for a long time, and when it comes to ASI, I'm a pluralist too.

[IDK, who are you @antiaudience]

What I don't get about this is - doesn't anyone need to keep a close eye on what Claude is doing? Claude does enough dumb stuff for me that I wouldn’t want it executing on a meeting transcript (also lol at how bad all the meeting transcripts I've ever seen are and at the idea of something taking actions based on said transcripts). I don't work at such a company though, so I'm actually curious.

[taoki @justalexoki]

told a friend about how we use ai to transcribe, summarize, extract actions from meetings, then how claude executes on those actions etc and he looked like i was talking about some alien tech. he just started using chatgpt two months ago. two legitimately different worlds

[William Rose @dodecahedra]

On this view, the reason why success in math class has always been a gateway to higher career opportunities is that it's a proxy for one's error rate in performing thousands of small discrete tasks.

[William Rose @dodecahedra]

A lot of success in high school math isn't about conceptual understanding at all, it's about attention to detail. Do you care enough to try like a maniac to not make any mistakes, as if your life depended on it? That's often what it takes to do a medium-length problem correctly.

[Matt Bateman @mbateman]

100% true and damning. Particularly the Hillsdale and Straussian flavors of Great Books.

This is less true of the Adlerian flavor, but even something like the UChicago program misses that civilization is now hylomorphically fused with engineering, and so to that extent fails to teach civilization.

> there is also an implicit teological vision built into the current version of the canon that sees the whole affair of human reason, and in particular the enlightenment, as culminating in WWI, the Holocaust, and the incoherence of modernism, thus presenting a nihilistic vision of human potential and of the civilization to which we are heirs; as the canon is frozen in the 1930s it does not incorporate any of the compelling ideas that have transformed the world since then

[T. Greer @Scholars_Stage]

I’ll take a stab.

The concept of the Great Books was invented in the 1910s and 1920s and is not some civilizational tradition as often claimed; the implicit version of western civilization it teaches is a false one, giving the Greeks too much credit and erasing the Germanic contribution wholesale; there is also an implicit teological vision built into the current version of the canon that sees the whole affair of human reason, and in particular the enlightenment, as culminating in WWI, the Holocaust, and the incoherence of modernism, thus presenting a nihilistic vision of human potential and of the civilization to which we are heirs; as the canon is frozen in the 1930s it does not incorporate any of the compelling ideas that have transformed the world since then; philosophy is a poor way to understand the world and is less useful for this purpose than history, poetry, rhetoric, language and all of the other things punted out of the canon to stuff more philosophy in; there is no correlation between studying these books and living a virtuous life, or even an efficacious one.

[IDK, who are you @antiaudience]

Claude has become terrible. I'm telling Claude to install Codex for me. It's training its replacement.

[History With Jacob @HistoryWJacob]

The concept of "teenager" is a modern invention.

For most of human history, a boy of 13 was already a man, apprenticed in a trade or fighting in a war.

George Washington was a professional surveyor at 16.

Alexander Hamilton managed a trading company at 14.

In medieval Europe, noble boys could be pages at 7 and squires by 14.

In Rome, a boy put on the "toga of manhood" at 14.

The idea that an 18 year-old is "still figuring things out" would have been incomprehensible to our ancestors.

I believe this is why we think teenagers are so troubled. They are men and women stuck in a society that treats them as children. Of course they are going to "rebel".

We should give them more responsibility and expect much more of them.

[IDK, who are you @antiaudience]

@benlandautaylor I did.