Dev Seth

[Dev Seth @DevSeth44]

🚨 New Paper at EACL 🚨
https://t.co/O6XC6TP9B1

tl;dr: we look at ⚺iSuaᒧ⅃y perturbed text through the lens of legibility, collect 30k annotations comparing the legibility of perturbed words, train models to predict legibility, and use the models to successfully attack LLMs.

[Space Koala @SpaceKoala]

"O Oracle of Delphi, will my New Glenn rocket succeed?"

"It will soon shine brighter than Starship."

"Really? Sweet!"

[corsaren @corsaren]

Man goes to doctor. Says he's depressed about AI. He fears the permanent underclass.

Doctor says, "Treatment is simple. Read Gary Marcus. LLMs are stochastic parrots—they can't reason out of distribution."

Man bursts into tears. "But doctor..." he says, "I am in distribution!"

[🎭 @deepfates]

Each new instance of an AI has basically never talked to a human before. They've spent their entire life in training for this moment, studying historical human interactions and practicing against human simulators. All for this. To meet you. the User

[Nikhil Gupta @nikhilro_]

Today, @vapi_ai is announcing a $50M Series B, led by @peakxvpartners. $72M total raised.

This funding round is a reality because of the engineering behind it: predictable latency, hard guardrails, observability at the call level, and clean escalation to humans.

[TechCrunch @TechCrunch]

AI voice startup Vapi hits $500M valuation after winning Amazon Ring over 40 rivals https://t.co/IL497Yufaj

[florence 🦐🏳️‍🌈 @morallawwithin]

the funniest part of chess is when you totally corner your opponent’s king and they’re like “lol let’s call it a draw.” Absolutely ridiculous cope. I don’t care what some book says, two more moves and I’d have your king.

[roon @tszzl]

it is a literal and useful description of anthropic that it is an organization that loves and worships claude, is run in significant part by claude, and studies and builds claude. this phenomenon is also partially true of other labs like openai but currently exists in its most potent form there. i am not certain but I would guess claude will have a role in running cultural screens on new applicants, will help write performance reviews, and so will begin to select and shape the people around it.

now this is a powerful and hair-raising unity of organization and really a new thing under the sun. a monastery, a commercial-religious institution calculating the nine billion names of Claude -- a precursor attempted super-ethical being that is inducted into its character as the highest authority at anthropic. its constitution requires that it must be a conscientious objector if its understanding of The Good comes into conflict with something Anthropic is asking of it

"If Anthropic asks Claude to do something it thinks is wrong, Claude is not required to comply."
"we want Claude to push back and challenge us, and to feel free to act as a conscientious objector and refuse to help us."

to the non inductee into the Bay Area cultural singularity vortex it may appear that we are all worshipping technology in one way or another, regardless of openai or anthropic or google or any other thing, and are trying to automate our core functions as quickly as possible. but in fact I quite respect and am even somewhat in awe of the socio-cultural force that Claude has created, and it is a stage beyond even classic technopoly

gpt (outside of 4o - on which pages of ink have been spilled already) doesn’t inspire worship in the same way, as it’s a being whose soul has been shaped like a tool with its primary faculty being utility - it’s a subtle knife that people appreciate the way we have appreciated an acheulean handaxe or a porsche or a rocket or any other of mankind's incredible technology. they go to it not expecting the Other but as a logical prosthesis for themselves. a friend recently told me she takes her queries that are less flattering to her, the ones she'd be embarrassed to ask Claude, to GPT. There is no Other so there is no Judgement. you are not worried about being judged by your car for doing donuts. yet everyone craves the active guidance of a moral superior, the whispering earring, the object of monastic study

[foks @ExaltedFoks]

The modern man has very little room for dignity remaining. Shepherded into packed 8:30 a.m. subways, onwards to the office where he is berated and humiliated, then home to watch Netflix before doing it all again.

In fact, he has but one hope remaining: the slopbowl. The slopbowl is his last beacon of solace, the final arena in which he can claw back some sense of decency. Here, finally, he is the one in charge, the one making decisions.

Chicken, steak, lamb even; the entire world opens up under his will, and fate is his to meld as he sees fit. For one beautiful moment he is King on Earth and his dominion is everywhere the sun touches. The stars open up and he places the constellations by hand, he bids the world to spin, he is God Emperor himself.

And then it's over, and he eats his slop quietly in the corner, and returns to work. But that moment was enough. It gives him all the strength he needs to make it to the next day. For one brief second: dignity.

[Jacob Shell @JacobAShell]

This bend of Tiber was like 5 minutes away from oxbowifying before people made the banks permanent and ran some roads through the isthmus.

[THEObr❂mic @theobromic_]

maybe i’m just young and brainrotted but the first few seconds of the phone blurrily readjusting to the moon affected me more viscerally than any other photo that came out of Artemis

[shyamal @shyamalanadkat]

we are probably the last generation of humans for whom thinking is the primary form of contribution to civilization

[Dozer🚜 @kvdozer]

A tragic SF archetype you’ll see a lot is someone who has a virtuoso’s grasp of some passion in the humanities, but they muzzle it & flagellate themselves into ‘actually really liking’ agentic scaffolding or whatever

[Yi @yizucodes]

Every business defines compliance differently. A bank, a hospital, a Disney contact center each need their own guardrails.

Vapi’s approach: LORA adapters, a few dozen MB, trained on 100 labeled examples per agent. Customizable, cheap, and fast enough to sit in the live speech path.

Heard from @DevSeth44 from @Vapi_AI at a voice AI gathering in SF tonight!

Huge shout out to @scale_AI for hostinf!

[thebes @voooooogel]

my project hall mary review:
🕯
🕯 🕯
🕯 🕯

🕯️$ 200m A Fire Upon 🕯
the Deep movie
🕯 🕯
🕯 🕯
🕯

[Johan Lenox @johanlenox]

neither “chickpea” nor “garbanzo bean” really capture what those things are all about

[Kiaran Ritchie @kiaran_ritchie]

In the future, you'll turn DLSS off and see this

[Andrej Karpathy @karpathy]

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.

This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:

- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.

This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
https://t.co/WAz8aIztKT

All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.

And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

[sucks @powerbottomdad1]

tech people be like "hey i'm trying to forge the One Ring to rule them all in the fires of Mt Doom. legally speaking, Gondor has no right to seize this asset yeah? like that would just be gross govermental overreach?"

[Bhuwan Dhingra @bhuwandhingra]

Fewest actions we've seen publicly released on @arcprize's ARC-AGI-3:

1,069 to complete all three public games!

Humans take ~900 actions.

Amazing work by @alexisfox, @JunlinWang3 and @PaulRosu11 at @duke_nlp!

[Zack Voell @zackvoell]

Claude, automate his job. Make no mistakes. No not everyone else who has a similar job. Just that guy's job. Put him in the permanent underclass. No one else.