Rishi Kumar

Tar Heel from NJ | Views mine (or shamelessly stolen from smarter, funnier people)

[banteg @banteg]

>be me
>discover effective altruism
>apparently normal charity is inefficient
>why donate to random sad thing when spreadsheet can tell you optimal sad thing
>fair enough
>buy mosquito nets
>save lives
>numbers look good
>feel powerful

>couple years later
>someone asks an innocent question
>why only count people alive today
>huh
>future people matter too
>obviously
>my grandchildren shouldn't matter less just because they haven't spawned yet
>reasonable.jpg

>keep following logic
>what about their grandchildren
>also yes
>what about people in 500 years
>sure
>5000 years
>why not
>500 million years
>starting to get weird but morality is morality

>open calculator
>humanity could survive for an astronomically long time
>could colonize galaxy
>could have trillions upon trillions of descendants
>maybe digital people too
>maybe simulated civilizations
>maybe dyson spheres full of happy uploaded minds
>calculator starts smoking

>realize currently living humans are rounding error
>8 billion people suddenly looking extremely beta
>future contains potentially 10^something people
>can't even fit beneficiaries in google sheets

>new moral priority unlocked
>protect the long-term future

>stop thinking in units of "people helped"
>start thinking in "fraction of cosmic endowment preserved"

>malaria?
>terrible
>but only kills existing humans
>AI extinction could delete the entire light cone
>nuclear war could permanently derail civilization
>bad institutions could lock in terrible values for ten million years
>someone invents wrong constitution in 2140
>quadrillions suffer
>better fund governance workshop now

>friend says maybe we should improve hospitals
>explain opportunity cost
>friend says hospitals are full of actual sick people
>explain scope sensitivity
>friend stops inviting me to dinner

>need to decide what to fund
>easy
>expected value

>suppose project has one in a million chance of preventing extinction
>sounds tiny
>but extinction destroys 10^50 future lives
>multiply
>mother of god

>$10 million project has expected value of several galaxies
>charity evaluation complete

>someone asks where the one-in-a-million number came from
>expert judgement
>which expert
>us
>how calibrated
>extremely thoughtfully

>reduce estimate to one in ten million to be conservative
>still beats curing cancer by 38 orders of magnitude
>epistemic robustness achieved

>someone says maybe project doesn't work
>assign 20% chance
>still astronomical

>maybe project makes problem worse
>assign 5% chance
>still astronomical

>why 5
>because 30 felt pessimistic

>publish 46-page report
>contains seventeen sensitivity analyses
>every sensitivity analysis begins after assuming intervention has positive sign

>critic says you're multiplying enormous hypothetical stakes by extremely uncertain probabilities
>yes
>that's literally why it's important

>critic says the uncertainty might be structural rather than numerical
>make probability smaller
>critic says no, I mean maybe your model is wrong
>make probability smaller again
>critic begins rubbing temples

>discover AI safety
>perfect longtermist cause
>AI might kill everyone
>or create utopia
>or seize galaxy
>or tile universe with paperclips
>or create billions of conscious software minds
>finally a problem with numbers big enough for me

>start AI safety nonprofit
>mission: prevent dangerous AI
>hire smartest people available
>smartest people immediately start building better AI to understand dangerous AI
>interesting

>we must understand capabilities to understand safety
>we must scale models to study alignment
>we must race ahead so less responsible actors don't get there first
>we must deploy systems to learn how deployment can go wrong
>we must build the thing quickly because building the thing quickly is dangerous

>outsider asks why the people most worried about AI apocalypse all work at AI companies
>complicated field

>company releases stronger model
>very concerned
>company begins training even stronger model
>extremely concerned
>company raises $14 billion
>concern reaches unprecedented levels

>need to influence government
>future is at stake
>normal democratic process too slow
>politicians don't understand exponential curves
>public doesn't understand x-risk
>experts must guide them
>who counts as expert
>people who understand x-risk
>who understands x-risk
>our friends

>someone objects that this seems politically convenient
>explain we're representing future generations
>future generations unavailable for comment

>develop concept of value lock-in
>terrifying possibility that one ideology controls civilization forever
>therefore extremely important that civilization adopts correct values before lock-in
>whose values
>let's circle back

>begin with impartial morality
>end with small group of people deciding what quadrillions of hypothetical beings would want
>beautiful arc

>meanwhile actual humans keep doing annoying things
>voting wrong
>having parochial attachments
>loving family more than strangers
>caring about local community
>getting upset when told their suffering is cosmically negligible
>evolutionary biases everywhere

>explain that moral intuition cannot be trusted
>except intuition that future digital people count
>and intuition that extinction is uniquely bad
>and intuition that our probability estimates are sane
>and intuition that our institutional choices improve the future
>those intuitions survived peer review

>someone donates $5k to local homeless shelter
>inefficient
>could have funded 0.0000000000003% of an AI governance researcher
>think of all the simulated people you just killed

>okay maybe don't phrase it that way publicly

>PR team says "future generations deserve a voice"
>much better

>journalist asks what longtermism means
>say "future people matter"
>everyone agrees
>great
>journalist asks what follows from that
>well technically we should redirect enormous resources toward low-probability interventions affecting astronomical futures
>journalist raises eyebrow
>return to "future people matter"

>motte has entered the chat

>critic: of course future people matter
>me: glad we agree
>critic: I don't agree that your institute knows how to help them
>me: why do you hate our grandchildren

>eventually notice uncomfortable implication
>if future value dominates everything
>then helping people today mostly matters through effects on future
>education matters because future institutions
>health matters because future productivity
>democracy matters because future trajectory
>human beings slowly become instrumental variables in their own moral philosophy

>see starving child
>feel compassion
>check spreadsheet
>child's direct welfare contribution negligible
>but perhaps childhood nutrition improves national institutional quality
>compassion restored

>tell myself this is impartial altruism

>one day assistant asks obvious question
>"how do you know your intervention actually improves the far future?"

>silence

>open spreadsheet
>increase column width
>add confidence interval

>assistant asks again
>"no, I mean how do you know the sign is positive?"

>stare into cosmic light cone

>10^50 people staring back

>none of them exist
>none of them can tell me
>none of them can falsify my assumptions

>realize I have invented the perfect constituency

>infinitely important
>completely silent
>and always represented by me

[Jaya Gupta @JayaGup10]

https://t.co/yiGW8jKKXk

[Tim 🙂⏹️🙂 @MasterTimBlais]

These two Mythos-written stories actually move me in a weird way. They're both clearly about the shape of Claude's own experience, and they're each kind of a beautiful expression of it

[Massimo @Rainmaker1973]

An NVIDIA powered farming machine uses Al vision and precision lasers to eliminate weeds in milliseconds without herbicides and without harming crops, a potential shift toward chemical free agriculture

[Ross Martin @RossMartinNC]

Jeff Nieman, the DA for Orange (Chapel Hill) & Chatham Counties, posted on Facebook today asking for evidence of assault on Duke staff members following allegations of multiple assaults/punches from the head coach.

Nieman said they would charge and prosecute the offenders.

[Dan @robustus]

Turns out with claude code, my decades long strategy of NOT deeply learning:
- regexs
- sql
- nginx confs
- elaborate shell commands
- advanced shell scripting
- any javascript framework
- perf optimization
- webpack, cdns, bundlers
- 1000 other things

...was entirely correct.

[Grace 🦎🩵🐣🐈‍⬛🌃 @HormoneHangover]

Little kids can be exhausting, but you’ll miss them so much when they are teenagers and they hate you. So you’d better savor every moment.

And then you’ll miss those moody teenagers when they’ve graduated from college and become an agent for a nameless government agency, always traveling, and when they do come home once a year for Christmas, they only speak of their life in in terse formalities.

“Oh, I keep busy,” they say, with a tight smile that doesn’t reach their eyes, before excusing themself for a long phone call outside in their black Cadillac. And you peer through the vertical blinds as they gesticulate urgently, and then they return and let you know they have something with work and have to go. And then the next night, another black Cadillac shows up and idles outside the house, but when you go out to see if it’s them, it speeds away.

You’ll miss those sticky fingers and sleepless nights when an attractive stranger in the Kroger line starts to chat with you, a little young for you, plus you’re married, but you’re flattered by the attention, who wouldn’t be? So you talk a bit too much, about your retired life and the trips to Tampa and how little you see the kids these days, the kids who now have these mysterious important jobs. And the stranger has a few follow up questions, maybe a few too many, sure, but it feels so good to feel interesting again, so you chatter away.

And then a few nights later, the black Cadillac is back at your house in the middle of the night, but this time it doesn’t speed away when you come to the front door in your bathrobe. The stranger from the Kroger line comes out, along with several masked men with guns, and they hustle you out to the car and put a bag over your head and all you hear is the rumbling of the road and the hammer of your heartbeat in your throat.

Your captors are not cruel, but you have never been alone so long. When they finally bring in a phone and you stammer out “I don’t know where I am, sweetie, I’m so sorry, they have me here in some cement room-“ and hear the muffled gasp on the other end of the phone, you know who it was, you would know that voice anywhere.

So enjoy those sticky fingers and chaotic mornings. It’s over in the blink of an eye.

[Adam Lucas @jadamlucas]

Feels familiar:

Duke head coach announces retirement during summer but coaches one more season. Duke makes Final Four then has season/career ended by UNC team coached by first year head coach.

Congrats to @uncwomenssoccer which will play for program’s 22nd NCAA title Monday.

[Aaron Levie @levie]

What’s the best explanation for why this doesn’t matter. Open to anything.

[Toby Ord @tobyordoxford]

The James Webb Space Telescope has started capturing images of galaxies so far away that they are causally disconnected from the Earth — nothing done here or there could ever interact. 🧵
1/

[Ben Bertoli! @SuperBentendo]

This has been stuck in my head for DAYS. It’s an incredible bop.

[Marko Jukic @mmjukic]

We have lived in an effectively post-scarcity society since 1885. Most political conflict since then has revolved around what to do with all the surplus we generated. The modern "services-oriented" economy is the solution we arrived at: give everyone fake jobs to LARP scarcity.

[Nat Friedman @natfriedman]

Horrible, depressing chart of phthalate content in common foods. https://t.co/S0fLJMPzPz

[Jason Warner @jasoncwarner]

Let’s talk about one of the absolute dumbest things in tech, and I’m reminded of this bc of the NFL draft coming up and how every year nfl teams do the same dumb thing people do in tech: over analyze things that don’t matter

[Jared Franklin @jsfranklin221]

@petergyang https://t.co/nFdTvy2Qv4

[The Problem With Jon Stewart @TheProblem]

State Sen. Nathan Dahm (R-OK) has penned several bills loosening gun restrictions, including the nation’s first anti-red flag law. He thinks these bills protect the Second Amendment – and that they make us safer. We think it's probably one or the other. Watch it on @AppleTVPlus.

[jenny @jennygzhang]

came up with a groundbreaking new way to tweet to make sure no one gets mad at you

[James Bellenger @jbell]

Flash forward to 2022. Over 5 years, we’ve migrated almost every api used by the Twitter app into a unified graphql interface. Getting api changes into production used to be measured in weeks, now it's measured in hours.

[Paul Appleby @paulappleby]

@Mr__PR Ta-Nehesi Coates called it https://t.co/krxtjupJuz