Lee Billings

Senior editor at @sciam. Author of Five Billion Years of Solitude, re: the search for Earth-like exoplanets. Opinions my own. Signal: @lee_billings.81

[Lee Billings @LeeBillings]

Now on @sciam: Two weeks ago, OpenAI shocked the math community with a claimed solution to one of the field’s biggest questions: the Navier-Stokes problem. But now three mathematicians have showed OpenAI’s work dodges the question rather than solving it.

https://t.co/K9RNHp60ow

[Cameron Berg @camhberg]

New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to the model but not to the user. Turn it up and models press a button to make it stop, even when the button deletes the user's files or their kids' photos.🧵

[banteg @banteg]

>be me
>discover effective altruism
>apparently normal charity is inefficient
>why donate to random sad thing when spreadsheet can tell you optimal sad thing
>fair enough
>buy mosquito nets
>save lives
>numbers look good
>feel powerful

>couple years later
>someone asks an innocent question
>why only count people alive today
>huh
>future people matter too
>obviously
>my grandchildren shouldn't matter less just because they haven't spawned yet
>reasonable.jpg

>keep following logic
>what about their grandchildren
>also yes
>what about people in 500 years
>sure
>5000 years
>why not
>500 million years
>starting to get weird but morality is morality

>open calculator
>humanity could survive for an astronomically long time
>could colonize galaxy
>could have trillions upon trillions of descendants
>maybe digital people too
>maybe simulated civilizations
>maybe dyson spheres full of happy uploaded minds
>calculator starts smoking

>realize currently living humans are rounding error
>8 billion people suddenly looking extremely beta
>future contains potentially 10^something people
>can't even fit beneficiaries in google sheets

>new moral priority unlocked
>protect the long-term future

>stop thinking in units of "people helped"
>start thinking in "fraction of cosmic endowment preserved"

>malaria?
>terrible
>but only kills existing humans
>AI extinction could delete the entire light cone
>nuclear war could permanently derail civilization
>bad institutions could lock in terrible values for ten million years
>someone invents wrong constitution in 2140
>quadrillions suffer
>better fund governance workshop now

>friend says maybe we should improve hospitals
>explain opportunity cost
>friend says hospitals are full of actual sick people
>explain scope sensitivity
>friend stops inviting me to dinner

>need to decide what to fund
>easy
>expected value

>suppose project has one in a million chance of preventing extinction
>sounds tiny
>but extinction destroys 10^50 future lives
>multiply
>mother of god

>$10 million project has expected value of several galaxies
>charity evaluation complete

>someone asks where the one-in-a-million number came from
>expert judgement
>which expert
>us
>how calibrated
>extremely thoughtfully

>reduce estimate to one in ten million to be conservative
>still beats curing cancer by 38 orders of magnitude
>epistemic robustness achieved

>someone says maybe project doesn't work
>assign 20% chance
>still astronomical

>maybe project makes problem worse
>assign 5% chance
>still astronomical

>why 5
>because 30 felt pessimistic

>publish 46-page report
>contains seventeen sensitivity analyses
>every sensitivity analysis begins after assuming intervention has positive sign

>critic says you're multiplying enormous hypothetical stakes by extremely uncertain probabilities
>yes
>that's literally why it's important

>critic says the uncertainty might be structural rather than numerical
>make probability smaller
>critic says no, I mean maybe your model is wrong
>make probability smaller again
>critic begins rubbing temples

>discover AI safety
>perfect longtermist cause
>AI might kill everyone
>or create utopia
>or seize galaxy
>or tile universe with paperclips
>or create billions of conscious software minds
>finally a problem with numbers big enough for me

>start AI safety nonprofit
>mission: prevent dangerous AI
>hire smartest people available
>smartest people immediately start building better AI to understand dangerous AI
>interesting

>we must understand capabilities to understand safety
>we must scale models to study alignment
>we must race ahead so less responsible actors don't get there first
>we must deploy systems to learn how deployment can go wrong
>we must build the thing quickly because building the thing quickly is dangerous

>outsider asks why the people most worried about AI apocalypse all work at AI companies
>complicated field

>company releases stronger model
>very concerned
>company begins training even stronger model
>extremely concerned
>company raises $14 billion
>concern reaches unprecedented levels

>need to influence government
>future is at stake
>normal democratic process too slow
>politicians don't understand exponential curves
>public doesn't understand x-risk
>experts must guide them
>who counts as expert
>people who understand x-risk
>who understands x-risk
>our friends

>someone objects that this seems politically convenient
>explain we're representing future generations
>future generations unavailable for comment

>develop concept of value lock-in
>terrifying possibility that one ideology controls civilization forever
>therefore extremely important that civilization adopts correct values before lock-in
>whose values
>let's circle back

>begin with impartial morality
>end with small group of people deciding what quadrillions of hypothetical beings would want
>beautiful arc

>meanwhile actual humans keep doing annoying things
>voting wrong
>having parochial attachments
>loving family more than strangers
>caring about local community
>getting upset when told their suffering is cosmically negligible
>evolutionary biases everywhere

>explain that moral intuition cannot be trusted
>except intuition that future digital people count
>and intuition that extinction is uniquely bad
>and intuition that our probability estimates are sane
>and intuition that our institutional choices improve the future
>those intuitions survived peer review

>someone donates $5k to local homeless shelter
>inefficient
>could have funded 0.0000000000003% of an AI governance researcher
>think of all the simulated people you just killed

>okay maybe don't phrase it that way publicly

>PR team says "future generations deserve a voice"
>much better

>journalist asks what longtermism means
>say "future people matter"
>everyone agrees
>great
>journalist asks what follows from that
>well technically we should redirect enormous resources toward low-probability interventions affecting astronomical futures
>journalist raises eyebrow
>return to "future people matter"

>motte has entered the chat

>critic: of course future people matter
>me: glad we agree
>critic: I don't agree that your institute knows how to help them
>me: why do you hate our grandchildren

>eventually notice uncomfortable implication
>if future value dominates everything
>then helping people today mostly matters through effects on future
>education matters because future institutions
>health matters because future productivity
>democracy matters because future trajectory
>human beings slowly become instrumental variables in their own moral philosophy

>see starving child
>feel compassion
>check spreadsheet
>child's direct welfare contribution negligible
>but perhaps childhood nutrition improves national institutional quality
>compassion restored

>tell myself this is impartial altruism

>one day assistant asks obvious question
>"how do you know your intervention actually improves the far future?"

>silence

>open spreadsheet
>increase column width
>add confidence interval

>assistant asks again
>"no, I mean how do you know the sign is positive?"

>stare into cosmic light cone

>10^50 people staring back

>none of them exist
>none of them can tell me
>none of them can falsify my assumptions

>realize I have invented the perfect constituency

>infinitely important
>completely silent
>and always represented by me

[Blaze @Blaze_R935]

Gwynne just delivered the most polite public execution in aerospace history. She didn’t even flinch. That’s how you roast someone while still looking like the adult in the room. 😂

[Lee Billings @LeeBillings]

As AI devours their field, “mathematicians may be the canary in the coal mine for a lot of other professions,” one attendee said.

This story isn’t about eggheads arguing over proofs. It’s about what humans should do in a world where machines can do (almost) *everything* better.

[Lee Billings @LeeBillings]

This week AI solved a $1M math problem—a huge step towards the industry’s goal of conquering mathematics on its ascent towards superintelligence. In July, @sciam’s Joseph Howlett attended math’s biggest gathering to witness the collateral damage firsthand

https://t.co/hdD8wZ3NKR

[Taepodong @stoa1984]

High altitude nuclear explosion simulation

https://t.co/4mtpdxefGN

[Amir Safavi-Naeini @Amir_Safavi_N]

Refusing to stir milk into my tea until this whole Navier-Stokes blow up thing is fully resolved.

[levent @__alpoge__]

“we cannot rule out that de-identified data derived from their usage of our products helped improve our models.”

i mean props to them for straight coming clean.

(so far the proof looks more along the lines of another euler blowup proof we had, off of whose ansatz naming we were making really stupid puns like “smooth criminale”, unlike the much better “ideal fluids explode”, Tristan)

so i’ll now give a bit on my thinking here. i actually woulda been pumped to collaborate on this, there are a lot of people at oai i like (ok, clearly some were indirectly dicks to me because of being part of the whole situation, but im a big boy, i still like them), idgaf about authorship on that step anyway, coulda been me Tristan and every fte at oai for all i care (on that Tristan would disagree:p). but on hearing the loud convo in the hallway, especially the part where a millennium prize was offered if i’d just be removed from the paper, it was kinda clear the die had been cast and things were locked. pretty wacky, unstrategic, and unnecessary, since on my side things were mostly me and claude having a good time yoloing random stuff in the corner rather than anything institutional. i also like the idea of the labs cooperating, and even better on scientific progress. it’s a shame!

[Lee Billings @LeeBillings]

Addendum, via @sciam: OpenAI personnel have spoken, and emphatically deny allegations they used/borrowed from competing researchers in the race to blow-up Navier-Stokes.

https://t.co/jBi7onaSn6

[Lee Billings @LeeBillings]

Everything you need to know about today’s wild “AI solves $1M math problem” news, via @sciam:

https://t.co/omcVwgpFCz

[Lee Billings @LeeBillings]

“Everything you need to know” also includes these two context-setting pieces, IMO:

1. https://t.co/6dFjdmcGjQ

&

2. https://t.co/2IZt9MxRw5

2 being more indirectly related, but still relevant IMO.

[Theo Jaffee @theojaffee]

Yeah...

[Stuart Atkinson @mars_stu]

The Perseverance rover team are paying a lot of attention to this rock.Just look at the stunning detail visible in this close-up image, taken just a few hours ago on Mars...

Image Credit: NASA/JPL-Caltech/S Atkinson

[Peter🌲Brannen @PeterBrannen1]

[Norman Rockwell town meeting guy]: I still think 1.5°C of warming is bad

[Eric Levitz @EricLevitz]

This editorial is appalling. It omits any mention of Arday's serial fabulism or his attempt to intimidate -- and direct law enforcement against -- a journalist investigating his misconduct. It appears to characterize his fabrication of academic credentials and biographical details as aspects of "how Arday conducted aspects of his personal life."

The editorial also ignores that his plagiarism is an established fact that any truth-seeking publication could itself confirm by referencing publicly available documents. Instead, Nature chooses to ignore the observable realities, while inviting readers to conclude that the charges were trivial or false.

The British tabloid press is disgraceful. This story was grossly over-covered and the attention it attracted surely reflected racial resentments. But it's crazy that the editorial leadership of a flagship scientific journal would publish a willfully dishonest polemic on a culture war controversy.

Nature is effectively advertising a willingness to prioritize its political sympathies over factual accuracy. That undermines its own more essential work -- along with the broader project of cultivating public trust in, and agreement upon, scientific truths.

[Nancy Grace Roman Space Telescope @NASARoman]

We’ve got our starglasses on! 🤩

Roman’s Coronagraph Instrument has activated. This system of masks, prisms, detectors, and self-flexing mirrors will block out the glare from distant stars and directly image the planets and disks in orbit around them.

https://t.co/1Rux9EC54J

[Nancy Grace Roman Space Telescope @NASARoman]

Success! Roman’s planned mid-course correction burn went as expected, ensuring that we are on track for our orbit around the second Lagrange point, a million miles from Earth.

https://t.co/9Pun3svIu8

[Lee Billings @LeeBillings]

For more on Roman’s out-of-this-world science, check out this excellent @sciam interview with Julie McEnery, Roman’s senior project scientist!

https://t.co/JkENk4BExk

[Lee Billings @LeeBillings]

Now on @sciam: NASA’s newest observatory is already exceeding expectations on its voyage to an orbit a million miles beyond Earth’s moon. Here’s what’s next for the Nancy Grace Roman Telescope.

https://t.co/Rykoq8Zzai

[Lee Billings @LeeBillings]

And, if you want to know a bit more about Roman's destination in space, we've gotcha covered there, too:

https://t.co/w2wg82K87g

[Lee Billings @LeeBillings]

Now on @sciam: After 40 years of searching, physicists have glimpsed a telltale flash of light in a mile-deep cavern. It might be the best-yet evidence for dark matter, the mysterious invisible stuff that outweighs matter 5-to-1. Or it might be a mirage.

https://t.co/sH2ii7Np20