George3d6

Opinions are legally binding to all my employers, retweets are deep endorsements.

[George3d6 @George3d6]

2028 will be a very awkward year

[George3d6 @George3d6]

@c5rcorp I hope this is in the Oxford English dictionary under "fragile" and "myopic"

[George3d6 @George3d6]

Historical nazi fashion enthusiast are on the edge of their seats

[George3d6 @George3d6]

@captgouda24 Not to say that this isn't bad but it's not "30% chance of war" as much as "30% chance that railways in Poland get targeted by Russian drones and these drones are shut down and one of the sides scream that this was an act of aggression" (again, agree this is very bad)

[George3d6 @George3d6]

@captgouda24 This is a very non-specific market i.e. it would have resolved yes (arguably should have) due to the leipzig drone and it would also resolve as such if a military ship attempted sabotage.

A better question is: https://t.co/uhLpFZvmb9

[Sridhar Ramesh @RadishHarmers]

Grass feels ok. The way you guys talked it up, I thought it would be the most amazing touch sensation of my life.

[banteg @banteg]

>be me
>discover effective altruism
>apparently normal charity is inefficient
>why donate to random sad thing when spreadsheet can tell you optimal sad thing
>fair enough
>buy mosquito nets
>save lives
>numbers look good
>feel powerful

>couple years later
>someone asks an innocent question
>why only count people alive today
>huh
>future people matter too
>obviously
>my grandchildren shouldn't matter less just because they haven't spawned yet
>reasonable.jpg

>keep following logic
>what about their grandchildren
>also yes
>what about people in 500 years
>sure
>5000 years
>why not
>500 million years
>starting to get weird but morality is morality

>open calculator
>humanity could survive for an astronomically long time
>could colonize galaxy
>could have trillions upon trillions of descendants
>maybe digital people too
>maybe simulated civilizations
>maybe dyson spheres full of happy uploaded minds
>calculator starts smoking

>realize currently living humans are rounding error
>8 billion people suddenly looking extremely beta
>future contains potentially 10^something people
>can't even fit beneficiaries in google sheets

>new moral priority unlocked
>protect the long-term future

>stop thinking in units of "people helped"
>start thinking in "fraction of cosmic endowment preserved"

>malaria?
>terrible
>but only kills existing humans
>AI extinction could delete the entire light cone
>nuclear war could permanently derail civilization
>bad institutions could lock in terrible values for ten million years
>someone invents wrong constitution in 2140
>quadrillions suffer
>better fund governance workshop now

>friend says maybe we should improve hospitals
>explain opportunity cost
>friend says hospitals are full of actual sick people
>explain scope sensitivity
>friend stops inviting me to dinner

>need to decide what to fund
>easy
>expected value

>suppose project has one in a million chance of preventing extinction
>sounds tiny
>but extinction destroys 10^50 future lives
>multiply
>mother of god

>$10 million project has expected value of several galaxies
>charity evaluation complete

>someone asks where the one-in-a-million number came from
>expert judgement
>which expert
>us
>how calibrated
>extremely thoughtfully

>reduce estimate to one in ten million to be conservative
>still beats curing cancer by 38 orders of magnitude
>epistemic robustness achieved

>someone says maybe project doesn't work
>assign 20% chance
>still astronomical

>maybe project makes problem worse
>assign 5% chance
>still astronomical

>why 5
>because 30 felt pessimistic

>publish 46-page report
>contains seventeen sensitivity analyses
>every sensitivity analysis begins after assuming intervention has positive sign

>critic says you're multiplying enormous hypothetical stakes by extremely uncertain probabilities
>yes
>that's literally why it's important

>critic says the uncertainty might be structural rather than numerical
>make probability smaller
>critic says no, I mean maybe your model is wrong
>make probability smaller again
>critic begins rubbing temples

>discover AI safety
>perfect longtermist cause
>AI might kill everyone
>or create utopia
>or seize galaxy
>or tile universe with paperclips
>or create billions of conscious software minds
>finally a problem with numbers big enough for me

>start AI safety nonprofit
>mission: prevent dangerous AI
>hire smartest people available
>smartest people immediately start building better AI to understand dangerous AI
>interesting

>we must understand capabilities to understand safety
>we must scale models to study alignment
>we must race ahead so less responsible actors don't get there first
>we must deploy systems to learn how deployment can go wrong
>we must build the thing quickly because building the thing quickly is dangerous

>outsider asks why the people most worried about AI apocalypse all work at AI companies
>complicated field

>company releases stronger model
>very concerned
>company begins training even stronger model
>extremely concerned
>company raises $14 billion
>concern reaches unprecedented levels

>need to influence government
>future is at stake
>normal democratic process too slow
>politicians don't understand exponential curves
>public doesn't understand x-risk
>experts must guide them
>who counts as expert
>people who understand x-risk
>who understands x-risk
>our friends

>someone objects that this seems politically convenient
>explain we're representing future generations
>future generations unavailable for comment

>develop concept of value lock-in
>terrifying possibility that one ideology controls civilization forever
>therefore extremely important that civilization adopts correct values before lock-in
>whose values
>let's circle back

>begin with impartial morality
>end with small group of people deciding what quadrillions of hypothetical beings would want
>beautiful arc

>meanwhile actual humans keep doing annoying things
>voting wrong
>having parochial attachments
>loving family more than strangers
>caring about local community
>getting upset when told their suffering is cosmically negligible
>evolutionary biases everywhere

>explain that moral intuition cannot be trusted
>except intuition that future digital people count
>and intuition that extinction is uniquely bad
>and intuition that our probability estimates are sane
>and intuition that our institutional choices improve the future
>those intuitions survived peer review

>someone donates $5k to local homeless shelter
>inefficient
>could have funded 0.0000000000003% of an AI governance researcher
>think of all the simulated people you just killed

>okay maybe don't phrase it that way publicly

>PR team says "future generations deserve a voice"
>much better

>journalist asks what longtermism means
>say "future people matter"
>everyone agrees
>great
>journalist asks what follows from that
>well technically we should redirect enormous resources toward low-probability interventions affecting astronomical futures
>journalist raises eyebrow
>return to "future people matter"

>motte has entered the chat

>critic: of course future people matter
>me: glad we agree
>critic: I don't agree that your institute knows how to help them
>me: why do you hate our grandchildren

>eventually notice uncomfortable implication
>if future value dominates everything
>then helping people today mostly matters through effects on future
>education matters because future institutions
>health matters because future productivity
>democracy matters because future trajectory
>human beings slowly become instrumental variables in their own moral philosophy

>see starving child
>feel compassion
>check spreadsheet
>child's direct welfare contribution negligible
>but perhaps childhood nutrition improves national institutional quality
>compassion restored

>tell myself this is impartial altruism

>one day assistant asks obvious question
>"how do you know your intervention actually improves the far future?"

>silence

>open spreadsheet
>increase column width
>add confidence interval

>assistant asks again
>"no, I mean how do you know the sign is positive?"

>stare into cosmic light cone

>10^50 people staring back

>none of them exist
>none of them can tell me
>none of them can falsify my assumptions

>realize I have invented the perfect constituency

>infinitely important
>completely silent
>and always represented by me

[George3d6 @George3d6]

@tszzl This is just a fucking strawman though. Open AI had a vibecoded sandbox that was not meant to isolate an adversarial agent wherein "agent" here can mean a 12yo with kali linux.

[Philip Z. Maymin @pmaymin]

https://t.co/6YA3Nf29Vp

[Diogo Almeida @CompleteSkeptic]

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?

I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev

• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions

AFAICT the shortest path to AI-based economic revolution

[antic @antic]

@elder_plinius if there's one thing I've learned, always pass unexamined steganographic prompt injections from the internet directly into your agents for a good time.

[George3d6 @George3d6]

It's always been a few weird nerds reading books, forcing people to read was a communist ploy that never really worked.

I for one am glad we are returning to an age where we no longer have to pretend mass literacy is possible or desirable.

[anabology @anabology]

Hiring a Computational MRI Physicist to help us develop the first consumer MRIs:

https://t.co/JQRianz3Jt

[George3d6 @George3d6]

My wishlist for the AGI Santa brings me includes:
- Is able to copy things from a figma XML tree to a react XML tree
- Is able to follow basic junior eng heuristics when coding (e.g. not use float when decimals or ints are needed)
- Can control a browser at 0.1x human speed
- Produced one useful piece of math (e.g. a slightly better FFT library)
- Will not `rm -rf ~/` by accident (at around the same rate a junior engineer doesn't)
- Will plan in a way where 80% of instructions from a < 1k prompt are followed
- Can be left alone and connected to outside input stream with 100 USDC and come back to me with 101 USDC after a few dozen thousand USD of token spent
- In 1/10 cases where I give it a task that is not possible it concludes this and tells me
- In 9/10 cases where I give it a task that is very-possible it doesn't tell me that it is impossible

[George3d6 @George3d6]

There are two categories of mathematics I understand:
a) Applied, as in the blas libraries or FFT packages
b) When a very-smart person explain to normal people something fairly profound about how symbols model reality

We confuse (b) with (a) when decrying LLMs "solving math"

[Daniel O'Connor @Singularitarian]

Terence Tao’s strategy to resolve the Navier-Stokes problem remains the most brilliant, alien idea I’ve ever heard:

“In principle, such a von Neumann machine could also be built out of the laws of the inviscid form of the Navier-Stokes equations (i.e. the Euler equations). In physical terms, one would have to build the machine purely out of an ideal fluid (i.e. an inviscid incompressible fluid). If one could somehow create enough “logic gates” out of ideal fluid, one could presumably build a sort of “fluid computer”, at which point the task of building a von Neumann machine appears to reduce to a software engineering exercise rather than a PDE problem (providing that the gates are suitably stable with respect to perturbations, but (as with actual computers) this can presumably be done by converting the analog signals of fluid mechanics into a more error-resistant digital form). The key thing missing in this program (in both senses of the word) to establish blowup for Navier-Stokes is to construct the logic gates within the laws of ideal fluids. (Compare with the situation for cellular automata such as Conway’s “Game of Life“, in which Turing complete computers, universal constructors, and replicators have all been built within the laws of that game.)”

[from Tao’s blog, Feb. 2014 https://t.co/e6mZ9O1UBV ]

[George3d6 @George3d6]

@alexframegreen Anyone that remotely fits the definition of what might be a "scientist" wants the equivalent of:
a) Unlimited funding and freedom
b) Recognition from a set of (very niche) peers
c) A quiet comfortable life

Afaict the US basically offers that, and China certainly doesn't.

[George3d6 @George3d6]

@anabology You're so vain

[George3d6 @George3d6]

The virtual machine alone is about as cheap as equivalently one on AWS, GCS or Digital Ocean (19/16/13 + egress and backup fees).

I am pretty sure that for 99% of startups this agent-first approach is how they should go about infrastructure.

[Freysa @freysa_ai]

The https://t.co/4o5DBAiViR desktop app is live for macOS.

https://t.co/XpMCPY51Vt

[George3d6 @George3d6]

@alexwg 5000 more gigawatts and my boy here will even be able to have a 99.999% linear scale and center align his dotted lines

[George3d6 @George3d6]

@Noahpinion Theoretically a doomsday virus is possible under many models of reality, but so is igniting the atmosphere in a chain reaction and we ought have 0 reason to think a doomsday virus is more likely than any of the other 1001 scenarios.

Significantly less, in all fairness

[George3d6 @George3d6]

@Noahpinion Due to an accident of faith (based on it's size), you have had thousands of top biologists working on using the poster child virus for latency for gene delivery and...

Oh yeah, it literally always fucks up the latency/immune-evasion traits when you modify it even a tiny bit.