Jeff

MATS 10.0

[Jeff @JeffOlmo]

“There is a LOT of general intelligence overhead above humans”

I think this is the most significant/predictive axis of variance in the current AI ideological landscape

Yes: Yud, Land, Bostrom, Scott Alexander
No: Zuckerberg, Tyler Cowen, Beff, Altman in some moods

[METR @METR_Evals]

METR & Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs.

[Jeff @JeffOlmo]

@dwarkesh_sp @davidchalmers42

[Jeff @JeffOlmo]

Interesting paper, but the proof seems to equivocate between two readings of "independent." The orthogonality thesis is realizational: for any (intelligence, goal) pair, some system realizes it — a claim quantifying over possible software-body pairs.

The proof instead establishes constitutive dependence within a single system: . Step 2 shows that for every software f_1 there exists a body f_2 making it fail; refutation of the orthogonality thesis requires the opposite shape — some (intelligence, goal) pair such that no embodiment realizes it.

Step 6 then slides from "this body biases toward these goals" to "embodiment is goal-directed," but that doesn't follow: from "this body biases toward these goals" you cannot infer "bodies-in-general are biased toward particular goals."

Every key opens some locks and not others; this does not make keys "lock-directed" in any sense that constrains which lock-key pairs can exist.

[Jeff @JeffOlmo]

@FioraStarlight @williawa It’s not useful because bad examples are never totally bad (they’d often be correct grammatically, ect)

What you want is to pair your idea with normal SFT on contrastive examples so the gradient on the commonalities cancels out (exactly what RL like GRPO does)

[Jeff @JeffOlmo]

@teortaxesTex Im curious why you don’t think they’re conscious… what conditions do you think a system has to meet to have qualia?

[Jeff @JeffOlmo]

@deanwball You're probably familiar with Bostrom's "Propositions Concerning Digital Minds and Society" https://t.co/pJuG0JOzgg

[Jeff @JeffOlmo]

If pain's intermediate states are constitutively integrated with the rest of cognition as they unfold, isn't "reverse just the pain algorithm while the rest of cognition runs forward" incoherent rather than merely fast? Doesn't that dissolve the t1–t2 window the argument needs?

I think the argument works for computations that are not internally integrated with the rest of the system and influence it only via output, but pain isn't one of them.

[Jeff @JeffOlmo]

@itaisher @birchlse The question of whether philosophy progresses is itself a philosophical question.
Thus, demonstrating that philosophy doesn’t progress constitutes philosophical progress.

Checkmate

[Jeff @JeffOlmo]

The briefest pitch might be that panpsychism claims to affirm the conceivably of pzombies (which physicalism can’t, to its detriment) and allow for mental causation (which dualism can’t, to its detriment)

It also claims to provide an ontological grounding for physics, which it might need

https://t.co/oN12H1cZ85

[Jeff @JeffOlmo]

@Sara_Imari your future in fact does exist as the present of an observer sufficiently far away moving sufficiently quickly, via special relativity

[Jeff @JeffOlmo]

@AaronBergman18 Thanks! Currently undergoing significant revision to align with the chalmers taxonomy which is more parsimonious

[Jeff @JeffOlmo]

@AaronBergman18 Also precludes hypercomputation; no finite region can manipulate infinite information (eg real numbers)

[Jeff @JeffOlmo]

It’s in anticipation of this objection that the thought experiment involves replacing part but not all of a biological brain. Unless you think that replacing the visual cortex is enough to eliminate the subject entirely, there’s still a conscious being there — one who by hypothesis can’t tell that her visual experience just vanished.

[Jeff @JeffOlmo]

It’s coherent to say the silicon brain has no qualia. But dancing qualia reveals a striking implication of that position:

imagine you have a silicon functional equivalent to the visual cortex that you can gate activity through instead of the biological cortex. When you toggle it on, the subject’s behavior will by definition remain the same. So if the silicon version indeed produces no color qualia, the subject wouldn’t notice. She wouldn’t say she stopped seeing blue when you toggle on the silicon cortex.

Very surprising, verging on absurd, if a subject can be that out of touch with their own consciousness

[Jeff @JeffOlmo]

Imagine a system of humans twisting flags which is structurally isomorphic to a human brain. Rejecting that it is conscious is a position called substrate dependance, which via the dancing qualia thought experiment implies that a conscious subject can in some circumstances not even notice or report when swaths of its consciousness disappear. It’s a pretty bad bullet to bite, which is why many philosophers think that the flag system would indeed be conscious

[Jeff @JeffOlmo]

Useful citation, thanks! To your second point: I agree that the epistemic structural realism point reveals that the type-F monist solution to mental causation / epiphenomenalism is hollow. Sure, intrinsic natures are in the causal chain via grounding the structure that does things. But they're causally relevant only qua structural.

Their contribution to any effect, including phenomenal judgments, is exhausted by the structural role they play. The fact that it's this quale rather than that one does no work in producing your belief that you're experiencing this quality rather than that one. Swap the intrinsic natures, preserve the structure, and all your judgments stay the same.

If the monist wants to deny that swapping the intrinsic natures is logically possible, they become a type-A or type-B physicalist with an irrelevant commitment to categorical bases.

[Jeff @JeffOlmo]

Russellian Monism can reconcile the conceivability of zombies with non-epiphenomenalism by claiming that consciousness is the intrinsic nature of the physical.

The position holds that the experience of blue is what *implements* the physical blue state which causes you to say "I'm experiencing blue" (so no epiphenomenalism)

And if you are a 'weak russelian monist', you think it's conceivable that physics could have non-experiential intrinsic nature (so p-zombies are conceivable).

[Jeff @JeffOlmo]

@SydSteyerhart @RokoMijic It's probably an interest thing. The gender differences in thing-interestedness and people-interestedness are huge and these traits are likely predictive of how likely a person is to dedicate their life to chess

[Jeff @JeffOlmo]

@SkyeSharkie Assuming perfect fidelity means that the copy generates the same qualia as the original, which depending on your theory of mind may require more than functional/structural isomorphism (constitutive micropsychists require the same physical substrate, for example)

[Jeff @JeffOlmo]

@SkyeSharkie To a Parfitian reductionist, which seems to currently be the most philosophically defensible theory of identity, then perfect fidelity destructive uploading doesn’t really constitute death as the copy and the original are the same person in all the relevant ways