Ray

15. building @SchemaML. @westlake_uni ai research intern @ 13.

[Midjourney @midjourney]

A technical dive inside our new "Midjourney Scanner"

[Ray @raygyli]

@itsCathyDi @WindsorNguyen funmaxxing

[Ray @raygyli]

@benkimbuilds @cursorcommunity dudr it was nice seeing you!

[Ray @raygyli]

had tons of fun at cafe cursor sf

[Ray @raygyli]

@arseniitrutnev @cursorcommunity fr

[Ray @raygyli]

of course, i was talking about raw intelligence in various domains; language models obviously have one advantage over expert humans - not needing any breaks, which means, effectively, they're only better than humans at the perceived quality of intelligence.

[Ray @raygyli]

tl;dr: beyond-expert human-level intelligence might not come from scaling language models alone, but from systems that combine symbolic compression with world model-like kant's "anticipation" idea.

---
llms as "besonnenheit" - "the ability to turn perceived information into symbols that encode meaning" implies language models are some form of compression of their "world" (ie the entire training corpus).

this implies that the act of scaling llms on their own *could* potentially lead to the "machine at human ability across all tasks" definition, and current models do already excel in many domains. however, since llms still rely on complete "learning" from their training sets, it might be hard for models to go beyond the skill/intelligence level of top talent in any domain (e.g. current models are sometimes able to prove old math problems but cannot come up with new ones). RL might help to break the bottleneck, but at least with current methods it's a whack-a-mole situation.

since world models have some form of intrinsic instinct to "expect" and what kant argues is what extends humans from our animalistic behavior, and especially if it extends beyond just high-level human life scenarios but also into abstract concepts like math&science, that is perhaps how world models could have potential to scale beyond human intelligence.

unless we combine those aspects of language modeling and world models into one, since at least for now, world models don't have a way to convey semantic information. people have tried similar ideas, e.g., LCMs (large concept models), but not much exploration seems to have been done.

---

example: i asked gpt image to generate a "very strange picture with a completely novel art style" (joining the trend of generating strange and creepy images)

the key thing is the visual structure of the image, which the language model plan, resembles existing classic 20th-century surrealism.

in contrast, the details and background that the final diffusion model gives a novel-ish feel, which we sometimes call "slop" – diffusion models lean kantian.

[Elena @VirtualElena]

extremely european voice: the llm vs world model debate is encapsulated & foreshadowed by the canonical texts of 18th century german aesthetic philosophy, specifically herder’s “treatise on the origin of language” and kant’s “conjectures on the beginning of human history.” both philosophers interrogated (and attempted to locate philosophically/temporally) the awakening of human reasoning/how we differ from animals. herder locates this quality in language and more specifically “besonnenheit” or the capacity to mark and stabilize experience in signs. meanwhile kant argues that the ability to visual and model future states (aka “conscious expectation of the future” itself the 3rd step in a 4 step process in which humans evolve out of animal instinct into reason/freedom…for those curious the 4 steps are 1) food 2) sex/modesty 3) future/anticipation 4) moral dominion and freedom) is THE thing that makes us human, the ability to have imagined futures. where you fall in the agi debate can largely be predicted by whose philosophy you align more with…herder’s or kant’s…

[Ray @raygyli]

guys any coffee shop recommendations in sf san francisco

[seth @sethsetse]

every-time someone posts a “just move to SF” or a “pay the California taxes” my rent increases by 7%

[signüll @signulll]

i feel 100x safer in a waymo than i do with a human driver.

[Ray @raygyli]

tl;dr: beyond-expert human-level intelligence might not come from scaling language models alone, but from systems that combine symbolic compression with world model-like kant's "anticipation" idea.

---
llms as "besonnenheit" - "the ability to turn perceived information into symbols that encode meaning" implies language models are some form of compression of their "world" (ie the entire training corpus).

this implies that the act of scaling llms on their own *could* potentially lead to the "machine at human ability across all tasks" definition, and current models do already excel in many domains. however, since llms still rely on complete "learning" from their training sets, it might be hard for models to go beyond the skill/intelligence level of top talent in any domain (e.g. current models are sometimes able to prove old math problems but cannot come up with new ones). RL might help to break the bottleneck, but at least with current methods it's a whack-a-mole situation.

since world models have some form of intrinsic instinct to "expect" and what kant argues is what extends humans from our animalistic behavior, and especially if it extends beyond just high-level human life scenarios but also into abstract concepts like math&science, that is perhaps how world models could have potential to scale beyond human intelligence.

unless we combine those aspects of language modeling and world models into one, since at least for now, world models don't have a way to convey semantic information. people have tried similar ideas, e.g., LCMs (large concept models), but not much exploration seems to have been done.

---

example: i asked gpt image to generate a "very strange picture with a completely novel art style" (joining the trend of generating strange and creepy images)

the key thing is the visual structure of the image, which the language model plan, resembles existing classic 20th-century surrealism.

in contrast, the details and background that the final diffusion model gives a novel-ish feel, which we sometimes call "slop" – diffusion models lean kantian.

[Ray @raygyli]

@VirtualElena https://t.co/hG04emW8fA

[Ray @raygyli]

tl;dr: beyond-expert human-level intelligence might not come from scaling language models alone, but from systems that combine symbolic compression with world model-like kant's "anticipation" idea.

---
llms as "besonnenheit" - "the ability to turn perceived information into symbols that encode meaning" implies language models are some form of compression of their "world" (ie the entire training corpus).

this implies that the act of scaling llms on their own *could* potentially lead to the "machine at human ability across all tasks" definition, and current models do already excel in many domains. however, since llms still rely on complete "learning" from their training sets, it might be hard for models to go beyond the skill/intelligence level of top talent in any domain (e.g. current models are sometimes able to prove old math problems but cannot come up with new ones). RL might help to break the bottleneck, but at least with current methods it's a whack-a-mole situation.

since world models have some form of intrinsic instinct to "expect" and what kant argues is what extends humans from our animalistic behavior, and especially if it extends beyond just high-level human life scenarios but also into abstract concepts like math&science, that is perhaps how world models could have potential to scale beyond human intelligence.

unless we combine those aspects of language modeling and world models into one, since at least for now, world models don't have a way to convey semantic information. people have tried similar ideas, e.g., LCMs (large concept models), but not much exploration seems to have been done.

---

example: i asked gpt image to generate a "very strange picture with a completely novel art style" (joining the trend of generating strange and creepy images)

the key thing is the visual structure of the image, which the language model plan, resembles existing classic 20th-century surrealism.

in contrast, the details and background that the final diffusion model gives a novel-ish feel, which we sometimes call "slop" – diffusion models lean kantian.

[Ray @raygyli]

tho beyond (individual) human intelligence, language models might still potentially work via ginormous self-improving agent clusters that form a full sicial system that encourages innovation (this is based on the idea that human civilization is a form of distributed superintelligence). however, this theoretical approach might be far more compute-inefficient and extremely slow than a single model that's beyond human intelligence across domains.

[Ray @raygyli]

need to get back to doing advanced math

[Ray @raygyli]

saw this book on a table in a hotel lobby btw

only in sf

[pri @macrodosi]

🥹

[Elena @VirtualElena]

extremely european voice: the llm vs world model debate is encapsulated & foreshadowed by the canonical texts of 18th century german aesthetic philosophy, specifically herder’s “treatise on the origin of language” and kant’s “conjectures on the beginning of human history.” both philosophers interrogated (and attempted to locate philosophically/temporally) the awakening of human reasoning/how we differ from animals. herder locates this quality in language and more specifically “besonnenheit” or the capacity to mark and stabilize experience in signs. meanwhile kant argues that the ability to visual and model future states (aka “conscious expectation of the future” itself the 3rd step in a 4 step process in which humans evolve out of animal instinct into reason/freedom…for those curious the 4 steps are 1) food 2) sex/modesty 3) future/anticipation 4) moral dominion and freedom) is THE thing that makes us human, the ability to have imagined futures. where you fall in the agi debate can largely be predicted by whose philosophy you align more with…herder’s or kant’s…

[Ray @raygyli]

all things considered, still an extremely breathtaking beautiful city (pics dont do justice)

[Ray @raygyli]

comically enough resorted to linkedin for housing search

[Ray @raygyli]

comically enough resorted to linkedin for housing search

[Ray @raygyli]

just landed in sf for the first time 3 days ago and can confirm this is true.

we booked a hotel for the first week and planned to spend this time looking for more stable housing for a 3-month stay (should've booked a longer stay in hindsight).

most of the properties are 4-6.5k+, and the ones that look like your typical sustainable home are 6k-10k+.

haven't gotten a single reply so far; phone calls redirected to voicemail.

[Ray @raygyli]

just landed in sf for the first time 3 days ago and can confirm this is true.

we booked a hotel for the first week and planned to spend this time looking for more stable housing for a 3-month stay (should've booked a longer stay in hindsight).

most of the properties are 4-6.5k+, and the ones that look like your typical sustainable home are 6k-10k+.

haven't gotten a single reply so far; phone calls redirected to voicemail.

[Ankit Gupta @agupta]

sadly for SF this might be true.

I just recommended to a founder that “15k for a 2br seems within normal these days”. He’s funded by YC so he can live with it but not a good sign for the ecosystem

[Ray @raygyli]

@EveCarruthers_ thanks :))

[Ray @raygyli]

hello, san francisco