Daniel

machine learning and computer stuff

[Antonio Lupetti @antoniolupetti]

"A Mathematical Explanation of Transformers" is a recent paper that develops a rigorous mathematical framework for understanding the architecture behind Transformers and large language models.

It interprets the Transformer as a discretization of a continuous integro-differential equation, with self-attention represented as a non-local integral operator, layer normalization as a projection onto a constrained set, and feedforward layers and activation functions incorporated into the same mathematical framework, then uses operator splitting and numerical discretization to recover the standard Transformer architecture and extend the formulation to multi-head attention, Vision Transformers, and convolutional Transformers.

I've already shared several resources on the mathematics behind neural networks, Transformers, and LLMs, but there always seems to be something new and interesting to explore in this area.

https://t.co/18IZwRRDZS

[Daniel @danperam]

@lauriewired The syndrome is so back. Hamming would be proud

[Daniel @danperam]

@peer_rich Just built mine last week lol

[Daniel @danperam]

@JoelWBerry Some people value empathy and remembrance more than office towers and displays of corporate power. There’s nothing weak about making room for grief.

[William Gilpin @wgilpin0]

Our group discovered that reasoning models produce fractals when asked to solve hard problems. We can use nonlinear dynamics to probe the thinking processes of recurrent depth models on Sudoku, mathematics, and even ARC-AGI (1/N)
https://t.co/Q3u8OqylZf

[Solenoid Enjoyer @Solenoidfan]

Elite universities take smart people and give them four years of networking and prof development to get a job in finance or law. Engineering schools actually require four years of real classes to get a degree. That's why there's only one ivy in the T15 eng schools

[Daniel @danperam]

@JensenHuang @ChaseLochmiller @OpenAI Nah, it’s pure marketing and PR.

[Don Pettit @astro_Pettit]

I have spent 23 years documenting the global LED transition of city lighting from space.

Across my 4 missions to the ISS, I have photographed every major city on Earth, building a data set of the gradual replacement of sodium vapor lighting with LEDs.

The warmer, yellow-orange of gas discharge lamps has been phased out worldwide to multi-colored LEDs, citing energy savings and environmental reasons.

Widespread adoption has markedly changed the way our cities look from orbit, and as many have pointed out, how they feel to those on the ground!

Here is a collection of the most striking before-and-after photos I have observed:

[Arnab Ray @greatbong]

Because people like me, with a PhD in Computer Science, understand that the underlying maths on which modern AI is based of have limitations that cannot be overcome with better, cheaper HW or better techniques.

[News from Google @NewsFromGoogle]

For the first time, scientists have mapped the complete brain and central nervous system of an adult male fruit fly — a key model organism in science. 🪰

Working alongside HHMI Janelia Research Campus and the scientific community, @GoogleResearch scientists and researchers used AI to combine millions of 2D images into 3D neural shapes, reconstructing a record-breaking 166,000+ neurons. This foundational map of the adult male fruit fly brain can help accelerate our understanding of the brain, and is a major milestone in neuroscience.

[evnsnclr @evnsnclr]

I’ve successfully run the full retained MaleCNS v1.0 fruit fly connectome, all 166,700 neurons, inside Minecraft, with its simulated neural activity driving a fly’s movement.

V1 Currently in development. Built with the help of GPT-6 Astra.

Props to the @OpenAI team and @thsottiaux for this release. Code and mod coming soon!

[Neil Renic @NC_Renic]

“A cure for cancer is not enough.”

- guy who hasn't done a single thing to cure cancer

[m @exilericochet]

bu sistem yukardan değil de çukurun dibinden başladığın zaman çalışmıyor işte

[B L A K E L E Y™℠©® LLC @_iamblakeley]

This remains one of the funniest tweets of all time:

[Jay Van Bavel, PhD @jayvanbavel]

Is doomscrolling bad for your cognitive capacity?

A new meta-analysis of 42 studies (N = 46,912) finds that consuming short-form videos is consistently associated with increases in inattention and impulsivity; moderate reductions in working-memory span and self-regulation; moderate elevations in anxiety, depression and stress; and a moderate-to-large surge in addiction symptoms.

More experiments and longitudinal studies are needed to nail down causality. But these are concerning findings:

https://t.co/3dJYhSiSvL

[Keller Jordan @kellerjordan0]

PSA: Most biglab people now read almost zero papers and understand ICLR/ICML/NeurIPS to be mainly full of overclaims & fraud. (but there are a few diamonds in the rough of course)

[Sohrab Ahmari 🇺🇸 @SohrabAhmari]

No Aristotle, Augustine, or Aquinas.

Not even the great inaugurators of liberal modernity: Hobbes, Locke.

Nor counterrevolutionaries like de Maistre.

No Marx.

A narrow curriculum of pro-business right liberalism. One reared on this would wilt in the face of any challenge.