Dezhe Jin

Computational neuroscientist and physicist at Penn State, works on neural mechanism of birdsong, spike-based neural computation, and data analysis.

[Dezhe Jin @djintwt]

Published a new paper!

Many brain elements are highly sensitive to temperature changes, but birdsong tempo is resistant to such fluctuations.

We show this is due to the use of axonal delays, and the prolonged opening of synaptic channels.

https://t.co/uO6D22V4Be

[Dezhe Jin @djintwt]

Is there a such thing as banning a journal? Or at least a ranking system that will point out garbage journals.

[Mauro Lago-Docampo @MuruLago]

In January, I reviewed a Review for a crappy MDPI journal.

It was 100% AI, missing citations, etc. I sent a massive report, and the authors withdrew it.

Today, I see it published without a single change in another even crappier MDPI journal.

[Eric Betzig @Eric_Betzig]

Reductionism has been a powerful force for understanding fundamental aspects of complex systems, but its very success has blinded most biologists to its crippling inability to make sense of complex emergent dynamic systems. We have the tools now to noninvasively observe and perturb living cells in their stunning holistic complexity, but the tyranny of reductionism has reserved the most important tool of all, AI, to approaches (e.g., structural biology, genomics, and spatial transcriptomics) ill-suited to their practitioners stated goal of building "virtual cells". It's been very frustrating convincing the gatekeepers of large HPC clusters otherwise.

[Shir R Maimon @ShirRMaimon]

Excited to share that our study is now published in Nature!

We found a sparse-to-dense coding transformation in the hippocampus that enables faster learning.
Big thanks to the entire team, especially @tamir_eliav @yahdef, Liora Las and Nachum Ulanovsky
https://t.co/bFt3KZzL19

[Ali H. Shaib @AliHShaib]

Hello world, meet 1,000× Expansion Microscopy.
1,000,000,000× expansion by volume! A gel that starts at a few centimeters will then expand to the volume of an Olympic swimming pool. https://t.co/E43kxx4O5M
In our new bioRxiv preprint, work carried out between MIT and UMG, led by Helena Hu in collaboration with scientists from the labs of @eboyden3 Ed Boyden, Silvio Rizzoli, and myself, we present Thousandfold Expansion Microscopy.
By enlarging biological specimens across multiple rounds of expansion, molecular-scale features, as small as the distances between adjacent amino acids, can be visualized with conventional optical microscopes.
Democratizing super-resolution microscopy.

[DINQ @dinq_me]

Xi Yin just left Harvard for OpenAI.

String theorist. Youngest full professor in Harvard history.

His words: "AI gives me 100x speedup. Weeks of output would take me 10 years."

Then: "I don't believe there's any human intellectual ability AI cannot replicate."

The man who said that is the one who would know.

#DINQ #AI #OpenAI

[Jérôme Lecoq @LecoqJerome]

Folks, we are doing those 3 neuronal recordings with the same visual prediction tasks for the OpenScope community project. The data is public for everyone to analyze.
Perhaps, you might be interested in the data... https://t.co/IfCPLiFKVT

[MONTREAL.AI @Montreal_AI]

Language models may not need to “build” hierarchies.

Hierarchies may fall out of the statistics of language.

A beautiful new paper by Andres Nava and Matthieu Wyart proposes a distributional theory for one of the most basic structures in meaning:

the “is-a” relation.

An owl is a bird.
A bird is an animal.
An animal is an organism.

This relation — hypernymy — looks like an ontology.

But the paper asks a sharper question:

Does hierarchical concept geometry in language models require a hierarchy-specific mechanism?

Or can it emerge from word co-occurrence alone?

Their answer is striking.

Start with a simple empirical fact:

words closer together in the WordNet hierarchy tend to co-occur more often.

“tree” and “plant” appear together more than “tree” and “organism.”

That decay in co-occurrence with semantic distance induces structure in the embedding Gram matrix.

Then the spectrum does the rest.

The leading eigenvectors first separate broad branches of the taxonomy, then progressively finer sub-branches.

This creates what the authors call hierarchical splitting geometry:

coarse-to-fine organization in representation space.

In the organism example, one principal direction separates plants from animals. Later directions split flowers from trees, birds from fish, and eventually finer distinctions like daisy vs. poppy.

That is the elegant part:

the geometry looks conceptual,
but the mechanism is spectral.

The authors prove this under mild positivity and decay assumptions on the co-occurrence kernel, confirm it across sampled WordNet subtrees in word2vec, and then show the same signature extends surprisingly well to Gemma 2B unembeddings.

This is not saying LLMs do not represent hierarchies.

They clearly do.

It is saying we should be careful about why that geometry exists.

Some elegant semantic structure may not be evidence of a specialized internal ontology.

It may be the mathematical shadow of pairwise word statistics.

That matters for interpretability.

If we find clean concept directions, orthogonal refinements, or taxonomic splits inside models, we should ask:

Is this a functional mechanism?

Or is it the spectrum of the data distribution made visible?

This paper pushes toward a more precise science of representation geometry.

Less mysticism.
More mechanism.

Less “the model learned an ontology.”
More “the co-occurrence kernel shaped an eigenspace.”

Full credit to the authors:
Andres Nava and Matthieu Wyart.

Paper:
Hierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence
https://t.co/Le1EHAVqJP

I’m attaching the first page because Figure 1 is worth studying closely.

The deep lesson:

meaning may become geometry not because the model was taught a taxonomy,

but because language itself already contains one in its statistics.

#AIResearch #Interpretability #LLM #NLP #RepresentationLearning #MachineLearning

[Steven Pinker @sapinker]

60 Percent of Grades at Harvard Were A’s. Enough Is Enough, say my colleagues David Laibson and @jasonfurman (and I couldn't agree more). https://t.co/RCXpFIPnT0

[Dezhe Jin @djintwt]

This is great!

Go even further. Expand the reviews to arXiv papers. Use the published reviews and comments to rate the original paper. Then publish rankings of the papers in subfields. Ratings need to sniff out the gaming and fraud.

[Colin Wright @SwipeWright]

ANNOUNCEMENT: WE’RE SAVING SCIENCE!

We’re often told that science is “self-correcting.”

But that’s not really true.

Science doesn’t correct itself like a thermostat adjusting the temperature in your house. Science is a human institution run by human beings. And human beings are vulnerable to career incentives, groupthink, moral fads, political pressure, and fear.

And when those forces capture academic journals, peer review stops being a filter for bad ideas and starts becoming more of a credentialing system for fashionable nonsense.

This isn’t exactly new.

In 1996, the physicist Alan Sokal managed to publish a totally gibberish article in the journal Social Text full of trendy postmodern jargon. His point was simple: if you flatter the ideological commitments of certain academic editors, nonsense can pass as real scholarship.

Two decades later, @ConceptualJames, @HPluckrose , and @peterboghossian pulled off the “grievance studies” hoax, placing over a half dozen absurd papers in peer-reviewed journals. One paper used dog parks to analyze rape culture and queer performativity. Another rewrote parts of Mein Kampf in the language of feminist theory.

The problem wasn’t just that fake papers got published. It was that they were completely indistinguishable from the real thing.

And today, the problem is even worse.

We now have serious SCIENCE journals publishing papers about feminist lesbians marrying brine shrimp. We have disturbing papers that aim to “queer” and sexualize infants. We have scholarship on “lesbian-queer-trans-canine relationalities” and “trans-dog intimacies.”

But while Clown World papers are concerning because it makes a complete mockery of academia, the same broken, ideologically captured system is also publishing research in legitimate science and medical journals that pushes sex and gender pseudoscience, relies on deeply flawed data, and influences policies on the medical transition of children and young adults.

That’s not funny. That affects real people. It affects medicine. It affects law. It affects children.

And when critics try to respond, they often discover there’s no serious mechanism for correction. Submitted Letters to the Editor often go completely ignored. Contrary evidence is rejected without comment. As a result, the best critiques are often relegated to personal blog posts, social media threads, or newspaper op-eds, while the original paper remains in the literature wearing the armor of “peer review.”

That is untenable.

So Kevin McCaffree, editor-in-chief of Theory and Society (@Theory_Society), and I decided to do something about it.

Today, in the Wall Street Journal, we announced a first-of-its-kind article type called “Peer Review.”

The idea is simple: publication should be the beginning of academic scrutiny, not the end of it.

A Peer Review article can critique a paper from any scholarly journal. It can address problems with methods, evidence, logic, definitions, theory, or interpretation. But it has to focus on the claims and arguments, not personal attacks.

Submissions are capped at 2,500 words and go through a straightforward merit review instead of endless gatekeeping and ideological screening. We ask just one basic question: Is this critique coherent, serious, reasonable, or even popular enough to deserve scholarly attention?

If yes, it gets published.

And the authors of the original paper get a built-in right of reply, so readers can see the critique and the response in a legitimate academic venue.

That’s how science is supposed to work.

Science becomes self-correcting only when real people build the mechanisms that allow correction to happen.
That’s what we’ve done.

Now it’s time for academics to use it.

Read our announcement on the @WSJ below.
🔗https://t.co/gqkDE7aaDC

[Dezhe Jin @djintwt]

Grades are useless these days. In person tests are the only honest indicators left for evaluations. I would advocate getting GREs back for graduate admissions as well.

[Michael 英泉 Eisen @mbeisen]

I fully support the reinstatement of standardized testing requirments for UC admissions. While perhaps well-intentioned, the removal of the SAT/ACT from the admissions process has clearly led to the lowering of standards in high schools across the state.

However, I didn't sign the "reinstate the SAT" letter circulated by Berkeley math profs that was released today. While it made point I agree with, its focus on STEM and its emphasis solely on math are misplaced and counterproductive.

While basic math skills are obviously a prerequisite for STEM fields, basic math reasoning is essential in a growing number of non-STEM fields, and we are letting down all UC students if we don't work to ensure they ALL have basic math competence.

More importantly, as bad as a decay in math skills is for STEM, the acute decay in reading and writing competence among STEM majors is arguably more of a threat to the future of STEM fields and society.

There is no reason to compartmentalize our expectations for UC admission in the way this letter suggests. What UC and our students need is for us to bring back standardized testing of math **and** verbal skills for **all** fields of study.

[Dezhe Jin @djintwt]

Just realized:

how annoying Taylor Swift songs are

if you try to work in Starbucks and her songs are on nonstop.

Please @Starbucks stop this nonsense playlist.

[Echo echo @EchoEcho693]

@PhilosophyOfPhy Landau also ranked physicists on logarithmic scale.
0 - Newton
0.5 - Einstein
1 - Bohr, Bose, Heisenberg, Dirac, Schrödinger, and Wigner
2 - Landau himself.

[Kenneth D Harris @kennethd_harris]

AI is about to transform how scientific research happens, and neuroscience is no exception. What the future looks like is up to us. I don’t have the answers; here are some thoughts from working on the IBL AI agent. Aiming to start a discussion.

[Dezhe Jin @djintwt]

@kennethd_harris This is awesome approach! Like that everyone is like a PI. But can a person be a PI without the ability of doing all detailed works by himself at least couple of times? We may still need to train researchers to know all glory details.

[Valerio Capraro @ValerioCapraro]

Finally, a big name has the courage to tell it: we are nowhere near AGI.

Demis Hassabis, CEO of Google DeepMind and Nobel laureate for AlphaFold, put it neat and clear:

"Today's systems are nowhere near [AGI]. Doesn't matter how many Erdős problems you solve… I think it's far, far from what a true invention, or someone like Ramanujan, would have been able to do."

This is the elephant in the room that many AI enthusiasts prefer not to see, or are actively trying to hide.

Erdős problems are well defined, often combinatorial, on finite spaces. They are exactly the kind of problems on which current AI can achieve spectacular performance with a lot of compute and knowledge.

A neural network can search a huge graph of possibilities. It can recombine existing knowledge at unprecedented scale. It can discover surprising solutions inside an already defined conceptual space.

But true invention is something else.

True invention is not only solving a problem.

It is inventing new objects, new dimensions, new connections. It is inventing new problems.

From resolving to inventing there is a discontinuity that we don't know how to bridge.

We are making extraordinary tools.
But we are nowhere close to AGI.

[Clash Report @clashreport]

Francis Fukuyama on the U.S. as a declining power:

American decline is a direct product of Trump's rise since 2016.

It is as if Trump had decided to do everything in his power to weaken the United States vis-à-vis China.

He has polarized an already polarized country, cut funding for basic scientific research, and attacked American universities which are the best in the world.

He and his colleagues have openly stated that their domestic opponents — the Democrats — are a far greater threat to the future of the United States than either China or Russia.

There is agreement among America's friends and rivals that the United States has become something of a rogue state that is contributing to global instability and disorder — as well as something of a laughingstock.

[Dr Kishore Paknikar @kpaknikar]

A short reply to this Nature editorial: Of course, science needs humans.

No AI has yet become obsessed with an idea for 20 years, argued endlessly over tea, ignored conventional wisdom, made glorious mistakes, and accidentally changed the world.

[Eric Betzig @Eric_Betzig]

I have the the luxury that I don't have to care for career reasons where my work is published, but I will do everything I can to convince my less fortunate co-authors to boycott the Nature Publishing Group and choose journals that make rapid publication a priority, or just post to bioRxiv and its kin. I also beseech hiring and tenure committees and grant reviewers to drop the lazy bullshit and start judging one's work based on the content of its "character" rather than the color of its "skin".

[Nick Davidov @Nick_Davidov]

The biggest bullshit move by DHS in its history. So everyone on a O1 or H1B visa would have to stop working legally in the US, go back to their country and wait for years of backlog? This includes top scientists in our universities, founders of billion dollar companies (at least 3 just in our portfolio would be affected by the way). And if we look at individual countries it becomes even more bs. Indians would have to wait decades. Russians don’t have anywhere to go (there is no US embassy in Russia, hello?).

This is the worst imaginable way to disrupt important work for the country and pretend you’re fighting some loophole.

[Dezhe Jin @djintwt]

Watch how our cat solves the problem…he is definitely intelligent.

Is current AI up to this task when thrown into this situation.