Joshua Rozner

PhD candidate @stanford
pondering the limits of statistical methods; linguist by curiosity
tech is (one of) the problem(s)

[Joshua Rozner @jsrozner]

@mboudry @simonfriederich Models have latent capabilities that are easily activated by finetuning. To use the biology analogy, a relatively minor mutation may produce substantially different behavior

https://t.co/kAgr8fK2DF
https://t.co/EokRJAbYw4

[Joshua Rozner @jsrozner]

@GaryMarcus I thought the The Algebraic Mind argued the human brain* is neurosymbolic. When humans solve problems, even symbolic manipulations still live in our brains

LLMs calling to external tools might approximate "neurosymbolic," but it would seem to be a different neurosymbolic?

[Joshua Rozner @jsrozner]

@Dr_Gingerballs Recent math results w LLMs suggest that path-finding in a latent space can be effective, much like alphago found good moves

Re "stealing" - one can understand this as "having ingested every past math proof (maps from other folks), models can now connect prev unconnected regions"

[Joshua Rozner @jsrozner]

@zeynep Certainly feels that way reviewing for a major NLP conference. Many more submissions (many likely AI-assisted), but they look like real papers. Reviewers also using AI to produce useless reviews (or too pressed for time to carefully review, or unqualified to review)

[Joshua Rozner @jsrozner]

@Dr_Gingerballs "We thought we had solved NP-hardness, but it turns out that we were actually just scaling compute time exponentially to search the solution space"

[Joshua Rozner @jsrozner]

@Dr_Gingerballs I agree. But we don't know how much success is coming from brute force (searching more of the exponential space of graph paths) versus the representation space improving by putting analogous structures closer together (and thus reducing the paths that need to be considered)

[Joshua Rozner @jsrozner]

@GaryMarcus The objective in Go is very clear: win the game. The objective for math proofs less so. Yes, you want to prove the conjecture, but knowing which conjectures or research ideas are worth pursuing is harder.

Relatedly, even SOTA Models are far from being good conference reviewers

[Joshua Rozner @jsrozner]

@GaryMarcus @shoshanazuboff literally wrote a book about Google, the Age of Surveillance Capitalism, in 2019. Under reasonable definitions, Google's business was always surveillance.

The economics have changed, and now they are pivoting to new forms of surveillance.

[Joshua Rozner @jsrozner]

@heynavtoor Okay, but next time could you please write your tweet yourself? As a human, I still prefer human writing.

[Joshua Rozner @jsrozner]

@RickKiessig @sama Can I read about it somewhere?

[Joshua Rozner @jsrozner]

@RickKiessig @sama The business model of most tech companies is building moats. Funnily, they're now all eager to build bridges over their moats with "MCP" (which is really just API).

Users might finally be able to do useful things with their own data, if the world can survive long enough.

[Joshua Rozner @jsrozner]

@RickKiessig @sama Did anything come of this work?

[Joshua Rozner @jsrozner]

@coryshain Our results converge with recent work suggesting that LMs acquire morphological, lexical, and syntactic structure from distributional input alone. Perturbation provides a minimal assumption, causal method for studying how and when this happens.
https://t.co/BeGy2IZkLB

[Joshua Rozner @jsrozner]

New paper with @coryshain: We develop "perturbation" - a simple approach to finding shared representations (common abstractions) in LMs. We perturb an LM by fine-tuning it on a single adversarial example and measure how this perturbation “infects” other examples.

[Joshua Rozner @jsrozner]

@coryshain Perturbation can be done using small naturalistic datasets. This suggests that experimenters can track the emergence of particular abstractions (linguistic or otherwise) over the course of training with relatively little compute and few examples.