At composability sync today, I explain what it takes to formalize a trivial inplace kernel w/ PTX semantics and weak memory model. Even for this simple case, it's still insanely complicated. Nothing in this video is novel, but maybe it's a helpful primer for someone!
We shrank 1 GB of Wikipedia to 105.39 MB, losslessly.
Introducing tufazip: the best entry on the Large Text Compression Benchmark using a neural network as its sole predictor. Everything above it mixes a transformer with hand-built models.
Full post 👇
https://t.co/cllPF90gsb
set up a linux x86_64 docker image to house dependencies with uv and run ml experiments locally on my older intel mac with codex.
i used to be bottlenecked by legacy device, but can now use updated libraries, e.g. pytorch, trl, transformers, llama.cpp, safetensors, etc.
Point72's founder Steve Cohen on why discipline is what sets people apart:
"The talent in this industry is exceptional. Everyone's really smart."
"The question is, can you be committed to things you want to accomplish? And that requires discipline."
"A baseball player, they might be a great hitter and a mediocre fielder. Can they become a great fielder?"
"If you're consistent in that approach, over time, you'll see improvement. And that's what it takes. And that's who you're competing with, people who are improving all the time."
"That requires a mindset of being disciplined, and not when you feel like it."
Calif has been dropping heater after heater. These nerds are a menace and need to be contained (renew their CrunchyRoll subscription) so they stop dropping exploits
London quant fund Quadrature made $8.7B in trading profits last year with just 246 employees
Quadrature was founded in 2010 by former D.E. Shaw traders Greg Skinner and Suneil Setiya. It runs mid-frequency stat arb, holding positions for hours to days.
It returned all outside capital in 2018 and now trades only for its founders.
It sits in a London quant group with QRT and G-Research that competes with Renaissance, D.E. Shaw and Two Sigma. Next, it plans to expand beyond equities into futures, options and currencies.
$8.7B across 246 people works out to about $35M of trading profit per employee.
i assume beforehand, just wondering what quality privileged context looks like, esp for tough to verify domains, esp at scale.
probs should just start with something simple like
math and get the patterns down
how do people do opsd hint generation at scale?
- are the hints generated before hand and within the training data?
- does a different model supervise at the same time?
reading for later
Self-Distilled Reasoner: On-Policy Self-Distillation
https://t.co/UErkXDgWBl
opd
https://t.co/YPjc0SJrki
A Brief Overview: On-Policy Self-Distillation In Large Language Models
https://t.co/P9FtljIoCA
Great lesson for a lot of investors and allocators.
Investing is NOT trading. You can have an amazing thesis and be incredibly smart, and lose staggering amounts of money if you are not skilled in the direct game of risk taking.
Sad story but just always a great reminder as we have seen this over and over again.