Hongwei Qin

Individual Researcher. Previously: Research Director @SenseTime_AI, PhD & B.E @Tsinghua_Uni

[Sam Altman @sama]

First, the good part of the Anthropic ads: they are funny, and I laughed.

But I wonder why Anthropic would go for something so clearly dishonest. Our most important principle for ads says that we won’t do exactly this; we would obviously never run ads in the way Anthropic depicts them. We are not stupid and we know our users would reject that.

I guess it’s on brand for Anthropic doublespeak to use a deceptive ad to critique theoretical deceptive ads that aren’t real, but a Super Bowl ad is not where I would expect it.

More importantly, we believe everyone deserves to use AI and are committed to free access, because we believe access creates agency. More Texans use ChatGPT for free than total people use Claude in the US, so we have a differently-shaped problem than they do. (If you want to pay for ChatGPT Plus or Pro, we don't show you ads.)

Anthropic serves an expensive product to rich people. We are glad they do that and we are doing that too, but we also feel strongly that we need to bring AI to billions of people who can’t pay for subscriptions.

Maybe even more importantly: Anthropic wants to control what people do with AI—they block companies they don't like from using their coding product (including us), they want to write the rules themselves for what people can and can't use AI for, and now they also want to tell other companies what their business models can be.

We are committed to broad, democratic decision making in addition to access. We are also committed to building the most resilient ecosystem for advanced AI. We care a great deal about safe, broadly beneficial AGI, and we know the only way to get there is to work with the world to prepare.

One authoritarian company won't get us there on their own, to say nothing of the other obvious risks. It is a dark path.

As for our Super Bowl ad: it’s about builders, and how anyone can now build anything.

We are enjoying watching so many people switch to Codex. There have now been 500,000 app downloads since launch on Monday, and we think builders are really going to love what’s coming in the next few weeks. I believe Codex is going to win.

We will continue to work hard to make even more intelligence available for lower and lower prices to our users.

This time belongs to the builders, not the people who want to control them.

[Hongwei Qin @HongweiQin]

time to switch to codex, andrej, it’s not that gpt-5 pro is amazing, cc is just too dumb. codex with gpt-5 high is the real answer

[Andrej Karpathy @karpathy]

I think congrats again to OpenAI for cooking with GPT-5 Pro. This is the third time I've struggled on something complex/gnarly for an hour on and off with CC, then 5 Pro goes off for 10 minutes and comes back with code that works out of the box. I had CC read the 5 Pro version and it wrote up 2 paragraphs admiring it (very wholesome). If you're not giving it your hardest problems you're probably missing out.

[Hongwei Qin @HongweiQin]

Totally vibe with this append-and-review note-taking style—it’s exactly what I’ve been doing! My main note (aka “todo note” or “idea note”) is similar: fresh thoughts always float at the top, marked clearly like “product idea:” or “research idea”.

# Three extra hacks I swear by:
1. Spin-off notes: If a quick idea starts ballooning, I just create a separate note and drop a link back in my main note. Keeps things tidy and manageable.
2. Date stamping: Each idea gets a quick timestamp at the end. Super helpful when revisiting ideas later, seeing how they’ve evolved over time.
3. Checkable tasks: If several items are part of a grouped task, I use checkboxes to tick them off when they’re done. Keeps progress super clear.

I also use Apple Notes—syncs super fast, is robust, and feels secure.

[Andrej Karpathy @karpathy]

Seeding my Bear ʕ•ᴥ•ʔ blog with more random posts, e.g. here's something I had on backlog for a while:

# The append-and-review note

An approach to note taking that I stumbled on and has worked for me quite well for many years. I find that it strikes a good balance of being super simple and easy to use but it also captures the majority of day-to-day note taking use cases.

Data structure. I maintain one single text note in the Apple Notes app just called "notes". Maintaining more than one note and managing and sorting them into folders and recursive substructures costs way too much cognitive bloat. A single note means CTRL+F is simple and trivial. Apple does a good job of optional offline editing, syncing between devices, and backup.

Append. Any time any idea or any todo or anything else comes to mind, I append it to the note on top, simply as text. Either when I'm on my computer when working, or my iPhone when on the go. I don't find that tagging these notes with any other structured metadata (dates, links, concepts, tags) is that useful and I don't do it by default. The only exception is that I use tags like "watch:", "listen:", or "read:", so they are easy to CTRL+F for when I'm looking for something to watch late at night, listen to during a run/walk, or read during a flight, etc.

Review. As things get added to the top, everything else starts to sink towards the bottom, almost as if under gravity. Every now and then, I fish through the notes by scrolling downwards and skimming. If I find anything that deserves to not leave my attention, I rescue it towards the top by simply copy pasting. Sometimes I merge, process, group or modify notes when they seem related. I delete a note only rarely. Notes that repeatedly don't deserve attention will naturally continue to sink. They are never lost, they just don't deserve the top of mind.

Example usage:

- Totally random idea springs to mind but I'm on the go and can't think about it, so I add it to the note, to get back around to later.
- Someone at a party mentions a movie I should watch.
- I see a glowing review of a book while doom scrolling through X.
- I sit down in the morning and write a small TODO list for what I'd like to achieve that day.
- I just need some writing surface for something I'm thinking about.
- I was going to post a tweet but I think it needs a bit more thought. Copy paste into notes to think through a bit more later.
- I find an interesting quote and I want to be reminded of it now and then.
- My future self should really think about this thing more.
- I'm reading a paper and I want to note some interesting numbers down.
- I'm working on something random and I just need a temporary surface to CTRL+C and CTRL+V a few things around.
- I keep forgetting that shell command that lists all Python files recursively so now I keep it in the note.
- I'm running a hyperparameter sweep of my neural network and I record the commands I ran and the eventual outcome of the experiment.
- I feel stressed that there are too many things on my mind and I worry that I'll lose them, so I just sit down and quickly dump them into a bullet point list.
- I realize while I'm re-ordering some of my notes that I've actually thought about the same thing a lot but from different perspectives. I process it a bit more, merge some of the notes into one. I feel additional insight.

When I note something down, I feel that I can immediately move on, wipe my working memory, and focus fully on something else at that time. I have confidence that I'll be able to revisit that idea later during review and process it when I have more time.

My note has grown quite giant over the last few years. It feels nice to scroll through some of the old things/thoughts that occupied me a long time ago. Sometimes ideas don't stand the repeated scrutiny of a review and they just sink deeper down. Sometimes I'm surprised that I've thought about something for so long. And sometimes an idea from a while ago is suddenly relevant in a new light.

One text note ftw.

[Hongwei Qin @HongweiQin]

I thought it was Reinforced Learning from Human Feedback, because born neuron weights are tuned 🤯

[Elon Musk @elonmusk]

Whoa, I just realized that raising a kid is basically 18 years of prompt engineering 🤯

[Hongwei Qin @HongweiQin]

Every researcher should try engineering optimization now and then, because its feedback loop is so short, the performance is always improving (most of the time), like a game and makes you so happy.

That, is quite the opposite for research. Research takes long time and most ideas don’t work.

Years ago I’ve spent some time with my team deploying state of the art in-house ISP pipeline to android devices. Good old days.

System optimizations are not so difficult to start for researchers and doing that gives you valuable insights into algorithms.

Give it a try, at least it gives you some confidence back 🫣

[Andrej Karpathy @karpathy]

Highly amusing update, ~18 hours later:

llm.c is now down to 26.2ms/iteration, exactly matching PyTorch (tf32 forward pass). We discovered a bug where we incorrectly called cuBLAS in fp32 mathmode 🤦‍♂️. And ademeure contributed a more optimized softmax kernel for very long rows (50,257 elements per row, in the last logits layer).

But the fun doesn’t stop because we still have a lot of tricks up the sleeve. Our attention kernel is naive attention, not flash attention, and materializes the (very large) preattention and postattention matrices of sizes (B, NH, T, T), also it makes unnecessary round-trips with yet-unfused GeLU non-linearities and permute/unpermute inside our attention. And we haven’t reached for more optimizations, e.g. CUDA Graphs, lossless compressible memory (?), etc.

So the updated chart looks bullish :D, and training LLMs faster than PyTorch with only ~2,000 lines of C code feels within reach. Backward pass let’s go.

[Hongwei Qin @HongweiQin]

Interesting, transformer architecture was illustrated with AI (Adobe Illustrator). So they knew it will last

[Aidan Gomez @aidangomez]

@jxmnop It was adobe illustrator

[Hongwei Qin @HongweiQin]

hitting in the face of works that distills from ChatGPT. This is an experimental proof of Schulman’s point that distilling brings hallucinations. ref: Schulman’s RLHF talk at Berkeley invited by Abbeel last month. p.s. so much low hanging fruits for the academy…

[Aran Komatsuzaki @arankomatsuzaki]

The False Promise of Imitating Proprietary LLMs

Open-sourced LLMs are adept at mimicking ChatGPT’s style but not its factuality. There exists a substantial capabilities gap, which requires better base LM.

https://t.co/GYORtrN14H

[Hongwei Qin @HongweiQin]

实时关注我并使用#Runtastic实时追踪功能来给我加油吧！ https://t.co/IP9K0llusE

[Hongwei Qin @HongweiQin]

实时关注我并使用#Runtastic实时追踪功能来给我加油吧！ https://t.co/6SIseN38UC

[Hongwei Qin @HongweiQin]

刚才使用#Runtastic PRO iPhone软件: https://t.co/uJgDd3ct1A在16m 34s内完成了3.02 公里的一项Runtastic跑步

[Hongwei Qin @HongweiQin]

刚才使用#Runtastic PRO Android软件: http://t.co/BMtY272tH6在1h 42m内完成了32.81 公里的一项一项Runtastic动感单车运动运动

[Hongwei Qin @HongweiQin]

累累累。。。

[Hongwei Qin @HongweiQin]

复习复习！也许是最后的考试吧

[Hongwei Qin @HongweiQin]

刚才使用#Runtastic PRO Android App: http://t.co/Pqv0w79IKa在15m 41s内完成了2.18 公里的一项Runtastic跑步运动运动

[Hongwei Qin @HongweiQin]

刚才使用#Runtastic PRO Android App: http://t.co/QvLKyb8bUw在23m 25s内完成了5.02 公里的一项Runtastic跑步运动运动

[Hongwei Qin @HongweiQin]

刚才使用#Runtastic PRO Android App: http://t.co/PlicU4bPI2在34m 12s内完成了5.8 公里的一项Runtastic跑步运动运动