pongtsu

[ali @waterloo_intern]

https://t.co/aivc7B4A7Z

[Thinking Machines @thinkymachines]

People talk, listen, watch, think, and collaborate at the same time, in real time. We've designed an AI that works with people the same way.

We share our approach, early results, and a quick look at our model in action.

https://t.co/AFJZ5kH7Ku

[Qwen @Alibaba_Qwen]

⚡ Meet Qwen3.6-35B-A3B：Now Open-Source！🚀🚀

A sparse MoE model, 35B total params, 3B active. Apache 2.0 license.

🔥 Agentic coding on par with models 10x its active size
📷 Strong multimodal perception and reasoning ability
🧠 Multimodal thinking + non-thinking modes

Efficient. Powerful. Versatile. Try it now👇

Blog：https://t.co/EXx5y466su
Qwen Studio：https://t.co/bg4tAU1p74
HuggingFace：https://t.co/w4pDX14DZS
ModelScope：https://t.co/SuRyLzdQiO
API（‘Qwen3.6-Flash’ on Model Studio）：Coming soon～ Stay tuned

[Muhammad Ayan @socialwithaayan]

🚨 BREAKING: Someone just built the exact tool Andrej Karpathy said someone should build.

48 hours after Karpathy posted his LLM Knowledge Bases workflow, this showed up on GitHub.

It's called Graphify. One command. Any folder. Full knowledge graph.

Point it at any folder. Run /graphify inside Claude Code. Walk away.

Here is what comes out the other side:

-> A navigable knowledge graph of everything in that folder
-> An Obsidian vault with backlinked articles
-> A wiki that starts at index. md and maps every concept cluster
-> Plain English Q&A over your entire codebase or research folder

You can ask it things like:

"What calls this function?"
"What connects these two concepts?"
"What are the most important nodes in this project?"

No vector database. No setup. No config files.

The token efficiency number is what got me:

71.5x fewer tokens per query compared to reading raw files.

That is not a small improvement. That is a completely different paradigm for how AI agents reason over large codebases.

What it supports:

-> Code in 13 programming languages
-> PDFs
-> Images via Claude Vision
-> Markdown files

Install in one line:

pip install graphify && graphify install

Then type /graphify in Claude Code and point it at anything.

Karpathy asked. Someone delivered in 48 hours.

That is the pace of 2026.

Open Source. Free.

[Andrej Karpathy @karpathy]

LLM Knowledge Bases

Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating knowledge (stored as markdown and images). The latest LLMs are quite good at it. So:

Data ingest:
I index source documents (articles, papers, repos, datasets, images, etc.) into a raw/ directory, then I use an LLM to incrementally "compile" a wiki, which is just a collection of .md files in a directory structure. The wiki includes summaries of all the data in raw/, backlinks, and then it categorizes data into concepts, writes articles for them, and links them all. To convert web articles into .md files I like to use the Obsidian Web Clipper extension, and then I also use a hotkey to download all the related images to local so that my LLM can easily reference them.

IDE:
I use Obsidian as the IDE "frontend" where I can view the raw data, the the compiled wiki, and the derived visualizations. Important to note that the LLM writes and maintains all of the data of the wiki, I rarely touch it directly. I've played with a few Obsidian plugins to render and view data in other ways (e.g. Marp for slides).

Q&A:
Where things get interesting is that once your wiki is big enough (e.g. mine on some recent research is ~100 articles and ~400K words), you can ask your LLM agent all kinds of complex questions against the wiki, and it will go off, research the answers, etc. I thought I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries of all the documents and it reads all the important related data fairly easily at this ~small scale.

Output:
Instead of getting answers in text/terminal, I like to have it render markdown files for me, or slide shows (Marp format), or matplotlib images, all of which I then view again in Obsidian. You can imagine many other visual output formats depending on the query. Often, I end up "filing" the outputs back into the wiki to enhance it for further queries. So my own explorations and queries always "add up" in the knowledge base.

Linting:
I've run some LLM "health checks" over the wiki to e.g. find inconsistent data, impute missing data (with web searchers), find interesting connections for new article candidates, etc., to incrementally clean up the wiki and enhance its overall data integrity. The LLMs are quite good at suggesting further questions to ask and look into.

Extra tools:
I find myself developing additional tools to process the data, e.g. I vibe coded a small and naive search engine over the wiki, which I both use directly (in a web ui), but more often I want to hand it off to an LLM via CLI as a tool for larger queries.

Further explorations:
As the repo grows, the natural desire is to also think about synthetic data generation + finetuning to have your LLM "know" the data in its weights instead of just context windows.

TLDR: raw data from a given number of sources is collected, then compiled by an LLM into a .md wiki, then operated on by various CLIs by the LLM to do Q&A and to incrementally enhance the wiki, and all of it viewable in Obsidian. You rarely ever write or edit the wiki manually, it's the domain of the LLM. I think there is room here for an incredible new product instead of a hacky collection of scripts.

[Ole Lehmann @itsolelehmann]

i can't believe nobody caught this.

Anthropic's entire growth marketing team was just ONE PERSON

(for 10 months, confirmed)

a single non-technical person ran paid search, paid social, app stores, email marketing, and SEO for the $380B company behind claude

here's exactly how one human is doing the job of a full marketing team:

it starts with a CSV.

1. he exports all his existing ads from his ad platforms along with their performance metrics (click-through rates, conversions, spend, etc)

2. feeds the whole file into claude code

3. and tells it to find what's underperforming.

claude analyzes the data, flags the weak ads, and generates new copy variations on the spot

this is where he gets clever:

he then splits the work into 2 specialized sub-agents:

1. one that only writes headlines (capped at 30 characters)

2. and one that only writes descriptions (capped at 90 characters).

each agent is tuned to its specific constraint so the quality is way higher than cramming both into a single prompt

so now he's got hundreds of fresh headlines and descriptions.

but that's just the text.

he still needs the actual visual ad creative, the images and banners that go on facebook, google, etc.

so he built a figma plugin that:

1. takes all those new headlines and descriptions
2. finds the ad templates in his figma files
3. and automatically swaps the copy into each one.

up to 100 ready-to-publish ad variations generated at half a second per batch.

what used to take hours of duplicating frames and copy-pasting text by hand

so now the ads are live.

the next question is which ones are actually working.

for that he built an MCP server (basically a custom integration that lets claude talk directly to external tools) connected to the meta ads API.

so he can ask claude things like:

• "which ads had the best conversion rate this week"
• or "where am i wasting spend"

and get real answers from live campaign data without ever opening the meta ads dashboard

and the part that ties it all together and closes the loop:

he set up a memory system that logs every hypothesis and experiment result across ad iterations.

so when he goes back to step one and generates the next batch of variations...

claude automatically pulls in what worked and what didn't from all previous rounds.

the system literally gets smarter every cycle.

that kind of systematic experimentation across hundreds of ads would normally need a dedicated analytics person just to track

the numbers from the doc:

ad creation went from 2 hours to 15 minutes. 10x more creative output.

and he's now testing more variations across more channels than most full marketing teams

a $380 billion company.

and their entire growth marketing operation (not GTM) = just one person and claude code lol

truly unbelievable

[Unsloth AI @UnslothAI]

We created a repo with 250+ notebooks for LLM training.

Train locally on your device with 3GB VRAM or free on Colab.

Learn the entire fine-tuning and inference workflow.

Supports RL, vision, audio, embedding, TTS models

GitHub: https://t.co/1qSdX8w8SO

[Andrej Karpathy @karpathy]

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.

This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:

- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.

This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
https://t.co/WAz8aIztKT

All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.

And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

[Thien Tran @gaunernst]

Wrote a blogpost. Hopefully it's the first of many to come. Feedback is welcome 🤗
https://t.co/3lJAvGil4f

[Andrej Karpathy @karpathy]

Love this project: nanoGPT -> recursive self-improvement benchmark. Good old nanoGPT keeps on giving and surprising :)

- First I wrote it as a small little repo to teach people the basics of training GPTs.
- Then it became a target and baseline for my port to direct C/CUDA re-implementation in llm.c.
- Then that was modded (by @kellerjordan0 et al.) into a (small-scale) LLM research harness. People iteratively optimized the training so that e.g. reproducing GPT-2 (124M) performance takes not 45 min (original) but now only 3 min!
- Now the idea is to use this process of optimizing the code as a benchmark for LLM coding agents. If humans can speed up LLM training from 45 to 3 minutes, how well do LLM Agents do, under different kinds of settings (e.g. with or without hints etc.)? (spoiler: in this paper, as a baseline and right now not that well, even with strong hints).

The idea of recursive self-improvement has of course been around for a long time. My usual rant on it is that it's not going to be this thing that didn't exist and then suddenly exists. Recursive self-improvement has already begun a long time ago and is under-way today in a smooth, incremental way. First, even basic software tools (e.g. coding IDEs) fall into the category because they speed up programmers in building the N+1 version. Any of our existing software infrastructure that speeds up development (google search, git, ...) qualifies. And then if you insist on AI as a special and distinct, most programmers now already routinely use LLM code completion or code diffs in their own programming workflows, collaborating in increasingly larger chunks of functionality and experimentation. This amount of collaboration will continue to grow.

It's worth also pointing out that nanoGPT is a super simple, tiny educational codebase (~750 lines of code) and for only the pretraining stage of building LLMs. Production-grade code bases are *significantly* (100-1000X?) bigger and more complex. But for the current level of AI capability, it is imo an excellent, interesting, tractable benchmark that I look forward to following.

[Rohan Paul @rohanpaul_ai]

LLMs struggle with precise, interpretable, and domain-aligned reasoning in finance.

FinCoT addresses this by embedding expert financial workflows as Mermaid diagrams into structured Chain-of-Thought prompts, greatly improving accuracy and interpretability without model fine-tuning.

Methods 🔧:

- FinCoT uses a system message to frame the financial task.

- It employs Guided Step-by-Step Execution with thinking and output tags for intermediate reasoning and the final answer.

- An Expert Reasoning Blueprint, a domain-specific Mermaid diagram, serves as a prompt hint, outlining recommended step-by-step problem-solving.

- A Re-Reading and Semi Self-Reflection step includes a brief self-check within the output to ensure consistency.

📌 FinCoT provides substantial accuracy gains; Qwen3-8B-Base improved 17.3 percentage points.

📌 The approach significantly reduces inference cost by decreasing generated tokens eight-fold.

📌 FinCoT delivers highly interpretable, expert-aligned reasoning traces for financial tasks.

----------------------------

Paper - arxiv. org/abs/2506.16123

Paper Title: "FinCoT: Grounding Chain-of-Thought in Expert Financial Reasoning"

[ℏεsam @Hesamation]

Top 10 highly technical YouTube channels to learn AI from scratch:

1\ Andrej Karpathy - a blend of general and technical content, Zero to Hero playlist is a must-watch https://t.co/tNNLTyYT86 @karpathy

2\ Umar Jamil - highly technical, implements ML and LLM techniques from scratch https://t.co/AP9ecr04wq @hkproj

3\ Simon Oz - technical low-level machine learning videos https://t.co/7StZUevNXp

4\ Tunadorable - paper review, implementation, triton https://t.co/vwOhYtui07

5\ GPU Mode - technical interviews and walkthroughs about anything related to GPUs https://t.co/gfRuxCeBEn

6\ AI Jason - AI experiments, software design, and new techniques beautifully explained https://t.co/7O7DRRxSQa @jasonzhou1993

7\ Ferdinand Mom - everything related to distributed training & inference https://t.co/gmV7vBQixg @FerdinandMom

8\ Welch Labs - unique in-depth look at machine learning complexities like nobody else https://t.co/ovEt9WnEx2 @welchlabs

9\ Artem Kirsanov - neuroscience and machine learning from a different look, great visuals https://t.co/sJBmDbP7Qy @ArtemKRSV

10\ David Ondrej - new models, building apps with AI, practical for developers https://t.co/BEOr0MgHag @DavidOndrej1

This list is for a practical and technical audience. It's so hard to hand-pick just 10 channels, there are so many great ones out there making great content.

If you know any other channels, LMK.

[Alex Reibman 🖇️ @AlexReibman]

OpenAI just launched their new AI Agents SDK

But before the reveal, we threw an exclusive hack night for San Francisco’s top AI developers from 8pm to 1am. OpenAI just reclaimed the throne.

Here here’s what we saw at Hack Night with @OpenAI and @AgentOpsAI (🧵):

[Benjamin F Spector @bfspector]

(1/7) Inspired by DeepSeek's FlashMLA, we're releasing ThunderMLA—a fused megakernel optimized for variable-prompt decoding! ⚡️🐱ThunderMLA is up to 35% faster than FlashMLA and just 400 LoC.

Blog: https://t.co/1t7N6WIDoZ
With @AaryanSinghal4, @realDanFu, and @hazyresearch!

[Nouamane Tazi @Nouamanetazi]

🚀 Excited to release *THE* Ultra-Scale Playbook - a comprehensive guide on training LLMs from 1 to 1000s of GPUs!

[Andrej Karpathy @karpathy]

I was given early access to Grok 3 earlier today, making me I think one of the first few who could run a quick vibe check.

Thinking
✅ First, Grok 3 clearly has an around state of the art thinking model ("Think" button) and did great out of the box on my Settler's of Catan question:

"Create a board game webpage showing a hex grid, just like in the game Settlers of Catan. Each hex grid is numbered from 1..N, where N is the total number of hex tiles. Make it generic, so one can change the number of "rings" using a slider. For example in Catan the radius is 3 hexes. Single html page please."

Few models get this right reliably. The top OpenAI thinking models (e.g. o1-pro, at $200/month) get it too, but all of DeepSeek-R1, Gemini 2.0 Flash Thinking, and Claude do not.

❌ It did not solve my "Emoji mystery" question where I give a smiling face with an attached message hidden inside Unicode variation selectors, even when I give a strong hint on how to decode it in the form of Rust code. The most progress I've seen is from DeepSeek-R1 which once partially decoded the message.

❓ It solved a few tic tac toe boards I gave it with a pretty nice/clean chain of thought (many SOTA models often fail these!). So I upped the difficulty and asked it to generate 3 "tricky" tic tac toe boards, which it failed on (generating nonsense boards / text), but then so did o1 pro.

✅ I uploaded GPT-2 paper. I asked a bunch of simple lookup questions, all worked great. Then asked to estimate the number of training flops it took to train GPT-2, with no searching. This is tricky because the number of tokens is not spelled out so it has to be partially estimated and partially calculated, stressing all of lookup, knowledge, and math. One example is 40GB of text ~= 40B characters ~= 40B bytes (assume ASCII) ~= 10B tokens (assume ~4 bytes/tok), at ~10 epochs ~= 100B token training run, at 1.5B params and with 2+4=6 flops/param/token, this is 100e9 X 1.5e9 X 6 ~= 1e21 FLOPs. Both Grok 3 and 4o fail this task, but Grok 3 with Thinking solves it great, while o1 pro (GPT thinking model) fails.

I like that the model *will* attempt to solve the Riemann hypothesis when asked to, similar to DeepSeek-R1 but unlike many other models that give up instantly (o1-pro, Claude, Gemini 2.0 Flash Thinking) and simply say that it is a great unsolved problem. I had to stop it eventually because I felt a bit bad for it, but it showed courage and who knows, maybe one day...

The impression overall I got here is that this is somewhere around o1-pro capability, and ahead of DeepSeek-R1, though of course we need actual, real evaluations to look at.

DeepSearch
Very neat offering that seems to combine something along the lines of what OpenAI / Perplexity call "Deep Research", together with thinking. Except instead of "Deep Research" it is "Deep Search" (sigh). Can produce high quality responses to various researchy / lookupy questions you could imagine have answers in article on the internet, e.g. a few I tried, which I stole from my recent search history on Perplexity, along with how it went:

- ✅ "What's up with the upcoming Apple Launch? Any rumors?"
- ✅ "Why is Palantir stock surging recently?"
- ✅ "White Lotus 3 where was it filmed and is it the same team as Seasons 1 and 2?"
- ✅ "What toothpaste does Bryan Johnson use?"
- ❌ "Singles Inferno Season 4 cast where are they now?"
- ❌ "What speech to text program has Simon Willison mentioned he's using?"

❌ I did find some sharp edges here. E.g. the model doesn't seem to like to reference X as a source by default, though you can explicitly ask it to. A few times I caught it hallucinating URLs that don't exist. A few times it said factual things that I think are incorrect and it didn't provide a citation for it (it probably doesn't exist). E.g. it told me that "Kim Jeong-su is still dating Kim Min-seol" of Singles Inferno Season 4, which surely is totally off, right? And when I asked it to create a report on the major LLM labs and their amount of total funding and estimate of employee count, it listed 12 major labs but not itself (xAI).

The impression I get of DeepSearch is that it's approximately around Perplexity DeepResearch offering (which is great!), but not yet at the level of OpenAI's recently released "Deep Research", which still feels more thorough and reliable (though still nowhere perfect, e.g. it, too, quite incorrectly excludes xAI as a "major LLM labs" when I tried with it...).

Random LLM "gotcha"s

I tried a few more fun / random LLM gotcha queries I like to try now and then. Gotchas are queries that specifically on the easy side for humans but on the hard side for LLMs, so I was curious which of them Grok 3 makes progress on.

✅ Grok 3 knows there are 3 "r" in "strawberry", but then it also told me there are only 3 "L" in LOLLAPALOOZA. Turning on Thinking solves this.
✅ Grok 3 told me 9.11 > 9.9. (common with other LLMs too), but again, turning on Thinking solves it.
✅ Few simple puzzles worked ok even without thinking, e.g. *"Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?"*. E.g. GPT4o says 2 (incorrectly).
❌ Sadly the model's sense of humor does not appear to be obviously improved. This is a common LLM issue with humor capability and general mode collapse, famously, e.g. 90% of 1,008 outputs asking ChatGPT for joke were repetitions of the same 25 jokes​. Even when prompted in more detail away from simple pun territory (e.g. give me a standup), I'm not sure that it is state of the art humor. Example generated joke: "*Why did the chicken join a band? Because it had the drumsticks and wanted to be a cluck-star!*". In quick testing, thinking did not help, possibly it made it a bit worse.
❌ Model still appears to be just a bit too overly sensitive to "complex ethical issues", e.g. generated a 1 page essay basically refusing to answer whether it might be ethically justifiable to misgender someone if it meant saving 1 million people from dying.
❌ Simon Willison's "*Generate an SVG of a pelican riding a bicycle*". It stresses the LLMs ability to lay out many elements on a 2D grid, which is very difficult because the LLMs can't "see" like people do, so it's arranging things in the dark, in text. Marking as fail because these pelicans are qutie good but, but still a bit broken (see image and comparisons). Claude's are best, but imo I suspect they specifically targeted SVG capability during training.

Summary. As far as a quick vibe check over ~2 hours this morning, Grok 3 + Thinking feels somewhere around the state of the art territory of OpenAI's strongest models (o1-pro, $200/month), and slightly better than DeepSeek-R1 and Gemini 2.0 Flash Thinking. Which is quite incredible considering that the team started from scratch ~1 year ago, this timescale to state of the art territory is unprecedented. Do also keep in mind the caveats - the models are stochastic and may give slightly different answers each time, and it is very early, so we'll have to wait for a lot more evaluations over a period of the next few days/weeks. The early LM arena results look quite encouraging indeed. For now, big congrats to the xAI team, they clearly have huge velocity and momentum and I am excited to add Grok 3 to my "LLM council" and hear what it thinks going forward.

[Alexander Doria @Dorialexander]

I release my first attempts at training a base model with GRPO. In a similar spirit to R0, this colab notebook transforms Pleias-350m into an RL poet without any post-training data, using only reward functions. https://t.co/tYSp8NYI1s

[Mark Minervini @markminervini]

A study was done on 77,000 accounts at a large discount broker from 1990 to 1996. The findings revealed:

1. Investors are more likely to allow a stock to reach a large loss than they are to allow a stock to attain a large gain; they hold losers too long and sell winners too quickly.

2. The probability of buying additional shares is greater for shares that have lost value than it is for shares that have gained value. Investors may readily double down their bets when stocks decline in value.

3. Investors are more likely to take a small gain than a small loss.

The results would be no different 50 years ago, or 50 years from now.

[Benjamin F Spector @bfspector]

We got early access to some of the very first Nvidia B200’s. We share initial benchmark results and wrote the fastest (public) attention kernel with 925+ BF16 TFLOPs:

Since the PTX instruction set released yesterday, @aaryan04 and I have been hard at work at @HazyResearch learning the new instructions and putting them to the silicon.

[Casper Hansen @casper_hansen_]

The DeepSeek R1 training procedure confused me at first. My brain refused to accept this powerful model could be incredibly straightforward.

Let me break down this elegant beast for you 🧵