Khai Tran

PhD student in AI/NLP @UQSchoolEECS
(he/him)

[Andrej Karpathy @karpathy]

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:

Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:

Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:

Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:

Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!

In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.

[Ansh Bhatnagar @iansh04_]

Anthropic engineer:

"99% of people use Claude Code like Google, and only 1% are running swarms of self-learning Claude agents

I'm running 100+ agents in a loop. I have Chief agent, PM agents - they manage the whole team"

in a 30-minute workshop, an Anthropic engineer revealed how to get max value from Claude Code at min. cost

this is worth more than another $500 vibe-coding course

watch today, then read how to build self-improving agentic systems with Fable in the article below

[Mahax @Mahaximus_]

Google just dropped the best 1-hour course on Graph Engineering: from one agent to a full 24/7 system

00:00 - What Graphs are
09:16 - Build an agent
21:15 - Graph engineering explained
41:03 - Graph engineering practice
52:21 - Self improving Graphs

Free, the best thing on Graph engineering I've come across

Watch it, then read the full guide on agents and graphs below.

[rari @0xwhrrari]

Google Brain founder Andrew Ng:

"Prompting will be dead in 6 months"

"Loops and graphs are replacing it"

In just 2 hours, he shows how to build agents that work, learn, and improve on their own

LLMs → Agents → Loops → Graphs → Self-Improving Systems

The first 15 minutes alone are more valuable than most $500 AI courses

Most people are still learning prompts while AI engineering is already moving beyond them

Watch the lecture first

Then read the full guide to loops and graphs below ↓

[Rohan Paul @rohanpaul_ai]

Good GPU performance summaries - in 6 mints.

[Rohan Paul @rohanpaul_ai]

Fei-Fei Li (@drfeifei) on limitations of LLMs.

"There's no language out there in nature. You don't go out in nature and there's words written in the sky for you.. There is a 3D world that follows laws of physics."

Language is purely generated signal.

[Thinking Machines @thinkymachines]

Today Thinking Machines Lab is launching our research blog, Connectionism. Our first blog post is “Defeating Nondeterminism in LLM Inference”

We believe that science is better when shared. Connectionism will cover topics as varied as our research is: from kernel numerics to prompt engineering. Here we share what we are working on and connect with the research community frequently and openly.

The name Connectionism is a throwback to an earlier era of AI; it was the name of the subfield in the 1980s that studied neural networks and their similarity to biological brains.

https://t.co/lrJioBmpbT

[Akshay 🚀 @akshay_pachaar]

A team at DeepMind wrote this piece on how you must think about GPUs. Essential for AI engineers and researchers:

[Ming Yin @MingYin_0312]

I implemented GRPO and DPO from scratch in vanilla Pytorch to unravel every piece of training details. Hope it could be helpful for those who care about the implementation details of the algorithms. 👉 https://t.co/1Exq7GTkLY #AI #RL #LLM

[Dorsa @dorsa_rohani]

Awesome that people find the ML from scratch repo useful! =)

Added new transformer implementations for you to explore

[Rohan Paul @rohanpaul_ai]

Beautiful @GoogleResearch paper.

LLMs can learn in context from examples in the prompt, can pick up new patterns while answering, yet their stored weights never change.

That behavior looks impossible if learning always means gradient descent.

The mechanisms through which this can happen are still largely unknown.

The authors ask whether the transformer’s own math hides an update inside the forward pass.

They show, each prompt token writes a rank 1 tweak onto the first weight matrix during the forward pass, turning the context into a temporary patch that steers the model like a 1‑step finetune.

Because that patch vanishes after the pass, the stored weights stay frozen, yet the model still adapts to the new pattern carried by the prompt.

🧵 Read on 👇

[elvis @omarsar0]

Small Language Models are the Future of Agentic AI

Lots to gain from building agentic systems with small language models.

Capabilities are increasing rapidly!

AI devs should be exploring SLMs.

Here are my notes:

[Ruben Hassid @rubenhassid]

BREAKING: Apple just proved AI "reasoning" models like Claude, DeepSeek-R1, and o3-mini don't actually reason at all.

They just memorize patterns really well.

Here's what Apple discovered:

(hint: we're not as close to AGI as the hype suggests)

[Jack Morris @jxmnop]

did you know people have been training neural networks on text since 2003?

everyone talks about Attention Is All You Need. but this is the real paper that got our field started. it was in 2003, in montreal.

i read it, and it was even more forward-thinking than i expected:

[Ziming Liu @ZimingLiu11]

Interested in the science of language models but tired of neural scaling laws? Here's a new perspective: our new paper presents neural thermodynamic laws -- thermodynamic concepts and laws naturally emerge in language model training!

AI is naturAl, not Artificial, after all.

[Pramod Goyal @goyal__pramod]

Most influential LLM papers and the ideas they introduced (post 2017)

A long thread 🧵

[Hieu Pham @hyhieu226]

https://t.co/4LQd6Lj0lY
The class is such an enjoyable journey to follow. In an era where most papers are useless, classes like this offer an excellent distillation into recent notable papers. Also, the class makes me dearly miss my time as a PhD student at CMU.

[leonie @helloiamleonie]

New to fine-tuning LLMs?
Confused by all the jargon?

Me, too.

So, I did a little deep dive into LLM fine-tuning.
Here’s what I understood:

[Tanishq Mathew Abraham, Ph.D. @iScienceLuvr]

Diffusion Guided Language Modeling

abs: https://t.co/PLvpRVAYa8

"In this paper we use a guided diffusion model to produce a latent proposal that steers an auto-regressive language model to generate text with desired properties. Our model inherits the unmatched fluency of the auto-regressive approach and the plug-and-play flexibility of diffusion."

[Prof Lennart Nacke, PhD @acagamic]

A mindmap for PhD students struggling with their thesis structure