Alvin Kimbowa

Ph.D Candidate @UBC SBME

[Greg Brockman @gdb]

the bottleneck is increasingly knowing what you want

[Hossein Mobahi @TheGradient]

Excited to share about what I’ve been working on over the past year: quantifying the capacity of a neuron😅

This led to a mathematical framework we call HOPE, which lets us rigorously deconstruct what deep networks might have learned. Paper: https://t.co/bDn03zBkS0

[François Chollet @fchollet]

The hardest part of any task is overcoming the activation energy. The rest is just riding the momentum.

[Image and Video Processing Papers @CIGX]

MonoUNet: A Robust Tiny Neural Network for Automated Knee Cartilage Segmentation on Point-of-Care Ultrasound Devices

Alvin Kimbowa, Arjun Parmar, Ibrahim Mujtaba, Will Wei, …
https://t.co/LGiuu4he8V [𝚎𝚎𝚜𝚜.𝙸𝚅 𝚌𝚜.𝙲𝚅]
💬Accepted to Ultrasound in Medicine &amp

[Akshay 🚀 @akshay_pachaar]

DeepSeek just fixed one of AI's oldest problems.

(using a 60-year-old algorithm)

Here's the story:

When deep learning took off, researchers hit a wall. You can't just stack layers endlessly. Signals either explode or vanish. Training deep networks was nearly impossible.

ResNets solved this in 2016 with residual connections:

output = input + what the layer learned

That "+" creates a direct highway for information. This is why we can now train networks with hundreds of layers.

Recently, researchers asked: what if we had multiple highways instead of one?

Hyper-Connections (HC) expanded that single lane into 4 parallel lanes with learnable matrices that mix information between streams.

The performance gains were real. But there was a problem:

Those mixing matrices compound across layers. A tiny 5% amplification per layer becomes 18x after 60 layers. The paper measured amplification reaching 3000x. Training collapses.

The usual fixes? Gradient clipping. Careful initialization. Hoping things work out.

These are hacks. And hacks don't scale.

DeepSeek went back to first principles. What mathematical constraint would guarantee stability?

The answer was sitting in a 1967 paper: the Sinkhorn-Knopp algorithm.

It forces mixing matrices to be "doubly stochastic," where rows and columns each sum to 1.

The results:

- 3000x instability reduced to 1.6x
- Stability guaranteed by math, not luck
- Only 6.7% additional training overhead

No hacks. Just math.

I've shared link to the paper in the next tweet.

[Jay | Indilingo @jaygala223]

How to Get Into AI Research in 2026

Finding good problems is harder than solving them. That's the most important lesson I've learned as an AI researcher at Microsoft Research India.

I started with a couple of undergrad research internships that I got purely through cold-emailing professors. No connections, just persistent emails that referenced their work and showed I had thought about it. Those experiences taught me how to read papers critically, spot gaps, generate ideas, and think about research properly.

If you're aiming for AI research roles at frontier labs in 2026, here are the practical things that have mattered most in my own path.

First, build genuinely strong technical foundations. Implement key components from scratch instead of relying only on high-level libraries. For example, I thought I understood KV caching well when I was optimizing it during my stint at Intel, but only when I coded it myself did I realize my knowledge was surface-level. A similar thing happened when I was working on speculative decoding. So, I suggest that you do the same and implement things like attention mechanisms, transformers, basic RL algorithms, or whatever interests you from scratch to truly understand them. Start in pytorch or jax and force yourself to write the core parts without copying existing code. This kind of hands-on work reveals gaps that reading alone never does, and in 2026, with ever-larger models, labs will need people who can reason about and innovate on low-level details.

Next, get early research experience. Cold-emailing professors still works. Read their recent papers, mention specific things that caught your attention, explain why their direction interests you, and suggest (modestly) how you could help. Keep the email short and genuine. Attach or link evidence of independent work, like a GitHub repo with your from-scratch implementations.

Those early projects (or independent work if internships are hard to get) will teach you the real skills: reading papers carefully, identifying open questions, iterating on ideas. Practice regularly. Read papers from arxiv or conference proceedings often, summarize papers, discuss them with peers if possible. Remember that identifying worthwhile problems is usually the bottleneck, not the technical solution once the problem is clear.

Crucially, don't chase hot topics just because everyone else is. Research only feels sustainable when the problem is personally compelling, when it nags at you enough that you think about it outside work hours. If a trendy area leaves you cold, skip it and find something that actually excites you, even if it's less popular. Genuine curiosity is what keeps you going through the inevitable dead ends.

For getting into frontier labs in 2026: strong publications help a lot, but demonstrated technical depth matters too (open-source contributions, personal projects with measurable impact). Advanced degrees are common but not strictly required if your work stands on its own. Put preprints on arxiv, target workshops or conferences, stay current by following active researchers.

Other notes: expect plenty of frustration and failed experiments; that's normal. Collaborate when you can. Think early about safety, robustness, and broader implications. And keep coding regularly.

Good luck 🤞

[Hieu Pham @hyhieu226]

There is a necessary skill in research and engineering that will get you a lot of hate.

It is the skill to look at someone's work, including your own, and including everything like ideas, papers, products, etc. and with solid reasons, say "this is bullshit."

[mitsuri @0xmitsurii]

Steve Jobs believe that this is the key to start a business.

[JNS @_devJNS]

one of the greatest or programming jokes of all time.

[Ryanair @Ryanair]

no thanks

[Mindset Machine  @mindsetmachine]

Steve Jobs definition of focus is insane:

[Sam Altman @sama]

Yesterday we did a livestream. TL;DR:

We have set internal goals of having an automated AI research intern by September of 2026 running on hundreds of thousands of GPUs, and a true automated AI researcher by March of 2028. We may totally fail at this goal, but given the extraordinary potential impacts we think it is in the public interest to be transparent about this.

We have a safety strategy that relies on 5 layers: Value alignment, Goal alignment, Reliability, Adversarial robustness, and System safety. Chain-of-thought faithfulness is a tool we are particularly excited about, but it somewhat fragile and requires drawing a boundary and a clear abstraction.

On the product side, we are trying to move towards a true platform, where people and companies building on top of our offerings will capture most of the value. Today people can build on our API and apps in ChatGPT; eventually, we want to offer an AI cloud that enables huge businesses.

We have currently committed to about 30 gigawatts of compute, with a total cost of ownership over the years of about $1.4 trillion. We are comfortable with this given what we see on the horizon for model capability growth and revenue growth. We would like to do more—we would like to build an AI factory that can make 1 gigawatt per week of new capacity, at a greatly reduced cost relative to today—but that will require more confidence in future models, revenue, and technological/financial innovation.

Our new structure is much simpler than our old one. We have a non-profit called OpenAI Foundation that governs a Public Benefit Corporation called OpenAI Group. The foundation initially owns 26% of the PBC, but it can increase with warrants over time if the PBC does super well. The PBC can attract the resources needed to achieve the mission.

Our mission, for both our non-profit and PBC, remains the same: to ensure that artificial general intelligence benefits all of humanity.

The nonprofit is initially committing $25 billion to health and curing disease, and AI resilience (all of the things that could help society have a successful transition to a post-AGI world, including technical safety but also things like economic impact, cyber security, and much more). The nonprofit now has the ability to actually deploy capital relatively quickly, unlike before.

In 2026 we expect that our AI systems may be able to make small new discoveries; in 2028 we could be looking at big ones. This is a really big deal; we think that science, and the institutions that let us widely distribute the fruits of science, are the most important ways that quality of life improves over time.

[Zahra Kadkhodaie @ZKadkhodaie]

Diffusion models learn probability densities by estimating the score with a neural network trained to denoise. What kind of representation arises within these networks, and how does this relate to the learned density? @EeroSimoncelli @StephaneMallat and I explored this question.

[Justin Johnson @jcjohnss]

10 years ago, deep learning was in its infancy. PyTorch didn't exist. Language models were recurrent, and not large. But it felt important: a new technology that would change everything.

That's why @drfeifei , @karpathy, and I started @cs231n back in 2015 - to teach the world's best deep learning class and make these new ideas accessible to everyone.

A lot has changed since then. But this year I was honored to return to the classroom for the 10th anniversary of cs231n and teach a new generation of students alongside @drfeifei, @eadeli, and Zane Durante.

As usual, lecture videos are free for all to enjoy:

https://t.co/QCHJRxgHzy

[Rudy Gilman @rgilman33]

DINO-v3 has a single high-magnitude channel on its residual pathway, channel 416. Turning off this single channel affects DINO's entire output by 50-80%. For context, turning off a random channel has an effect of less than one percent.

The model builds up channel 416 in its last two layer-scale operations, using a single high-magnitude weight in each op to drastically ramp up channel 416's magnitude. This channel doesn't depend on the input, every image fires with a constant overlay.

After bringing channel 416 up to a value of about ten thousand, DINO-v3 then scales it down in the final layer-norm to almost nothing, removing it without a trace.

[Andrej Karpathy @karpathy]

Continuing the journey of optimal LLM-assisted coding experience. In particular, I find that instead of narrowing in on a perfect one thing my usage is increasingly diversifying across a few workflows that I "stitch up" the pros/cons of:

Personally the bread & butter (~75%?) of my LLM assistance continues to be just (Cursor) tab complete. This is because I find that writing concrete chunks of code/comments myself and in the right part of the code is a high bandwidth way of communicating "task specification" to the LLM, i.e. it's primarily about task specification bits - it takes too many bits and too much latency to communicate what I want in text, and it's faster to just demonstrate it in the code and in the right place. Sometimes the tab complete model is annoying so I toggle it on/off a lot.

Next layer up is highlighting a concrete chunk of code and asking for some kind of a modification.

Next layer up is Claude Code / Codex / etc, running on the side of Cursor, which I go to for larger chunks of functionality that are also fairly easy to specify in a prompt. These are super helpful, but still mixed overall and slightly frustrating at times. I don't run in YOLO mode because they can go off-track and do dumb things you didn't want/need and I ESC fairly often. I also haven't learned to be productive using more than one instance in parallel - one already feels hard enough. I haven't figured out a good way to keep CLAUDE[.]md good or up to date. I often have to do a pass of "cleanups" for coding style, or matters of code taste. E.g. they are too defensive and often over-use try/catch statements, they often over-complicate abstractions, they overbloat code (e.g. a nested if-the-else constructs when a list comprehension or a one-liner if-then-else would work), or they duplicate code chunks instead of creating a nice helper function, things like that... they basically don't have a sense of taste. They are indispensable in cases where I inch into a more vibe-coding territory where I'm less familiar (e.g. writing some rust recently, or sql commands, or anything else I've done less of before). I also tried CC to teach me things alongside the code it was writing but that didn't work at all - it really wants to just write code a lot more than it wants to explain anything along the way. I tried to get CC to do hyperparameter tuning, which was highly amusing. They are also super helpful in all kinds of lower-stakes one-off custom visualization or utilities or debugging code that I would never write otherwise because it would have taken way too long. E.g. CC can hammer out 1,000 lines of one-off extensive visualization/code just to identify a specific bug, which gets all deleted right after we find it. It's the code post-scarcity era - you can just create and then delete thousands of lines of super custom, super ephemeral code now, it's ok, it's not this precious costly thing anymore.

Final layer of defense is GPT5 Pro, which I go to for the hardest things. E.g. it has happened to me a few times now that I / Cursor / CC are all stuck on a bug for 10 minutes, but when I copy paste the whole thing to 5 Pro, it goes off for 10 minutes but then actually finds a really subtle bug. It is very strong. It can dig up all kinds of esoteric docs and papers and such. I've also used it for other meatier tasks, e.g. suggestions on how to clean up abstractions (mixed results, sometimes good ideas but not all), or an entire literature review around how people do this or that and it comes back with good relevant resources / pointers.

Anyway, coding feels completely blown open with possibility across a number of "kinds" of coding and then a number of tools with their pros/cons. It's hard to avoid the feeling of anxiety around not being at the frontier of what is collectively possible, hence random sunday shower of thoughts and a good amount of curiosity about what others are finding.

[AI at Meta @AIatMeta]

Introducing DINOv3: a state-of-the-art computer vision model trained with self-supervised learning (SSL) that produces powerful, high-resolution image features. For the first time, a single frozen vision backbone outperforms specialized solutions on multiple long-standing dense prediction tasks.

Learn more about DINOv3 here: https://t.co/lQpKhJLTZQ

[Z Fellows @zfellows]

Psst...
Are you technical and tinkering on side projects?

We will give you $10,000 to skip class/job for a week and go all in on your idea.

Apply here:
https://t.co/VWPyNDB1x7