Jimmy Neuron

I work as a Solutions Architect at AWS. I specialize in Neuron running on Inferentia and Trainium! This is my personal account.

[Jimmy Neuron @JimNeuron]

Neuron SDK 2.29 for Trainium was released. Neuron Kernel Interface (NKI) is out of beta and gets a CPU emulator. Interesting news is that NxDI (the library that parallelizes inference across multiple cores) no longer supports Inferentia2 and Trainium1. #AWS #Trainium

[Jimmy Neuron @JimNeuron]

Gemma 4, now available on Trainium: https://t.co/QO5lkG4gKT

[Jimmy Neuron @JimNeuron]

SDK 2.28 released this week. Includes Qwen-VL support! And a lot more granular access on the Neuron Kernel Interface. Plus an even more amazing performance explorer! https://t.co/p3JF5EceOs #AWS #trainium #inferentia #neuronSDK

[Jimmy Neuron @JimNeuron]

@SemiAnalysis_ Glad to see all the modern Neuron devices included. More options == better choices.

[Jimmy Neuron @JimNeuron]

You get 144 devices maximum in an UltraServer, so that would be two of these beasts.

[Jimmy Neuron @JimNeuron]

Trainium3 UltraServer. Your only chance to see one outside of an #AWS datacenter. If you count, you can see the 9 compute sleds on top, the NeuronLink sleds in the middle, and the 9 compute sleds on the bottom. 4 devices per sled = 72 devices. #trainium #AI

[Jimmy Neuron @JimNeuron]

Trainium3! Straight from the AWS re:Invent floor. Two have heat sinks, two are uncovered for you to admire. 4 per sled. That is important when you see it in the rack. #AWS #Trainium #AI

[Jimmy Neuron @JimNeuron]

@AWSAI I'll be there on Saturday leading the workshop. DM me if you have trouble getting approved.

[Jimmy Neuron @JimNeuron]

Trainium users will soon have trn2.3xlarge instances. Just posted on capacity blocks in Melbourne. These are a single Trainium2 device.

If you run 8B models with data parallelism of 3 on an inf2.48xlarge, this could be a really good fit. Also good for kernel development.

[Jimmy Neuron @JimNeuron]

If you want to upgrade your current Trainium or Inferentia instance to the latest (2.26) Neuron SDK, I've got some scripts here to help: https://t.co/ilJRJQ0lZC

[Jimmy Neuron @JimNeuron]

Trying to get some LeRobot policies working with AWS Trainium. Currently blocked by https://t.co/t0OqbAAmfK (I was testing with the 0.3.3. version).

[Jimmy Neuron @JimNeuron]

If you work with the AWS Neuron SDK for Trainium chips, check out my utility to tell you what version of the SDK you are actually running and help you with changes: https://t.co/AaKEPXWOhp

[Amazon @amazon]

Take a tour through @awscloud Trainium's microchip metropolis:

🏙️ The Systolic Array: Our "downtown" where intensive calculations run around the clock, with billions of transistors packed into an area smaller than a postage stamp

🛣️ The Data Bus: Like city highways and transit systems, moving information at near light speed

🏠 Memory Cells: Our "outer boroughs" - strategic storage spaces holding massive datasets for AI processing

⚡ The Interposer: The underground infrastructure creating vital connections, just like a city's power lines and utilities

🌇 The Greater Metropolitan Area: Where interconnected chips work in concert to form Ultra Servers, powering the next wave of generative AI

Learn how AWS chip architects think like city planners when designing and building the tiny devices that are helping to advance the development of gen AI.

[Jimmy Neuron @JimNeuron]

I reached a high point in my career I never even imagined:

I created a .tar.gz file correctly, the first time, without looking anything up!

I didn't even use the backspace key!!

[Jimmy Neuron @JimNeuron]

I'm working with this team. DM me with questions.

[Jimmy Neuron @JimNeuron]

Funding for academic AI research! If you are with an academic institution, AWS has $110 million in credits available for you to train a model, build your model, or work on hardware optimization. Using our custom AI chip, Trainium. Call for proposals: https://t.co/s4C1z6o5rB

[Jimmy Neuron @JimNeuron]

All the details here: https://t.co/fmUql0zRvD

[Jimmy Neuron @JimNeuron]

Neuron SDK 2.23 released! The NxDI library for parallelizing your models on Inferentia and Trainium is out of beta!! Jax 0.5.3 support, some NKI changes, and performance improvements.