Neuron SDK 2.29 for Trainium was released. Neuron Kernel Interface (NKI) is out of beta and gets a CPU emulator. Interesting news is that NxDI (the library that parallelizes inference across multiple cores) no longer supports Inferentia2 and Trainium1. #AWS#Trainium
SDK 2.28 released this week. Includes Qwen-VL support! And a lot more granular access on the Neuron Kernel Interface. Plus an even more amazing performance explorer! https://t.co/p3JF5EceOs #AWS#trainium#inferentia#neuronSDK
Trainium3 UltraServer. Your only chance to see one outside of an #AWS datacenter. If you count, you can see the 9 compute sleds on top, the NeuronLink sleds in the middle, and the 9 compute sleds on the bottom. 4 devices per sled = 72 devices. #trainium#AI
Trainium3! Straight from the AWS re:Invent floor. Two have heat sinks, two are uncovered for you to admire. 4 per sled. That is important when you see it in the rack. #AWS#Trainium#AI
Trainium users will soon have trn2.3xlarge instances. Just posted on capacity blocks in Melbourne. These are a single Trainium2 device.
If you run 8B models with data parallelism of 3 on an inf2.48xlarge, this could be a really good fit. Also good for kernel development.
If you want to upgrade your current Trainium or Inferentia instance to the latest (2.26) Neuron SDK, I've got some scripts here to help: https://t.co/ilJRJQ0lZC
If you work with the AWS Neuron SDK for Trainium chips, check out my utility to tell you what version of the SDK you are actually running and help you with changes: https://t.co/AaKEPXWOhp
Take a tour through @awscloud Trainium's microchip metropolis:
🏙️ The Systolic Array: Our "downtown" where intensive calculations run around the clock, with billions of transistors packed into an area smaller than a postage stamp
🛣️ The Data Bus: Like city highways and transit systems, moving information at near light speed
🏠 Memory Cells: Our "outer boroughs" - strategic storage spaces holding massive datasets for AI processing
⚡ The Interposer: The underground infrastructure creating vital connections, just like a city's power lines and utilities
🌇 The Greater Metropolitan Area: Where interconnected chips work in concert to form Ultra Servers, powering the next wave of generative AI
Learn how AWS chip architects think like city planners when designing and building the tiny devices that are helping to advance the development of gen AI.
I reached a high point in my career I never even imagined:
I created a .tar.gz file correctly, the first time, without looking anything up!
I didn't even use the backspace key!!
Funding for academic AI research! If you are with an academic institution, AWS has $110 million in credits available for you to train a model, build your model, or work on hardware optimization. Using our custom AI chip, Trainium. Call for proposals: https://t.co/s4C1z6o5rB
Neuron SDK 2.23 released! The NxDI library for parallelizing your models on Inferentia and Trainium is out of beta!! Jax 0.5.3 support, some NKI changes, and performance improvements.