Matt Hawkins

CEO @CUDO_Compute, Passionate about creating innovative technology to improve the world with sustainable computing.

[CUDO Compute @Cudo_Compute]

Compute capacity gets the headlines. Power availability is what actually decides who builds, where, and how fast.

Power availability and grid capacity now rank as the leading factors in AI infrastructure location decisions.
But having power on paper isn't enough.

A site needs the capacity to support the workload, a realistic route to getting that power online and the resilience to keep it running. Land without a clear power pathway is an option, not an asset.

As @WSP’s Anushka Devaser puts it:
“The biggest misunderstanding is assuming securing power is only about price or MW capacity.”

AI infrastructure needs more than an available MW figure. It needs firm, resilient power that can support the workload in practice.

That’s why land, power and compute need to be considered together. Decisions around one quickly affect what’s possible with the others.

Read our Land. Power. Compute. research below.

[CUDO Compute @Cudo_Compute]

21.3% → 55.2% MFU.
That’s more than a 2× difference in useful work on comparable-era hardware.

GPT-3 175B achieved 21.3% MFU. MegaScale reached 55.2%.

The difference is not just the GPU. It’s the system around it.

DeepSeek-V3 is a useful example.

It ran on 2,048 NVIDIA H800 GPUs, hardware built on the same Hopper architecture as the H100 but with NVLink bandwidth reduced from 900 GB/s to 400 GB/s for export compliance.

That single difference in interconnect reshaped the entire training strategy.

The team avoided tensor parallelism because it was inefficient under the constrained NVLink, built a bidirectional pipeline scheme to overlap computation and communication, and relied on eight 400G InfiniBand NICs per node to compensate for the loss of intra-node bandwidth.

The chip was nearly the same. The configuration determined what the cluster could actually do.

Even after that hardware-aware co-design, third-party analysis estimates the run landed at around 23% model FLOPs utilization, a figure the DeepSeek team never had to disclose and one that rarely appears in headline infrastructure specifications.

GPU model and count only tell part of the story. What matters is how effectively the entire system turns that hardware into useful compute.

Read the article below.

[Pete Hill @CudoPete]

So when you're sizing compute at the moment, someone will always tell you they've got available capacity.

The real question isn't how much, it’s; when is it truly available?

[Matt Hawkins @HawkinsTech]

Model choice will keep changing.
The need for reliable land, power and compute will not.

Taking control of an AI model is getting easier. Running it well is not.

Look at the last four weeks.
Meituan open-sourced LongCat-2.0 (1.6T params, ~48B active).

Moonshot shipped Kimi K3 (2.8T params) this week, the full weights went public. Largest open model ever released.

Moonshot paused new subscriptions within 48 hours of launch. Not for lack of demand, for lack of compute to serve it.
Now anyone can download K3. Moonshot's own guidance: run it on a cluster of 64+ accelerators.

Once a workflow becomes core, the questions change:
What does each output cost?
Where does the data go?
Can we guarantee capacity?
Can we switch models without rebuilding?

Open ≠ cheaper. Utilisation and system design decide the economics. Self-hosting wins when a workload runs hot and steady. Otherwise a managed API usually wins.

Either way, someone still has to secure the compute and run it under load.

Capital markets noticed. @Togethercompute just raised $800M at an $8.3B valuation.

Open models don't remove the need for AI infrastructure. They create more buyers for it.

At @Cudo_Compute we build for both sides. Start on demand, prove the use case, move to dedicated or managed capacity as usage gets predictable.

Model choice will keep changing. The need for reliable land, power and compute will not.

[Matt Hawkins @HawkinsTech]

This is how @nvidia sign a deal 😁 at #GTC2024

[Matt Hawkins @HawkinsTech]

If you’re looking to run multilingual LLM’s, @StabilityAI have the highest performing small footprint LLMs. Great for running at very low cost on GPU cloud. How long till someone has this running on your pair of sunglasses when abroad 😎

[Stability AI @StabilityAI]

We’ve published our technical report on Stable LM 2 1.6B; our state-of-the-art small language model trained on multilingual data. This in-depth study covers the training methods utilized for both the base and instruction-tuned versions of the model. (1/3)

[Matt Hawkins @HawkinsTech]

Remarkable to see Claude 3 Opus not just finding the ‘needle’, but also questioning the ‘haystack’ itself! 🧠 This meta-awareness points towards a new horizon in AI, where machines aren’t just tools but participants in problem-solving.
What could this mean for the future of AI-driven research and do you find this scary ? #MachineLearning #AI #Innovation

[Alex Albert @alexalbert__]

Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in-the-haystack eval.

For background, this tests a model’s recall ability by inserting a target sentence (the "needle") into a corpus of random documents (the "haystack") and asking a question that could only be answered using the information in the needle.

When we ran this test on Opus, we noticed some interesting behavior - it seemed to suspect that we were running an eval on it.

Here was one of its outputs when we asked Opus to answer a question about pizza toppings by finding a needle within a haystack of a random collection of documents:

Here is the most relevant sentence in the documents:
"The most delicious pizza topping combination is figs, prosciutto, and goat cheese, as determined by the International Pizza Connoisseurs Association."
However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings.

Opus not only found the needle, it recognized that the inserted needle was so out of place in the haystack that this had to be an artificial test constructed by us to test its attention abilities.

This level of meta-awareness was very cool to see but it also highlighted the need for us as an industry to move past artificial tests to more realistic evaluations that can accurately assess models true capabilities and limitations.

[Matt Hawkins @HawkinsTech]

@matt_arn @CUDOS_ There has been reviews of some other types of verifiable work, but not specifically around LLM’s. This is a good point and I’ll bring that up 👍

[Matt Hawkins @HawkinsTech]

One of the next exciting areas in media for GPU power and #AI is lip-syncing. Alibaba researchers unveiled EMO, an AI that adds lip-syncing to videos. Just give it a still image and any audio, incredible! These aren’t just lip-syncing they are face syncing!

[Matt Hawkins @HawkinsTech]

@CloudPerseus @CUDOS_ Typical text to audio 8GB or 12GB are fine, it depends on the software. It will vary for video to audio though.

[Matt Hawkins @HawkinsTech]

@petkanics @ericxtang @marcolazzarini @atterX_ @CUDOS_ @Livepeer I’ll dm you @petkanics let’s have call

[Matt Hawkins @HawkinsTech]

@marcolazzarini @atterX_ @CUDOS_ @ericxtang @Livepeer It would be great to work with @Livepeer , @ericxtang I’ll reach out to him

[Matt Hawkins @HawkinsTech]

@bindureddy We’re a sustainable distributed GPU compute network, be great to see where we could collaborate on this.

[Matt Hawkins @HawkinsTech]

@KryptoSWE @CUDOS_ Yes we’re working on getting online stats live in Jan

[Matt Hawkins @HawkinsTech]

@StableDiffusion can now read brain waves to evaluate what your are thinking of. Below is a trained AI model using just brain waves to draw (bottom) when someone was looking at the image (top). The power of Open source AI which you can train yourself.

[Matt Hawkins @HawkinsTech]

Amazing news that the industry as a whole is moving to #carbonneutrality .One step at a time, we should get to the stage where all blockchains and crypto can be carbon neutral.

[Michael Saylor @saylor]

Yesterday I was pleased to host a meeting between @elonmusk & the leading Bitcoin miners in North America. The miners have agreed to form the Bitcoin Mining Council to promote energy usage transparency & accelerate sustainability initiatives worldwide.

[Matt Hawkins @HawkinsTech]

Could you fit one of these under your desk? #IBM’s 50 Qubit quantum computer shown off at #CES2020 #IBMQuantum still a long way off #QuantumComputing effecting crypto but it has great potential for shared computing

[Matt Hawkins @HawkinsTech]

Working with @AMD in Toronto, let’s make better use of people underused hardware for #CloudComputing

[Matt Hawkins @HawkinsTech]

Great to see you @IvanOnTech at Korea Blockchain week