dstack

Control plane for agents & engineers to provision compute and run training & inference across NVIDIA, AMD, and other chips — on clouds, Kubernetes, and on-prem.

[dstack @dstackai]

Infrastructure orchestration is becoming an agent skill.

When agents run experiments, training, and eval autonomously, they now can provision compute, schedule workloads, and track state, all inside the loop.

New post on what this means for platform teams, providers, and the orchestration layer

https://t.co/T2jS9FLCOo

[dstack @dstackai]

dstack 0.20.22 is out 🚀

Tenstorrent Blackhole support has landed.

Run dev, training, and inference workloads on Blackhole PCIe cards and systems.

Thanks to the @tenstorrent team and @artem_aero for collaborating on this.

https://t.co/fX5qHA7pCS

[dstack @dstackai]

@DavidSHolz 👀

[Konstantin Willeke @KonstantinWille]

@DavidSHolz @HotAisle +1 for dstack. Fantastic tool if you have heterogeneous compute across different clusters or clouds

[JarvisLabsAI @jarvislabsai]

Most GPU workflows come with too much infra overhead.

Spin up a VM, install dependencies, copy code over, expose ports, watch logs, and remember to shut everything down after.

With JarvisLabs now available as a @dstackai backend, most of that goes away.

You define the machine and the workload in a YAML file. Run dstack apply. That's it.

dstack manages the infrastructure lifecycle. JarvisLabs provides the GPUs.

We wrote a short tutorial covering the full setup, from connecting your account to running a nanochat training job on H100s

Works for training runs, evals, benchmarks, inference services, and GPU dev environments.

[dstack @dstackai]

Training models or serving inference on AMD GPUs?

We’ve refreshed the AMD accelerator example in the dstack docs, covering on-prem fleets, cloud GPU provisioning, dev environments, training jobs, and production-grade inference.

https://t.co/WffI8cKY7t

[dstack @dstackai]

@tenstorrent @artem_aero The release also adds @vast_ai specific backend options for more precise GPU offer filtering.

Full release notes:
https://t.co/fX5qHA7pCS

[dstack @dstackai]

dstack 0.20.22 is out 🚀

Tenstorrent Blackhole support has landed.

Run dev, training, and inference workloads on Blackhole PCIe cards and systems.

Thanks to the @tenstorrent team and @artem_aero for collaborating on this.

https://t.co/fX5qHA7pCS

[dstack @dstackai]

@tenstorrent @artem_aero Read the updated Tenstorrent example for running dev environments, tasks, and services: https://t.co/B741hom0mR

[dstack @dstackai]

Training models or serving inference on AMD GPUs?

We’ve refreshed the AMD accelerator example in the dstack docs, covering on-prem fleets, cloud GPU provisioning, dev environments, training jobs, and production-grade inference.

https://t.co/WffI8cKY7t

[RadixArk @radixark]

Thanks @dstackai for this end-to-end example using Miles with dstack for RL training. Try it and let us know what you build. 👇

[dstack @dstackai]

dstack 0.20.21 is out 🚀

⚡ Kubernetes: multiple clusters via kubeconfig contexts, each exposed as a backend region

⚡ New @jarvislabsai backend: H100/H200 on demand now; RTX PRO 6000 to follow

https://t.co/ORAtgzq7HJ

[dstack @dstackai]

The community asked us for an example of how to use @radixark Miles with dstack for RL training.

Since Miles uses Ray and dstack can run Ray, using Miles with dstack is quite straightforward.

Here’s a new example of running Miles on a multi-node cluster provisioned and managed by dstack:

https://t.co/TOxsg8aBLr

[dstack @dstackai]

@jarvislabsai @andrey_cheptsov Excited to have JarvisLabs as a supported cloud provider! The community needs more on-demand GPUs.

[JarvisLabsAI @jarvislabsai]

JarvisLabs is now a @dstackai backend.

That means: if you've been using dstack to orchestrate runs across GPU clouds, you can now point it at our infrastructure with one YAML block.

type: jarvislabs
creds:
api_key: ...

One YAML. One command. GPU running in few seconds.

L4, H100, and CPU VMs, more on the way.

dstack handles the instance lifecycle. We provide the GPU.

If you've been waiting for it, give it a spin.

[Andrey Cheptsov @andrey_cheptsov]

Needed to test a small training run on @AMD MI300X.

Passed my @digitalocean API key to @dstackai, asked Codex to submit the run, done.

Suspiciously easy.

[LMSYS Org @lmsysorg]

PD-disaggregated SGLang on @AMD GPUs, deployed across multi-node clusters with a single config. Great walkthrough from @dstackai

[dstack @dstackai]

Read how to deploy PD-disaggregated inference on multi-node @AMD GPU clusters using @lmsysorg SGLang and dstack.

One config for an autoscaling endpoint across cloud or on-prem clusters, with @lightseekorg SMG routing and Mooncake KV transfer over RoCE.

https://t.co/CNrpK5shOd

[dstack @dstackai]

dstack 0.20.21 is out 🚀

⚡ Kubernetes: multiple clusters via kubeconfig contexts, each exposed as a backend region

⚡ New @jarvislabsai backend: H100/H200 on demand now; RTX PRO 6000 to follow

https://t.co/ORAtgzq7HJ

[Andrey Cheptsov @andrey_cheptsov]

High-performance LLM inference on @AMD GPU clusters is now much easier to deploy than many people think.

At @dstackai, we’re building an open-source orchestration layer for running inference across GPU vendors, clouds, and on-prem clusters.

It’s exciting to see what becomes possible with projects like @lmsysorg SGLang and SMG.

https://t.co/hCV5YrBEDO

[dstack @dstackai]

Read how to deploy PD-disaggregated inference on multi-node @AMD GPU clusters using @lmsysorg SGLang and dstack.

One config for an autoscaling endpoint across cloud or on-prem clusters, with @lightseekorg SMG routing and Mooncake KV transfer over RoCE.

https://t.co/CNrpK5shOd