Wojciech Kozyra

[Software Mansion @swmansion]

A live chess broadcast that detects the game state and renders the best move in real time - with no human in the loop, all thanks to the new Smelter side channel API.

How it works 👇

[Software Mansion @swmansion]

One video, many sources – all rendered on the GPU in real time 🚀

Combine Twitch, Kick, HLS, or WebRTC streams into a single output with just a few lines ✨

Try it yourself – check out the demo: https://t.co/nb0BlKfftM

[Software Mansion @swmansion]

It turns out that the most popular solutions, like FFmpeg or GStreamer, aren’t really the best fit for composing live streams together. 👀

So we dug into FFmpeg alternatives & what actually works for dynamic, real-time video mixing. 🎥

[Software Mansion @swmansion]

Building a custom React renderer lets you extend React's capabilities to different platforms. 🚀

If you want to learn how to map your own API to React components using react-reconciler, check our article for a deep dive into the process.👇
https://t.co/K9RZ6msns2

[Software Mansion @swmansion]

With React, you can:

✅ Compose a simple gaming stream
✅ Incorporate it into an app
✅ And send it right to Twitch 🚀

Curious how? Read our newest blogpost & learn the process step-by-step – link in the comments!👇

[Software Mansion @swmansion]

The best part about Smelter? With React components, you can reuse stream setups across environments 🎉

Whether you're in the browser, on a server, or running locally, just define your scene, apply effects, and add overlays once to achieve the same effect everywhere.

[Elixir by Software Mansion @swmansionElixir]

Smelter makes programmatic video composition simple and flexible:

✅ Lets you deploy on your own infrastructure
✅ Supports multiple formats and protocols like MP4, WHIP, or DeckLink devices
✅ Enables GPU-accelerated hardware decoding and rendering

[Software Mansion @swmansion]

Streaming on Twitch and need an easy way to add your webcam, screen, and overlays?

With Smelter, you can create rich video streams straight in your browser.

This means no need for servers & lower infrastructure costs 👌

[Elixir by Software Mansion @swmansionElixir]

Not everything needs to be live👀 Sometimes you just need a quick way to process videos offline without having to buy or learn complex software

With Smelter, you can convert, mix, and generate multimedia files offline with easy-to-use React components

No servers, no extra setup

[Wojciech Kozyra @wkozyra95]

@The__gabbanator @swmansion To clarify, the way our React SDK works, is that React renders to JSON instead to DOM. When something changes in the state we send update to the Smelter media server to change video layout.

[Wojciech Kozyra @wkozyra95]

@The__gabbanator @swmansion Currently no, we are considering support for running React part in React Native would be relatively easy if actual rendering happens on the server, but moving everything to mobile device is a lot larger challenge.

[Software Mansion @swmansion]

Meet Smelter – our new toolkit for programmatic video composition! 🚀

With Smelter you can:
✅combine multimedia from different sources into a single video or live stream
✅enrich your content with text, custom shaders, and embedded websites
✅do it all effortlessly with ready-to-use React components

[Wojciech Kozyra @wkozyra95]

@t3dotgg @sebastienlorber @swmansion @react Of course, we also plan to add WHIP/WHEP and RTMP to handle those specific cases better.

[Wojciech Kozyra @wkozyra95]

@t3dotgg @sebastienlorber @swmansion @react Yup definitely, but RTP turned out to be a good initial protocol for us.

It might be problematic if you want to stream from OBS, or send output to Twitch/YouTube, but it is great if you want to embed compositor in your existing solution, especially if it's WebRTC based.

[Wojciech Kozyra @wkozyra95]

@BarryCarlyon @TurkeyDev As for the performance it depends where it runs. If it runs on the instance with GPU then rendering part is negligible compared to encoding and decoding.

Rendering is a lot more expensive without GPU. From my testing I was able to handle few streams on AWS 4 vCPU instance.

[Wojciech Kozyra @wkozyra95]

@BarryCarlyon @TurkeyDev We have POC of compiling our rendering engine to WASM (and WebGL), so soon it will be possible to run that purely in the browser without running any additional infrastructure.

[Wojciech Kozyra @wkozyra95]

@BarryCarlyon @TurkeyDev This is headless only actually. The React there might be confusing, SDK only supports Node.js runtime and it does not run in the browser.

It works by running React code in Node.js, but instead of updating browser DOM it sends request to the compositor server.

[Wojciech Kozyra @wkozyra95]

@Kaiyes_ @swmansion @react You can use tools like FFmpeg, GStreamer, or Membrane Framework to convert whatever format you have to RTP.

Membrane specifically has an integration with compositor, but with this approach you would not be able to use React.

[Wojciech Kozyra @wkozyra95]

@Kaiyes_ @swmansion @react Currently, we only support RTP protocol, so you need something to deliver your streams in this format. However, we plan to add WebRTC (WHIP/WHEP) very soon and after that most likely RTMP too.

[Wojciech Kozyra @wkozyra95]

@Kaiyes_ @swmansion @react - Layout like this should not be a problem.
- You can register multiple outputs, so one of them would be a stream and the second one would just record to mp4.

As for the live stream part, it depends what you need.