Julian Ng-Thow-Hing

cs @ cmu
prev. meta superintelligence labs

[Julian Ng-Thow-Hing @hing3074]

Performance is pretty good too 👀

built during my time at @Meta Superintelligence Labs!

[PyTorch @PyTorch]

ExecuTorch WebGPU, PyTorch models on the GPU, in the browser.

Running a PyTorch model in the browser has usually meant converting it to another framework's format first. With ExecuTorch's new WebGPU backend, you export once with torch.export, and the same .pte file runs in WebGPU browsers such as Chrome and Safari. Everything runs locally in the tab, so prompts, photos and audio never leave the device.

The backend already runs Llama 3.2, Qwen3, Gemma 3, SmolVLM, SAM 2, Depth Anything V2, D-FINE, Real-ESRGAN and Whisper, and supports LoRA and DiReFT fine-tuning on-device.

By Julian Ng-Thow-Hing (@hing3074), Siddartha Pothapragada, Stephen Jia, Digant Desai (@digant23), and Bilgin Cagatay (@BilginCatay).

Read the post: https://t.co/Cwd35vpnch
Try the models in the ExecuTorch WebGPU Playground: https://t.co/8sDhPAgrvK

[PyTorch @PyTorch]

ExecuTorch WebGPU, PyTorch models on the GPU, in the browser.

Running a PyTorch model in the browser has usually meant converting it to another framework's format first. With ExecuTorch's new WebGPU backend, you export once with torch.export, and the same .pte file runs in WebGPU browsers such as Chrome and Safari. Everything runs locally in the tab, so prompts, photos and audio never leave the device.

The backend already runs Llama 3.2, Qwen3, Gemma 3, SmolVLM, SAM 2, Depth Anything V2, D-FINE, Real-ESRGAN and Whisper, and supports LoRA and DiReFT fine-tuning on-device.

By Julian Ng-Thow-Hing (@hing3074), Siddartha Pothapragada, Stephen Jia, Digant Desai (@digant23), and Bilgin Cagatay (@BilginCatay).

Read the post: https://t.co/Cwd35vpnch
Try the models in the ExecuTorch WebGPU Playground: https://t.co/8sDhPAgrvK

[ayesha @ayesha_fatiima]

HOW DID ELON EVEN DO ALL THIS IN ONE LIFETIME 😭

[Julian Ng-Thow-Hing @hing3074]

Just presented my inference research with @Meta Superintelligence Labs at the @PyTorch Conference!

my research focused on building a new path for fast PyTorch inference and fine-tuning directly in the browser without the cloud. We demonstrated competitive performance against alternative frameworks like WebLLM, llama.cpp, and ONNX Runtime Web, and showcased a few examples of on-device AI experiences!

try out a few examples here: https://t.co/sT55J7Njzv

[Julian Ng-Thow-Hing @hing3074]

@davidchung29 looks great david

[Julian Ng-Thow-Hing @hing3074]

@rasmustherat @PrimeIntellect Yep

[Julian Ng-Thow-Hing @hing3074]

@alexandr_wang This is HUGE

[Julian Ng-Thow-Hing @hing3074]

@ToddDongb9xl Nice man

[Julian Ng-Thow-Hing @hing3074]

@rasmustherat @willcb well said

[Julian Ng-Thow-Hing @hing3074]

@southyrnn thank you for the update.

[Julian Ng-Thow-Hing @hing3074]

@vamsibatchuk what???

[Julian Ng-Thow-Hing @hing3074]

@osanseviero my dm

[Julian Ng-Thow-Hing @hing3074]

@ToddDongb9xl @andres_pinot real man

[Julian Ng-Thow-Hing @hing3074]

@ToddDongb9xl nice man

[Julian Ng-Thow-Hing @hing3074]

@ToddDongb9xl ?

[Julian Ng-Thow-Hing @hing3074]

@southyrnn how did you make this?

[Julian Ng-Thow-Hing @hing3074]

@southyrnn veary cool!

[Julian Ng-Thow-Hing @hing3074]

@southyrnn very cool!

[Julian Ng-Thow-Hing @hing3074]

@southyrnn 😁