Gautham Mysore

Head of Audio and Video AI Research @AdobeResearch

[Gautham Mysore @GauthamMysore]

Come work with us to invent the future of filmmaking! We are co-inventing models and workflows with filmmakers…you!

[Gabi C. Duncombe @gcduncombe]

Filmmakers & AI Storytellers – join a small team at Adobe pushing the boundaries of GenAI + film. Prototype new workflows, shape future tools, and tell great stories. If this excites you, we want to meet you! Tell us about yourself & why you’re the perfect fit: [email protected]

[Gautham Mysore @GauthamMysore]

It's a big day for GenAI at Adobe! Today, our video generation and automatic translation tools are publicly available. Give it a try! I'm super proud of the team and all our wonderful collaborators across Adobe for what we have built.

Oh, and if you look through the available Firefly tools, there is a sneak peak at more audio and video GenAI tools that will be coming soon :)

[Adobe @Adobe]

The ultimate in creative AI. Now with video. #AdobeFirefly Generate Video gives you unparalleled creative control and IP-friendly outputs.

Check out what's new with Firefly and get ready to create like never before. https://t.co/phnOwQj8RX

[Gautham Mysore @GauthamMysore]

Join our wonderful music generation research team! We have 2 new positions. If you are excited about building tools to unleash people’s creativity and an expert in GenAI and audio, this could be for you.

[Zachary Novack @zacknovack]

Our paper "Presto! Distilling Steps and Layers for Accelerating Music Generation" has been accepted at #ICLR2025 @iclr_conf! Check out the demo video and links below, excited to see everyone in Singapore 🇸🇬!

[Dreaming Tulpa 🥓👑 @dreamingtulpa]

adobe is cooking a new method to generate sound from vocal imitations 👀

[Justin Salamon @justin_salamon]

Really cool to see our AI for bioacoustics work in the MIT Technology Review! BirdVox was a fantastic project that beyond posing a fascinating research problem, introduced me to the world of bird watching and deepened my appreciation of the natural world
https://t.co/sY02S3f6lI

[Justin Salamon @justin_salamon]

📢 Audio AI Job opportunity at Adobe!

The Sound Design AI Group (SODA) is looking for an exceptional research engineer to join us in building the future of AI-assisted audio and video creation.

Strong ML background, GenAI experience a plus.

Details: https://t.co/IOlA7Rlwhg

[Ziyang Chen @CzyangChen]

🎥 Introducing MultiFoley, a video-aware audio generation method with multimodal controls! 🔊

We can
⌨️Make a typewriter sound like a piano 🎹
🐱Make a cat meow like a lion roars! 🦁
⏱️Perfectly time existing SFX 💥 to a video

[Justin Salamon @justin_salamon]

Hiring research interns to work on AI for Sound Design! 💥🛸🐲

Join the Sound Design AI Group (SODA) for a summer 2025 internship advancing the bleeding edge of audio AI for video with @pseetharaman @urinieto and me.

A chance to impact millions of users:
https://t.co/6QY6KchaQ8

[Gautham Mysore @GauthamMysore]

Awesome work @justin_salamon , @pseetharaman, @urinieto , and @hugggof!

[scott belsky @scottbelsky]

using your voice as an “audio sketch” to generate sound effects, part of the #ProjectSuperSonic sneak from our labs.

[Bilawal Sidhu @bilawalsidhu]

Project SuperSonic at Adobe Sneaks: From sketch to fully generated sound effects — cool audio to audio AI demo! And just the required amount of awkwardness 😂

[Gautham Mysore @GauthamMysore]

"Sound is 50 percent of the moviegoing experience" - George Lucas

[Justin Salamon @justin_salamon]

Adobe just announced Generative Extend for Premiere Pro (beta) at #AdobeMAX! Use GenAI to extend your video clip *including the audio*

@pseetharaman @urinieto and me in the Sound Design AI Group at @AdobeResearch worked on the audio part and we're so excited to see it go out!

[Gautham Mysore @GauthamMysore]

We announced generative extend at #AdobeMAX! It lets you extend a video when the camera starts rolling a little too late or cuts a little too soon, both of which are common occurrences. Along with the video, it also extends audio. Not just simple "room tone" but also complex soundscapes with all kinds of ambient sounds and sound effects.

[Adobe Video @AdobeVideo]

🚨 Live from #AdobeMAX: Generative Extend is now in #PremierePro (beta)! 🚨

Seamlessly add extra frames and audio to your footage for that perfectly timed edit. Download the beta to try it out. Learn more here: https://t.co/Jy0jAaC7aB

[Gautham Mysore @GauthamMysore]

Our creator-centric approach to generative video! Prompt coherence, control, and the peace of mind of being commercially safe. We're excited about this and think it will get your creative juices flowing.

[Adobe Video @AdobeVideo]

Say hello to the all-new Adobe Firefly Video Model! 👋 This exciting advancement allows users to generate dynamic video across @CreativeCloud, @AdobeExpCloud and @AdobeExpress – and is designed to be commercially safe.

Coming to beta later this year. https://t.co/bB5e6vDO76

[The Nobel Prize @NobelPrize]

BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Physics to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.”

[Gautham Mysore @GauthamMysore]

We have an amazing lineup of keynote speakers at ISMIR this year!

[ISMIR Conference @ISMIRConf]

🚨🎶ISMIR Keynote Speakers!🎶🚨
We're thrilled to announce the keynote speakers for ISMIR 2024!
We can't wait to hear from these incredible folks! Read more here https://t.co/9xUa0yzwxr
#ismir2024 #keynotespeakers

[Gautham Mysore @GauthamMysore]

Near instantaneous music generation! Rapidly iterate on your creative ideas at near interactive speeds. Kudos to our music generation team and UCSD collaborators!

[AK @_akhaliq]

Presto!

Distilling Steps and Layers for Accelerating Music Generation

Despite advances in diffusion-based text-to-music (TTM) methods, efficient, high-quality generation remains a challenge. We introduce Presto!, an approach to inference acceleration for score-based diffusion transformers via reducing both sampling steps and cost per step. To reduce steps, we develop a new score-based distribution matching distillation (DMD) method for the EDM-family of diffusion models, the first GAN-based distillation method for TTM. To reduce the cost per step, we develop a simple, but powerful improvement to a recent layer distillation method that improves learning via better preserving hidden state variance. Finally, we combine our step and layer distillation methods together for a dual-faceted approach. We evaluate our step and layer distillation methods independently and show each yield best-in-class performance. Our combined distillation method can generate high-quality outputs with improved diversity, accelerating our base model by 10-18x (230/435ms latency for 32 second mono/stereo 44.1kHz, 15x faster than comparable SOTA) -- the fastest high-quality TTM to our knowledge.

[Gautham Mysore @GauthamMysore]

Near instantaneous music generation! Rapidly iterate on your creative ideas at near interactive speeds. Kudos to our music generation team and UCSD collaborators!

[Zachary Novack @zacknovack]

Ultra-fast text-to-music generation w/o degrading quality?

Introducing Presto! Distilling Steps and Layers for Accelerating Music Generation

🎹: https://t.co/kTTAYKKtTU
📖: https://t.co/Newhxe6lI6
w/@__gzhu__ @CasebeerJonah @BergKirkpatrick @McAuleyLabUCSD @NicholasJBryan

🧵

[Adobe Research @AdobeResearch]

Adobe Research Principal Scientist @AaronHertzmann won the Computer Graphics Achievement award by @SIGGRAPH,one of the highest honors in the field! Learn about his new theory of perception, and his work at the intersection of art and computer graphics. https://t.co/7Tk2Sfqe5l

[Gautham Mysore @GauthamMysore]

I'm hiring for a rockstar research leader in generative video with a focus on control. This is a rare opportunity to build and lead a research group to invent the future of the field for creators, from filmmakers to social video creators.
https://t.co/bm81PKwfRf

#videoai #videogenai #cocreate