Dezzan007 @dezzan007 - Twitter Profile

Google presents MusicRL Aligning Music Generation to Human Preferences paper page: https://t.co/FL4jDRdXpi propose MusicRL, the first music generation system finetuned from human feedback. Appreciation of text-to-music models is particularly subjective since the concept of musicality as well as the specific intention behind a caption are user-dependent (e.g. a caption such as "upbeat work-out music" can map to a retro guitar solo or a techno pop beat). Not only this makes supervised training of such models challenging, but it also calls for integrating continuous human feedback in their post-deployment finetuning. MusicRL is a pretrained autoregressive MusicLM (Agostinelli et al., 2023) model of discrete audio tokens finetuned with reinforcement learning to maximise sequence-level rewards. We design reward functions related specifically to text-adherence and audio quality with the help from selected raters, and use those to finetune MusicLM into MusicRL-R. We deploy MusicLM to users and collect a substantial dataset comprising 300,000 pairwise preferences. Using Reinforcement Learning from Human Feedback (RLHF), we train MusicRL-U, the first text-to-music model that incorporates human feedback at scale. Human evaluations show that both MusicRL-R and MusicRL-U are preferred to the baseline. Ultimately, MusicRL-RU combines the two approaches and results in the best model according to human raters. Ablation studies shed light on the musical attributes influencing human preferences, indicating that text adherence and quality only account for a part of it. This underscores the prevalence of subjectivity in musical appreciation and calls for further involvement of human listeners in the finetuning of music generation models.

_akhaliq's tweet photo. Google presents MusicRL

Aligning Music Generation to Human Preferences

paper page: https://t.co/FL4jDRdXpi

propose MusicRL, the first music generation system finetuned from human feedback. Appreciation of text-to-music models is particularly subjective since the concept of musicality as well as the specific intention behind a caption are user-dependent (e.g. a caption such as "upbeat work-out music" can map to a retro guitar solo or a techno pop beat). Not only this makes supervised training of such models challenging, but it also calls for integrating continuous human feedback in their post-deployment finetuning. MusicRL is a pretrained autoregressive MusicLM (Agostinelli et al., 2023) model of discrete audio tokens finetuned with reinforcement learning to maximise sequence-level rewards. We design reward functions related specifically to text-adherence and audio quality with the help from selected raters, and use those to finetune MusicLM into MusicRL-R. We deploy MusicLM to users and collect a substantial dataset comprising 300,000 pairwise preferences. Using Reinforcement Learning from Human Feedback (RLHF), we train MusicRL-U, the first text-to-music model that incorporates human feedback at scale. Human evaluations show that both MusicRL-R and MusicRL-U are preferred to the baseline. Ultimately, MusicRL-RU combines the two approaches and results in the best model according to human raters. Ablation studies shed light on the musical attributes influencing human preferences, indicating that text adherence and quality only account for a part of it. This underscores the prevalence of subjectivity in musical appreciation and calls for further involvement of human listeners in the finetuning of music generation models.

0

274

50

111

41K

Dezzan007 @Dezzan007

almost 3 years ago

I also believe that the lack of portability of AR glasses might hinder their widespread adoption. However, in the home setting, they still have a great opportunity to offer a more immersive experience than devices like smartphones.

0

20

Dezzan007 @Dezzan007

almost 3 years ago

Since the usage scenario of AR glasses is primarily centered around the home, I believe that companies like Apple and Meta will leverage AR glasses more in the home setting, such as in scenarios like home office, audio-visual entertainment, and interactive online gaming. In these

1

0

102

Dezzan007 @Dezzan007

almost 3 years ago

three scenarios, I believe the main reason the advantages of AR have not been fully utilized is primarily due to the clarity of the display and the software ecosystem. I believe that these two issues will see significant improvements within the next one or two years.

1

0

30

Dezzan007 retweeted

Amir

@virtual_amir

almost 3 years ago · Minneapolis

Text to 3D is getting warmer. ➡️ @LumaLabsAI Genie text input ➡️ Animated with @Adobe Mixamo ➡️ AR with @the8thwall ➡️ Meta Quest 3 Passthrough AR via web browser Next up just need to connect it to @OpenAI and you have yourself a conversation with Albert. Education system is about to change. 🚀🚀🚀 #madewithgenie #ar @MetaQuestVR #education #ai