Meta has announced the open-sourcing of AudioCraft, a suite of generative AI tools for creating Music and Audio from text prompts. Content creators can now use simple text descriptions to generate complex audio landscapes, compose melodies, and simulate virtual orchestras.
AudioCraft comprises three core components. AudioGen enables the generation of various audio effects and soundscapes, while MusicGen creates musical compositions and melodies based on descriptions. EnCodec, a neural network-based audio compression codec, has recently been improved to generate higher quality music with fewer artifacts.
The AudioGen tool can create a variety of audio sound effects such as barking dogs, honking car horns, and footsteps on a wooden floor. Meanwhile, MusicGen can produce songs of various genres from scratch based on descriptions like “Pop dance track with catchy melodies, tropical percussions, and upbeat rhythms, perfect for the beach.”
Meta provides several audio samples on its website for evaluation. While these samples demonstrate state-of-the-art capabilities, they may not be high quality enough to replace professionally produced commercial audio effects or music.
Meta acknowledges that generative AI models focused on text and images have gained significant attention, but generative audio tools have lagged behind in development. By releasing AudioCraft under the MIT License, Meta aims to contribute accessible audio and musical experimentation tools to the broader community.
However, Meta is not the only company exploring AI-powered audio and music generators. OpenAI’s Jukebox, Google’s MusicLM, and the independent research team’s Riffusion are notable attempts in this field.
Generating high-fidelity audio is a complex task that requires modeling complex signals and patterns. Music is especially challenging due to its local and long-range patterns. While symbolic representations like MIDI or piano rolls have been used, they fail to capture all the expressive and stylistic elements of music. Recent advancements in self-supervised audio representation learning and hierarchical models have made progress in audio generation.
Meta states that MusicGen was trained on 20,000 hours of music owned or licensed by the company specifically for this purpose. This move towards using ethical training material may address concerns raised by critics of generative AI models.
The open-sourcing of AudioCraft presents an opportunity for open source developers to integrate these audio models into their work. This could lead to the development of interesting and user-friendly generative audio tools in the future. The code for AudioCraft tools is available on GitHub for those with coding expertise.
The post Meta Open-Sources AudioCraft: AI Tools for Music and Audio Generation appeared first on TS2 SPACE.
