Imagine being able to experiment with new compositions as a professional musician without having to play a single note on an instrument. Or a small business owner easily adding music to their most recent Instagram video advertisement. Meta’s most recent AI tool, AudioCraft, which creates high-quality, realistic audio and music from text, makes that promise.
The solution fills a gap in the market for generative AI, where audio production has traditionally lagged behind. Although there has already been considerable progress in this area, the corporation notes that the current solutions are quite difficult to use, not very open, and difficult to access for experimentation.
AudioCraft consists of three models: MusicGen, AudioGen, and EnCodec. MusicGen, which was trained with Meta-owned and specifically licensed music, generates music from text prompts, while AudioGen, which was trained in public sound effects, generates audio from text prompts.
“Today, we’re excited to release an improved version of our EnCodec decoder, which allows higher quality music generation with fewer artifacts. We’re also releasing our pre-trained AudioGen models, which let you generate environmental sounds and sound effects like a dog barking, cars honking, or footsteps on a wooden floor. And lastly, we’re sharing all of the AudioCraft model weights and code,” Meta said on its website.
According to Meta, it offers users "the full recipe to play with the existing models" that Meta has been building over the last years, simplifying the overall creation of generative models for audio compared to earlier work in the field.
Compared to models created for photos, video, and text, the new tools aim to address the difficulties in the field of AI audio production. Meta noted that it has been difficult to simulate the complexity of audio signals and patterns at various scales, notably for music.
“Music is arguably the most challenging type of audio to generate as it’s composed of local and long-range patterns, from a suite of notes to a global musical structure with multiple instruments,” the company said.
Related: Meta Buys 3D Smart Glass Maker Luxexcel
According to Meta, the user-friendly design of AudioCraft enables artists and makers to get ideas, brainstorm, and iterate on songs in creative ways.
Meta envisions that MusicGen could potentially evolve into a new type of instrument, akin to the impact synthesizers had when they first emerged.
Meta's AudioCraft is anticipated to compete with Google's MusicLM, a program that was released to the public in May and can likewise produce high-fidelity music from text input and humming.
