Google’s MusicLM turns text into music, will the AI system strike a chord with listeners?

There has been a lot of buzz around OpenAI’s ChatGPT since it was unveiled later last year and its rival by Google called Bard.

Meanwhile other tech companies, including those in China, are also catching up.

However, Google’s latest AI system called MusicLM can generate music in any genre with a text description. Moreover, it can even transform a whistled or hummed melody into other instruments.

Also read: ChatGPT, Bard & Ernie: The three musketeers of AI

The tech giant has recently released a research paper titled MusicLM: Generating Music From Text.

Discover the stories of your interest

Although MusicLM certainly isn’t the first generative AI system for songs, the research paper says that it can outperform other systems in terms of its “quality and adherence to the caption”.The company has also uploaded a bunch of samples that it produced using the model.

However, the company has no immediate plans to release it, fearing its misuse, according to the research paper. “We have no plans to release models at this point,” concludes the paper, citing risks of “potential misappropriation of creative content.”

Also read: ETtech Explainer: Big Tech battle it out for AI control

What are its features?

According to the research paper, MusicLM was trained on a dataset of 280,000 hours of music to produce songs that make sense for complex descriptions.

MusicLM samples include five-minute pieces produced from only one or two words like melodic techno, as well as 30-second samples that sound like entire songs and are formed from paragraph-long descriptions.

MusicLM is also capable of transforming a collection of sequentially written descriptions into a musical story or narrative built on existing melodies.

It can also be instructed via a combination of picture and caption, or generate audio that’s played by a specific type of instrument in a certain genre.

What are its limitations?

The researchers said MusicLM produces high-quality music at 24 kHz, “consistent over several minutes, while being faithful to the text conditioning signal.”

Google researchers have also published an AI training dataset of 5,500 pieces of music to support other researchers working on automated song generation.

However, like other AI systems, MusicLM has its own limitations. While it can technically generate vocals, including choral harmonies, the music samples lack clarity.

According to the paper, the researchers found that the model misunderstands negations and does not adhere to precise temporal ordering described in the text.

They further added that future work may focus on lyrics generation, along with improvement of text conditioning and vocal quality. Another aspect is the modeling of high-level song structure like introduction, verse, and chorus.

Threats

The research paper also highlighted that AI system like MusicLM pose many ethical challenges, including a tendency to “incorporate copyrighted material from training data into the generated songs.”

During an experiment, they found that about 1% of the music the system generated was directly replicated from the songs on which it trained.