By Arthur Chionh
4 Dataroots colleagues, no professional music production experience, a heap of Artificial Intelligence (AI)-generated samples of music and lyrics. How did all these end up in the Song of The Machines? In this final instalment of our blogpost series on Beatroots and the 2022 AI Song Contest, we dive into the world of digital music production with AI.
Digital Audio Workstations (DAW)
Digital Audio Workstations (DAW) are software used for music production. Maybe you’ve heard of DAWs like ‘Audacity’ and ‘GarageBand’! These are just a few in a wide variety of DAWs. DAWs can vary in terms of functional complexity, open-source, supported operating systems and plugins, among other aspects. Sufficiently functional DAWs let you edit music at the finest granularity of beats and pitch, much like how Photoshop lets you play with pixels. Such editing power works well with AI-generated music samples, which can be too simplistic at times:
‘Simplistic’ AI-generated music sample: one pitch per beat:
‘Simplistic’ AI-generated music sample, but edited using basic chord structure:
With AI-generated music samples and lyrics ready, the Beatroots team needed a DAW to:
- Edit and fine-tune our music samples
- Provide suitable instruments and audio effects for our music samples
- Mix everything and master our final submission
- Sprinkle AI magic every step of the way ✨
Despite the variety of DAWs and being spoilt for choice, we ultimately chose to work with Ableton Live (for simplicity we’ll refer to it as ‘Ableton’). It became an easy decision to make when we AI-loving humanoids discovered that Ableton supported the Magenta Studio plugin!
Ableton and Magenta Studio
Magenta Studio lets you work seamlessly with AI in Ableton. With a few clicks, music samples can be generated using AI, then seamlessly edited or mixed using Ableton’s DAW functionality. Visit the official Magenta Studio website for a thorough step-by-step on how to use the Magenta Studio plugin in Ableton, and details of its under-the-hood AI.
2 Magenta Studio tools were especially interesting for us - ‘Generate’ and ‘Drumify’.
‘Generate’ uses Variational Auto Encoders (VAE) to create 4-bar music samples of a melody or drums. It is very impressive in creating music without input, and allowing us to immediately edit it in Ableton without export-import inconvenience. However, we preferred relying on our music generation tools DoriVAE and CocoZoë that could produce harmonized trio samples (i.e. melody, bass and drums) instead of only melody or drums. In this case we had to choose creative output over convenience.
‘Drumify’ took our breaths away and we were able to use it for our final product. Drums give a good beat and rhythm to a song. Listen to the significant difference drums can make:
Melody without drums:
Melody with drums, fully-generated by ‘Drumify’:
Given an input melody, AI’s ability to generate perfect drums is clearly a huge accomplishment for AI in music. Without a doubt, Magenta Studio’s ‘Drumify’ was a great AI-ddition to our tools and final song!
Ableton - AI vs Humanoid Battleground
As participants in the AI Song Contest, the Beatroots team naturally wanted to use as much AI as possible. But we also realized that our musicality (or sense of music) as humanoids was irreplaceable. It still takes a (talented) humanoid to tweak an off-pitch note, time a lyric well (prosody), adjust the volumes of multiple music tracks and so on…and finally package everything into a song that appeals to the musicality of other humanoids.
How much should humanoids change in AI-generated music/lyrics? Stay true to what the AI generated, or listen to our inner humanoid musicality? Ableton was the battleground for our constant tug-of-war between AI and humanoids. Too many decisions were made to document them all! But these are some aspects of music production where AI can’t do what humanoids can (for now):
- ‘Cherry-picking’ music samples and lyrics
- Bridging or transitions between different parts of a song
- Mixing and applying audio effects e.g. adjusting individual track volumes so they sound good together, pitching or auto-tuning voices, starting and stopping individual tracks at the right time
Project ideas for any wannabe music AI developer 💡
Conclusion
Among many DAW options, we chose Ableton for its support of Magenta Studio AI tools. Editing, adjusting and mixing AI-generated music and lyrics in Ableton was often an AI versus humanoid showdown - how much should we change, while ensuring our song was as AI-generated as possible, and still truly a Song of the MACHINES?
All in all, we thoroughly enjoyed the music production process - experimenting with Ableton’s functionalities and watching our AI-generated music and lyrics come together. We hope you’ve enjoyed this behind-the-scenes blogpost series about Beatroots, and are as hyped as us for more AI developments (in music and in general)! Stay tuned and see you in our next dataroots blogpost!
You might also like
dataroots
dataroots