Music with RNN: myth or reality?


Last week I shared my excitement about my involvement in NTR’s new work on neural networks/RNN and promised to share what I learn and backstories  about the project itself.

Remember when I told you about my “other job as front woman for Vkhore? Well, like most bands, compose a lot of our own music.


I know from my own experience how hard that is — composing isn’t some off-the-shelf hobby.

But what if people with no musical training (formal or not) and no technical skills could use a computer application, choose the style of music to generate and listen to the results right then and there?

Sounds more like science fiction, but so do a lot of AI projects.

On a more personal level, training our own RNN to compose music means I’ve been getting a crash course about Machine Learning.

Over the past week I’ve been reading up on recent developments in Machine Learning in general and, more specifically, neural networks composing music.


There are already a number of them in existence. There is Magenta from Google; a tutorial that allows people  to generate music with a recurrent neural network. But it’s a simple model without stellar musical results.

What I wanted to know is if an RNN is can actually learn to compose music that has well-defined parts, i.e., the structure of music: verses, choruses, bridges, codas, etc.

Based on my research, there’s already been a good deal of development to make that happen. Originally, music generation was mainly focused on creating a single melody. You might  be interested in the discussions on Hacker News and Reddit about a year ago.  More recently, work on polyphonic music modeling, centered around time series probability density estimation, has met with partial success.

NTR’s goal is to build a generative model using a deep neural network architecture that will create music with  both harmony and melody.

We want our RNN to be able to create music that is as close to music composed by humans as possible.

I asked my colleague and friend Natasha Kazachenko, who is responsible for training our neural network to generate music, several questions to better understand exactly what we are doing. (It’s much easier to learn about a highly technical subject when you work in tech with good friends who are patient enough to explain stuff to a non-techie.)  I will share her answers next week.

I learned long ago that it is normal human psychology to attribute human traits, emotions, even gender, to non-human entities and techies (yes, they are human) are no different.

NTR Lab’s neural network is female.

Her name is Isadei.

Leave a Reply

Your email address will not be published. Required fields are marked *