Music with RNN: myth or reality?

 

Last week I shared my excitement about my involvement in NTR’s new work on neural networks/RNN and promised to share what I learn and backstories  about the project itself.

Remember when I told you about my “other job as front woman for Vkhore? Well, like most bands, compose a lot of our own music.

музыка

I know from my own experience how hard that is — composing isn’t some off-the-shelf hobby.

But what if people with no musical training (formal or not) and no technical skills could use a computer application, choose the style of music to generate and listen to the results right then and there?

Sounds more like science fiction, but so do a lot of AI projects.

On a more personal level, training our own RNN to compose music means I’ve been getting a crash course about Machine Learning.

Over the past week I’ve been reading up on recent developments in Machine Learning in general and, more specifically, neural networks composing music.

 

There are already a number of them in existence. There is Magenta from Google; a tutorial that allows people  to generate music with a recurrent neural network. But it’s a simple model without stellar musical results.

What I wanted to know is if an RNN is can actually learn to compose music that has well-defined parts, i.e., the structure of music: verses, choruses, bridges, codas, etc.

Based on my research, there’s already been a good deal of development to make that happen. Originally, music generation was mainly focused on creating a single melody. You might  be interested in the discussions on Hacker News and Reddit about a year ago.  More recently, work on polyphonic music modeling, centered around time series probability density estimation, has met with partial success.

NTR’s goal is to build a generative model using a deep neural network architecture that will create music with  both harmony and melody.

We want our RNN to be able to create music that is as close to music composed by humans as possible.

I asked my colleague and friend Natasha Kazachenko, who is responsible for training our neural network to generate music, several questions to better understand exactly what we are doing. (It’s much easier to learn about a highly technical subject when you work in tech with good friends who are patient enough to explain stuff to a non-techie.)  I will share her answers next week.

I learned long ago that it is normal human psychology to attribute human traits, emotions, even gender, to non-human entities and techies (yes, they are human) are no different.

NTR Lab’s neural network is female.

Her name is Isadei.

inceptionizm gallary

What do you know about neural networks?

As some of you know, I started my career as a professional journalist, so when my boss asked me if I thought I could distinguish the original text of a contemporary Russian poet from the text generated by artificial intelligence I said yes. I tried it on a test and scored 8 out of 10 correctly.

Score one for us humans, but I have to say that it got me curious about the AI and how it works. One thing I learned is that there are many examples of machine generated creative writing on the internet, such as CuratedA.

Curiosity + journalist = research, so I started reading. But remember. I am not a techie, although I work for a very cool tech company, so even though I wrote doesn’t mean I really understand. But I’m learning!

First, I learned that artificial neural networks are mathematical or computational models inspired by the structure and functioning of the biological neural networks that are found in the nervous system of living organisms. Dr. Robert Hecht-Nielsen, inventor of one of the first neurocomputers, defines a neural network as “…a computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs.”

Next, I learned about recurrent neural networks (RNNs). They are a class of artificial neural network architecture inspired by the cyclical connectivity of neurons in the brain that uses iterative function loops to store information.

RNNs have several properties that make them an attractive choice for sequence labeling. Because they can learn what to store and what to ignore, they are perfect to use for context information they also have several drawbacks that have limited their application to real-world sequence labeling problems.

In plain language, this means that while they accept many different types and representations of data and can recognize sequential patterns in the presence of sequential distortions they are a long way from being perfect.

That said, there are already many real-world situations where RNNs are being used, such as time series analysis, natural language processing, speech recognition and Medical Event Detection.

Okay, so we have an idea about what RNNs are, why they are exciting and how they work. But the most exciting to me was to find out there were already RNNs doing art or there are people figured out how to train RNN character-level language models.

There is the algorithm developed at the University of Tubingen in Germany, which can extract the style from one image (say a painting by Van Gogh), and apply it to the content of a different image or Google’s inceptionism technique that transforms images.

dreams of google neural network

(Google, esperimento "Inceptionism")
(Google, esperimento “Inceptionism”)

Neural networks have operated in visual arts as a creative mechanism and in the the study of human aesthetic preferences, but, most exciting to me as a musician, artificial neural networks have been used to actually generate music.

I learned that neural networks are fundamental to artificial life and that the entities created can become interactive art pieces or even create the art itself.

These lifelike forms rely on generative models, which are a rapidly advancing area of research. As these models improve and the training and datasets scale, we can expect samples that depict entirely plausible images or videos.

Deus ex machina!, I thought when I read that the net had begun to sound human — like Her, instead of Siri.  Of course, if your background is technical you are probably more interested in the way this stuff actually works.

As I said at the start, I’m not techie and reading through this stuff sometimes makes my head hurt, so why am I excited?  Because I work for a company that has started training our own recurrent neural network and I’m actually involved in the project!

I will tell you more next week.

Let’s talk about “good leader”!

 

The last few weeks I’ve been reading many articles about leadership and sharing them and my ideas with you.

Leadership and culture within companies are a relatively new focus among Russian business and I find them fascinating. My friend Miki writes about them for her company and sometimes we have very interesting conversations.

We had one the other night and I think you will find it interesting, too. I’ve italicized Miki’s words and left mine in plain type, so it won’t be confusing.

ленин уорхолмао уорхол

Andy Warhol’s  Red Lenin and Mao

Do people there talk much about leadership?

Yes, we talk about that, but ideas about company culture and its importance are fairly new. Culture was ignored because we thought it was less important for a company then R&D, selling or production. I think Russians never got that it all goes together.

What is your impression of the stuff you have read from the US?

Sometimes I think you guys talk about culture too much and too meticulously. Russians mostly don’t like to talk about culture and leadership. We aren’t used to talking about culture because we are
just starting to recognize its importance.

What do you personally think of the idea of leadership?

Hmm, that’s complicated :) I think that it’s rocket science to be the perfect leader
and to build the right culture in a company. I think it’s mostly depends on your personal
human qualities, but doesn’t depend on economic situation and on the country where you work.

Do you think a person can learn leadership better in school or from doing it and analyzing after?

If you are a strong person with a strategic and analytic mind you still need to learn how to communicate with people, how to take responsibility and how to build the company’s culture in order to be a leader.

You could analyze the experience of great leaders and start to do stuff to get your own experience and analyze that. Mistakes are not bad, as long as you notice, acknowledge and analyze them. Also, it’s good to accept advice from someone more experienced than you, as long as their values are similar to yours.

I think I’m writing very obvious things)) I don’t have any real experience.

Do you think there is such a thing as a “perfect leader”?

I think it’s not about being strong and forcing people; it’s about to be smart enough to be different and flexible if it’s needed.

Leading is getting people to do what you want them to do willingly, but so is manipulation. I think it’s defined as negative, because it’s always used that way. Do you think a good leader must be a good person?

Yes; I don’t think a good leader forces at all, I think he gets the ability to inspire people to follow him. A really good leader is smart enough to listen employee’s opinion and find compromises. He is not strict, he is flexible, he is not egoistic, he always can change his mind if finds out someone’s opinion is more reasonable.

My own opinion is that the actions come and are described as leadership after the fact.

I agree with that. There is a quote from Spinoza (I think) that freedom is a necessity only after you have had it. I believe that a good leader is a flexible person who is able to inspire some people to follow and finds ways to convince the rest without using force.

I agree that culture is a reflection of a leader’s values, but nothing you’ve said has changed my mind. I just don’t believe that leadership is inherently good, “…history tells us that people are more than willing to be shaped in socially unacceptable directions—think Jeff Skilling or Hitler. Of course, if the leader crashes and burns, his constituents will claim that they were led astray unwillingly and/or had no idea what their leader was up to.”

When you or anyone talks about a “good leader” they are referring to someone whose values they share. Founders bring their values, which become the basis of the company culture, but that doesn’t mean the culture is good. I have seen horrible cultures in very successful companies. What is the culture at NTR?

Good, there is a company culture with values which are similar to mine.

And what are your leaders like?

I like Nick’s style. He works hard to build good culture in our company.

So what kind of leader is he? What makes him good?

He is smart and he inspires people and never forces stuff in a bad way (not too strict or manipulating); employees have opportunities to think differently. I always have a chance to argue something he says and defend my own points.

You have asked me, now it is my turn. How would you describe a good leader.

That’s easy; my ideal of a leader was described by Lao Tzu way back in the 6th Century BC. You can read it here.

I would love to continue this discussion by hearing what you think about leadership. What is your “good leader” definition?  And any other thoughts you have. Miki and I promise to respond.

Bad News Everyone

You can’t lead wearing rose-colored glasses.

That means talking about bad along with the good.

We live in world of crisis. And whether people are looking globally, or just locally, they would need to be six feet under not to have noticed.

Here are a few examples, in case you’ve been traveling off-planet recently.

  •        The Russian economy has been in a deep and painful recession for the last 18 months. Our national economy is down 6% second quarter 2016 compared to 2015.
  •        Dozens of tech companies are laying off.
  •       Official unemployment stands at 4.9, but the government numbers are notoriously low, since it only counts those actively looking (millions have given up).

Bad news, be it internal, regarding products and process, or external, with regards to your markets or the general macro-economic trends, will happen. Period.

But many leaders don’t like receiving bad news, which is really stupid.

Start with internal bad news.

According to Darrell Bracken, C.E.O. of Logitech, “The most dangerous thing is to be sitting in an office and nobody’s telling you what’s wrong.”

If you don’t know what’s wrong it can’t be fixed.

GoodNewsEveryone

OpenTable’s CEO Christa Quarles, whose team was wasting time trying to make everything perfect before showing her, passionately believes news should be “early, often, ugly,” which allows for faster course correction.

But that only works if your company culture is built on the precept that the messenger won’t be killed, which is also the basis for innovation, especially if it disrupts anything.

Avoiding external bad news is even stupider.

Why?

First, because companies put enormous effort put into hiring the smartest people possible, so expecting them to be dumb and not notice the economy or how the company is doing is downright idiotic.

Second, because what you don’t address will give rise to rumors that are far worse than the actual news — particularly when it comes to layoffs.

This is also the time when age plays against many founders, because they have little-to-no experience communicating any kind of bad news.

good news1

While there are dozens of approaches to communicating bad news, a good place to start is with Miki’s six basic steps.

Finally, the one sure thing about bad news — you can’t make it go away by ignoring it.

But you can make it worse.

Are you a leader?

I was born in the USSR during difficult times.

As a child, watching the adults around me, I realized that to be strong you need to cultivate the leader inside you.

My parents always told me that the main thing is to be yourself. I think I was lucky because I learned early on that it was better to do whatever myself, instead of waiting for someone else to do it, which made it easier for me to exist in those hard times — fortunately they passed.  

In other words, I developed a lot of initiative and initiative is the number one requirement of being a leader.

super hero or superhero  girl power concept
image source: here

Now, when I read books about being a leader and leadership, I almost always become annoyed. As Wally Bock explains in his blog post, Leadership “wisdom” that makes me crazy

I know from my childhood that not everyone is willing/able to lead and that’s OK. I also learned that nobody can lead all the time.

Of course, I am talking about real leaders as opposed to positional leaders — not all bosses are leaders and not all leaders are bosses.

Simply put, we are all different. Being a leader is an attitude that says “action speaks louder than words” as opposed to education or a set of memorized to-do’s.

In short, real leadership is only learned by doing.

That’s why these actions that demonstrate real leadership and they are underpinned/driven by your natural human qualities of character.

But even with initiative and attitude it still takes work to become a truly authentic leader. Authentic leaders “are sensitive to the impact their words and actions have on others” and that constant awareness is honed by substantial experience no matter your role.

So, if you think leadership is your thing don’t wait to be a boss or get an MBA to do it. Show your initiative and lead any time you get the chance. People will notice and positional leadership opportunities will appear.

Vision

 

I have my second job in the music band named Wkhore.  If you are curious, you can see and follow us hereThat’s me up front singing .  As a frontwoman and a leader I’m the first who thinks about future of the band.  I have my  vision. 

fprma1

forma2

That’s  why this issue is particularly important for me.

Late last year we touched on the importance of vision and what how to turn a vision into reality.

As Eric Ries and all Lean Startup practitioners will tell you “forming a clear vision for your startup is perhaps the most important early thing an entrepreneur can do” — emphasis on clear.

Vision provides guidance to the organization by articulating not only what its product is supposed to achieve, but also sets its values and the environment in which it will happen.

Fred Wilson wrote: A big vision is critical for a big success. You have to know where you want to be in a decade or more. That’s where the long roadmap comes in. … it all starts with a plan. I know companies that are great at communicating but don’t really have a coherent plan. All the communicating in the world won’t help them.

So figure out where you are taking your company. Answer the basic question on everyone’s mind, “what are we doing?” and you will be on a path to building a loyal, hardworking, and motivated team”

But none of these skills will help if, in fact, you are pointed in the wrong direction.

That’s why founders who excel are also great listeners — and not just to those who agree with them.

The greatest danger of vision happens when it is the property of one person; one person who will do anything to sell the vision — anything except share and modify it.

That is why every founder needs to follow the lead of the Romans. When Roman generals rode through the streets for their victory parade they were required to have a person in the chariot who kept repeating “Remember, you are not a god.”

Leadership. Motivation

 

There are thousands of articles and entire books on motivation, but for this blog I always look for stuff that is clear and simple and can be used by founders and bosses at any level.

motivation1

image source 

I was introduced to Wally Bock’s blog by a colleague and have found his approach to leadership actions a good fit to my criteria.

In his post, Wally identifies in 8 basic things you need to do to keep your people motivated. They sound simple, but providing them consistently requires focus and effort from you.

Guest post from Wally Bock.

One last time: how do you motivate people?

Apr 07, 2016 03:00 pm | Wally Bock

Continue reading Leadership. Motivation

Leadership. Intro.

Hey everyone, I’m back!

Remember I said I was going on vacation? What I didn’t mention was that it was an unwired vacation.

I was high in the mountains, with friends and no internet access. We played, sang, talked and I had time alone to think and dream. As Miki said in her post, it truly was magical. Take a look!

алтай 2016 1алтай 2016

I came back refreshed and recharged and ready for all the new tasks and projects my boss gave me. Now I can see that my company needs me and I truly make a difference.

Leaving the constant noise gave me time to think and one of the things I found myself thinking about was how entwined leadership and management need to be for a startup to be truly successful.

I agree with Miki that Warren Bennis’ ideas are out of date — if they were ever in. They don’t play well when combined with today’s knowledge workers.

I wondered if “experts” thought that, too, and decided to provide you with solid information to help you become a stronger entrepreneur and a better boss at whatever level.

So the upcoming new series is about the leadership and management. Not just high-level, but how-to’s on specific actions you need to perform. We’ll cover only one topic a week, so you will be able to save and use the series as a reference to help you through the challenges you will face.

Some of the topics we’ll cover are delegation, the pros and cons of ego, visions, culture and structure sans bureaucracy.

What I really hope is that you will write me and suggest the specific topics you want covered or issues you are struggling with. (Easy and safe to ask, because you will stay anonymous.) Write me at ykazantseva@sinergo.ru.

I also hope that you will take a few minutes to add your approaches/solutions when the topic is one you have faced.

I’m not a chatbot

 

My friend in Facebook turned out to be a bot 

I felt just as I would chatting  with a human — but it was a chatbot.

I suddenly remembered Spike Jones’s  futuristic love story Her. It is an exploration into the possibilities of human connection in an environment of disconnectedness and into our evolving definition of what’s real and what’s artificial.  

Are chatbots dangerous? Not yet.

They are artificial intelligence software that can fool a human as Cleverbot and Eugene Goostman (computer program) did.

It was couple of years ago and after a chatbot passed the Turing test that they became a hot topic. With all the hype around AI, it’s sometimes difficult to separate fact from fiction for a generation brought up on science fiction films and stories about the interaction of people and machines.

HER

I’ll give it my best try.

Chatbots exist (live?) inside messaging apps like Facebook (M), Apple (Siri), Google (WeChat) and Slack and can do almost everything in virtual world: order your tacos, schedule your meetings, call Uber, even protest your parking tickets (London and NYC for now).

Do they think? No!

They  are just programs that simulate human conversation. To understand that there is no magic and machines are still controlled by humans even If they still seem creepy, review some of the Deep Learning techniques used to build conversational agents.

There is a wave of recent startups working to create new conversation systems between consumers and services. They build consumer apps like Operator or x.ai; bot platforms like Chatfuel and bot libraries, such as Howdy’s Botkit.

Do people prefer to talk to bots instead of human? Sometimes.

Many people, especially those under 35, don’t like calling businesses. In fact, they prefer to avoid conversations whenever possible —  but not all.

In other words, lazy is the new smart and bots are better than friends.

That said, companies that buy into the belief that all their customers, or even a large majority, don’t want good, well-trained, responsive, human customer service are in for a rude awakening. On the other hand, very few companies offer that now.

How many are there? More than you think.

There are certainly plenty of them out there and the number will just keep increasing.

Chatbots have actually existed since the beginning of the internet. But they’re coming back in force now, because there are more channels to use them (various messaging apps) and you can do more stuff with them.

The current jump started last year when large companies began to hint at all the things they’re going to do with the new bots And the biggest “kick-off” arguably happened during the Facebook F8 conference in April 2016.

Do we need to get ready for the chatbot revolution?  

Yes. They are the future of customer engagement, but will not totally replace humans — at least yet. Personalized customer service is harder to create with chatbots and this has created a large resistance to their growing popularity, especially from those like my friend Miki, who has bad hearing and only calls with uncommon problems. The kind that are rarely included in automated menus.

We aren’t to the point of the Rise of the Machines, although they seem smarter and more ironic than some humans.

Clev: What is your aim in life?

To rule the world.

Clev: Then mine is to rule you.

One more thing I want to share with you. 

Starting tomorrow I’m on vacation, but only from NTR.

I’m spending the next two weeks touring with my band (my night job:)

If you are curious, you can see and follow us here. After all, music is music, even if you don’t understand the lyrics. Right? And, again, if you are curious, that’s me up front singing. If I played an instrument no one would listen!

Have lots of summer fun and I’ll be back on August 5th.

Image source: here

Win a T-shirt: Share the changes you have seen

As a recent thirty-something who spent her career as a journalist before joining NTR tech is new to me. that means I don’t have the historical references that help make sense of current happenings.

But journalists are great at asking questions, so I asked a few people what changes they’ve seen in tech. And I’d like to hear about the changes you have seen. Share them in comments to enter the drawing for an NTR T-shirt.

футболкафутболка1

 Dean Wiltse, CEO at FoodLogiQ

A lot happened on the US economy. 2 periods of market collapse and in 2001 was financial global boom. Dot.com and then 2008 the World Economy Crisis. So in between and before it was really isolated artificial economy and business itself had really focused on survivor. I think those of us who have managed business through those periods realised that we always be like there is no tomorrow.

The benefit is the education level has changed and the perception of business has changed. The people of the US realised that education outside is that good or better in many cases especially in technology. And we realised that outside Us we can also find people who appreciate a good job and work ethic. And we started to use outsourcing

I think any time past 2008 is good time for starting business. Now is more difficult to start a new company with a new idea. It’s always about business model

It is said that the only thing that you can always count on is that things will change; this is especially true in the startup world.

Martin Zwilling

Founder & CEO, Startup Professionals, Inc.

I’m seeing a real resurgence of entrepreneurial spirit, and more startup activity than ever before since the recent recession.

In my perspective, the primary change has been a huge decline in the “cost of entry” to be an entrepreneur. For example, 25 years ago, it would cost up to $1 million for software development and hardware to establish a new e-commerce web site. Now, due to powerful tools and templates, you can set up a new e-commerce site for a few hundred dollars or less.

Jeremy Boudinet

Director of Marketing at Ambition

  1. For entrepreneurs, access to venture capital has greatly increased.
  2. Technology has enabled the democratization of entrepreneurship – it’s much easier and cheaper to launch a startup now than it was 25 years ago.
  3.  There’s a true startup ecosystem – tons more experienced talent, networking and mentorship groups, etc. Also make it easier.

Bottom line: It’s 1000x easier to launch a startup now than it was 25 years ago.

With that said, it’s still incredibly, insanely difficult and competitive, but the barrier to entry is as low as it’s ever been.

Nick Mikhaylovsky,

Co-founder/President, NTR Lab

The word “startup” did not exist in Russia before 2006. We used the word “business” to speak about startups. In 2006 other entrepreneurs brought the word “startup” from the Valley.  A full understanding that this is a special type of businesses came in 2012. In 2006 was also heard that there are venture capital firms. It was all like a myth: someone somewhere gives you a lot of money for your business.

There are 3 major categories in which startups have changed. They are market, technology and people. In the market came the realization of what a startup is and the knowledge of the infrastructure with accelerators and venture firms, that has spread its their effect all over the world. The technology changed a lot, especially with the advent of the cloud; development costs plummeted, which meant it was far easier to do a software startup.

15 years ago, most startups failed  because they could not work out what they wanted to develop. Now technology in general has evolved greatly.

The high level of technology in general has led to the fact that startups don’t fail because they are unable to develop the technology. There are no startups failing due to technology. The most  frequent  reason now is that they are unable to develop and support the business model.

Start-ups that are founded in hard times, such as now, are more likely to be successful. Because, first of all, there is less competition and, secondly, in any downturn large corporations are investing in technology companies. Now is not the crisis; now is the correction

Be sure to share the changes you’ve seen below for a chance to win an NTR T-shirt.