OpenAI + DOTA2: 180 Years of Learning Per Day

OpenAI + DOTA2: 180 Years of Learning Per Day

December 12, 2019 100 By Stanley Isaacs


Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. You know that I am always excited to tell
you about news where AI players manage to beat humans at more and more complex games. Today we are going to talk about DOTA 2 which
is a multiplayer online battle arena game with a huge cult following and world championship
events with a prize pool of over 40 million dollars. This is not just some game, and just to demonstrate
how competitive it is and how quickly it is growing, last time we talked about this in
Two Minute Papers episode 180 where an AI beat some of the best players of the game
in a limited 1 versus 1 setting, and the prize pool was 20 million dollars back then. This was a huge milestone as this game requires
long-term strategic planning, has incomplete information and a high-dimensional continuous
action space, which is a classical nightmare situation for any AI. Then, the next milestone was set to defeat
a human team in the full 5 versus 5 game, and I promised to report back when there is
something new on this project. So here we go. If you look through the forums and our YouTube
comments, it is generally believed that this is so complex that it would never ever happen. I would agree that the search space is indeed
stupendously large and the problem is notoriously difficult, but whoever thinks this will never
be solved has clearly not been watching enough Two Minute Papers. Now, you better hold on to your papers right
away, because this video dropped 10 months ago, in August 2017, and since then, the AI
has played 180 years worth of gameplay against itself every single day.80% of these games it played against itself,
and 20% against its past self, and even though five of these bots are supposed to work together
as a team, there is no explicit communication channel between them. And now, it is ready to play 5 versus 5 matches. Some limitations still apply, but since then,
the AI was able get a firm understanding of the importance of teamfighting, predicting
the outcome of future actions and encounters, ganking, or in other words, ambushing unsuspecting
opponents and many other important pieces of the game. The May 15th version of the AI was evenly
matched against OpenAI’s in-house team, which is a formidable result, and I find it really
amusing that these scientists were beaten by their own algorithm. This is, however, not a world class DOTA 2
team. And the crazy part is that the next version
of the AI was tested three weeks later, and it not only beat the in-house team easily,
but also defeated several other teams and a semi-professional team as well. As it is often incorrectly said on several
forums that these algorithms defeat humans because they can click faster, so I will note
that that these bots perform about 150-170 actions per minute, which is approximately
in line with an intermediate human player, and it is also to be noted that DOTA2 is not
that sensitive to this metric. More clicking does really not mean more winning
here at all. The human players were also able to train
with an earlier version of this AI. There will be an upcoming event on July 28th
where these bots will challenge a team of top players, so stay tuned for some more updates
on this! There is no paper yet, but I’ve put a link
to a blog post and the full video in the description, and it is a gold mine of information and was
such a joy to read through. So, what do you think? Who will win? And is a 5 versus 5 game in DOTA 2 more complex
than playing Starcraft 2? If you wish to hear more about this, please
consider helping us tell this story to more people and convert them into Fellow Scholars
by supporting the series through Patreon, and as always, we also accept Bitcoin, Ethereum,
and Litecoin, the addresses are in the video description. And if you are now in the mood to learn some
more about DOTA 2, I recommend taking a look at Day9’s channel, I’ve put a link to a relevant
series in the video description. Highly recommended. So there you go, a fresh Two Minute Papers
episode, that’s not two minutes and it’s not about a paper. Yet. Love it. Thanks for watching and for your generous
support, and I’ll see you next time!