Build your first game AI tutorial with TensorFlow

This paper introduces the basic principles of deep reinforcement learning through a simple Catch game, and gives a complete implementation of TensorFlow code with Keras as the front end, which is a good choice for in-depth deep learning.

Last year, DeepMind's AlphaGo defeated World Go Champion Li Shizhen with a score of 4-1. More than 200 million viewers watched reinforce learning on the world stage. A few years ago, DeepMind created a robot that could play Atari games, causing an uproar. Since then the company was quickly acquired by Google.

Many researchers believe that reinforcement learning is the best way to create ArTIficial General Intelligence. This is an exciting area with many unsolved challenges and great potential.

Reinforcement learning may seem very challenging at first, but it is not difficult to get started. In this article, we will create a simple robot based on Keras that will allow it to play Catch games.

Catch is a very simple arcade game that you might have played as a child. The rules of the game are as follows: the fruit falls from the top of the screen, the player must grab them with a basket; the player gets one point for each fruit grab; the player loses one point for each missing fruit. The goal here is to let the computer play Catch games by itself. However, we will not use such a beautiful game interface. Instead, we'll use a simple game version to simplify the task:

When playing a Catch game, the player has to decide three possible behaviors. Players can move the basket to the left, right or left. This decision depends on the current state of the game. That is, it depends on where the fruit falls and the position of the basket. Our goal is to create a model that selects the action that leads to the highest score given the content of the game's screen.

This task can be seen as a simple classification problem. We can let game experts play this game many times and record their behavior. The model can then be trained by selecting a "correct" action similar to that of a game expert.

But this is actually not the way humans learn. Humans can learn games like Catch without guidance. This is very useful. Imagine if you want to learn something as simple as Catch every time, you have to hire a group of experts to play this game thousands of times! This must be very expensive and slow.

In reinforcement learning, the model does not train based on the marked data, but through past experiences.

Reinforcement learning is inspired by behavioral psychology. We do not provide "correct" behavior for the model, but rather reward and punish. The model accepts information about the current state of the environment (eg, a computer game screen). Then it will output an action, just like a gamepad. The environment will respond to this action and provide the next status and rewards and punishments.

Based on this, the model learns and looks for behavior that maximizes rewards.

In fact, there are many ways to do this. Let's take a look at Q-Learning. Q-Learning caused a sensation when using Q-Learning to train a computer to play Atari games. Now, Q-Learning is still a significant concept. Most modern reinforcement learning algorithms are some of the improvements in Q-Learning.