The bird does not yet move on its own after installing the base game from the introduction tutorial. We have to press the spacebar to let it fly. In this tutorial, we are going to change that. After this tutorial, the bird will fly using the neural network we described in a previous tutorial.
However, to get the bird flying by itself, we need to make some steps in this tutorial. First, we need to create the neural network. The neural network will have its own class. Hereafter, the code for the game will be adapted, allowing the bird to use the neural network to fly.
For every part of the code, you will first need to think about what it should look like, based on a small assignment. We will provide hints which can help you solve the assignments. After that you can look at the answer to see the correct code corresponding to the assignment.
To let the bird fly based on the neural network, we will first create the neural network. As a starting point, you need to create a new python file called nnet.py. In this file, you will import the following libraries and create a neural network class.
# Import packages
import numpy as np
import scipy.special
import random
# Import all defined values of Flappy Bird variables.
from defs import *
# Set a seed to generate always the same flying pattern result
np.random.seed(16) # seed 16 will result in a good bird after 8 iterations
# Create a neural network class
class Nnet:
In this class, there will be three functions. One function is aimed at initializing the neural network, while the second function processes the inputs to usable outputs. Lastly, we have a function which retrieves the maximum output from all possible outputs.
For the first function you will need to initialize the neural network by defining the elements. Before the contents of the function are written the starting conditions should be defined. Think about what the initialization function would need as input to be able to initialize the neural network.
def __init__(self, num_input, num_hidden, num_output):
""" Initialization function for the neural network
:param num_input: integer, number of input nodes
:param num_hidden: integer, number of hidden nodes
:param num_output: integer, number of output nodes
"""
Now the contents of the function can be written. Start by writing a piece of code to initialize the number of input, hidden and output nodes.
# Save the number of input, hidden, and output nodes as a property of the class
self.num_input = num_input
self.num_hidden = num_hidden
self.num_output = num_output
Now the initialization function should generate random weights between -0.5 and 0.5 for alle connections. Write a piece of code which generates these weights.
# Generate random weights between -0.5 and 0.5 for all connections
self.weight_input_hidden = np.random.uniform(-0.5, 0.5, size=(self.num_hidden, self.num_input))
self.weight_hidden_output = np.random.uniform(-0.5, 0.5, size=(self.num_output, self.num_hidden))
Finally you will need to write a piece of code to create the activation function. We will use a lambda[1] for this, but there are also other solutions like a small function.
# Define the sigmoid (expit) activation function
self.activation_function = lambda x: scipy.special.expit(x)
After doing all the assignments in the previous tabs and putting the code together the following function has been written for the initialization of the neural network.
def __init__(self, num_input, num_hidden, num_output):
""" Initialization function for the neural network
:param num_input: integer, number of input nodes
:param num_hidden: integer, number of hidden nodes
:param num_output: integer, number of output nodes
"""
# Save the number of input, hidden, and output nodes as a property of the class
self.num_input = num_input
self.num_hidden = num_hidden
self.num_output = num_output
# Generate random weights between -0.5 and 0.5 for all connections
self.weight_input_hidden = np.random.uniform(-0.5, 0.5, size=(self.num_hidden, self.num_input))
self.weight_hidden_output = np.random.uniform(-0.5, 0.5, size=(self.num_output, self.num_hidden))
# Define the sigmoid (expit) activation function
self.activation_function = lambda x: scipy.special.expit(x)
For the next part, you need to create a function which will transform the input values to output values.
Create the function and define the starting conditions of the function. You will receive the input list as a list for this function.
def get_outputs(self, inputs_list):
""" Function to transform input values to final output values of the neural network
:param inputs_list: list, containing floats as input values for the neural network
:return:
final_outputs: array, two-dimensional containing float numbers representing neural network output values
"""
Before you will be able to use the input it needs to be transposed[2] into a two-dimensional array. Write a piece of code which will create a two-dimensional array from the input list.
# Turn the one dimensional input list in a two dimensional array to use it
inputs = np.array(inputs_list, ndmin=2).T # .T is a NumPy function to transpose an array/matrix
In the next part of the function the input values need to be converted into the hidden values. This can be done by multiplying them with the correct weights and then use the activation function on the values. Write a piece of code that will convert the input values into hidden values.
# Multiply the input values by the weights between the input and hidden layer to get the hidden input values
hidden_inputs = np.dot(self.weight_input_hidden, inputs)
# Use the activation function to transform the hidden input values
hidden_outputs = self.activation_function(hidden_inputs)
After converting the input values into hidden values, the next step is to convert the hidden values into output values. Write a piece of code that does this in the same way as described in the previous assignment. And then to finish the function the array of output values will be returned.
# Multiply the hidden output values by the weights between the hidden and output layer to get the final input values of the output nodes
final_inputs = np.dot(self.weight_hidden_output, hidden_outputs)
# use the activation function to transform the input values of the output nodes to final output values
final_outputs = self.activation_function(final_inputs)
# Return the two dimensional array of final output values
return final_outputs
After doing all the assignments in the previous tabs and putting the code together the following function has been written for processing inputs to outputs in the neural network.
def get_outputs(self, inputs_list):
""" Function to transform input values to final output values of the neural network
:param inputs_list: list, containing floats as input values for the neural network
:return:
final_outputs: array, two-dimensional containing float numbers representing neural network output values
"""
# Turn the one dimensional input list in a two dimensional array to use it
inputs = np.array(inputs_list, ndmin=2).T
# Multiply the input values by the weights between the input and hidden layer to get the hidden input values
hidden_inputs = np.dot(self.weight_input_hidden, inputs)
# Use the activation function to transform the hidden input values
hidden_outputs = self.activation_function(hidden_inputs)
# Multiply the hidden output values by the weights between the hidden and output layer to get the final input values of the output nodes
final_inputs = np.dot(self.weight_hidden_output, hidden_outputs)
# use the activation function to transform the input values of the output nodes to final output values
final_outputs = self.activation_function(final_inputs)
# Return the two dimensional array of final output values
return final_outputs
In the previous python assignments, we got an array with final outputs. In the next function we will use the array to get the maximum output value.
Create the function and define the starting conditions of the function. You will receive the input list as a list for this function.
def get_max_value(self, inputs_list):
""" Function to call the highest output value of the neural network output array
:param inputs_list: list, containing floats as input values for the neural network
:return:
np.max(outputs): float, highest output value of the neural network
"""
To get the maximum output, we first need to call the function we created in the previous python assignments, get_outputs. We then need to return the maximum output from this array to decide if the bird will need to flap or not. Write a piece of code that does this task.
# Call the function to get a final output array of the neural network
outputs = self.get_outputs(inputs_list)
# Get the highest value from the array and return this
return np.max(outputs)
After doing all the assignments in the previous tabs and putting the code together the following function has been written for getting the maximum value of the output in the neural network.
def get_max_value(self, inputs_list):
""" Function to call the highest output value of the neural network output array
:param inputs_list: list, containing floats as input values for the neural network
:return:
np.max(outputs): float, highest output value of the neural network
"""
# Call the function to get a final output array of the neural network
outputs = self.get_outputs(inputs_list)
# Get the highest value from the array and return this
return np.max(outputs)
Here the code for the game itself will be adapted to allow the bird to fly by using the neural network we just created. The assignments and code pieces are divided into three categories: standard values, input, and output.
In this first part, you will define some standard values that the AI will use.
Define the following values in the defs.py file
#How many birds are spawned each generation
GENERATION_SIZE = 60
#Number of input, hidden and output nodes
NNET_INPUTS = 2
NNET_HIDDEN = 5
NNET_OUTPUTS = 1
# Set the 'flap value'; flap when the value is higher or equal to 0.5
JUMP_CHANCE = 0.5
# Define the highest and lowest y-position for the bird to fly through the pipes
MAX_Y_DIFF = DISPLAY_H - PIPE_MIN - PIPE_GAP_SIZE / 2
MIN_Y_DIFF = PIPE_GAP_SIZE / 2 - PIPE_MAX
# Transform the lowest y-position to an absolute number
Y_SHIFT = abs(MIN_Y_DIFF)
# Define the maximal shift in distance
NORMALIZER = abs(MIN_Y_DIFF) + MAX_Y_DIFF
In this part, you will determine the input for the bird. By correctly doing this, the bird will “know” where the next gap between the pipes will be.
Before the bird can use the neural network, it should be imported into the bird.py file and it should be initialized.
At the top of the bird.py file put the following line:
from nnet import Nnet
In the __init__ function of the bird.py file put the following line:
# Initialize the neural network class with the given parameters
self.nnet = Nnet(NNET_INPUTS, NNET_HIDDEN, NNET_OUTPUTS)
To get the inputs a function should be created. Create this function and define the starting conditions.
def get_inputs(self, pipes):
""" Function to get the input values for the first layer, input layer
:param pipes: list, containing sets of pipe objects
:return:
inputs: list, containing floats as input values for the neural network
"""
Next you will need to initialize the values of the y-position of the pipe and the x-position of the pipe. Write a piece of code that will initialize those values.
# Initialize the closest x-position of a pipe with a too big value
closest = DISPLAY_W * 2
# Initialize the y-position of a pipe
bottom_y = 0
Now the correct values for closes (the right side of the closest top pipe) and bottom_y (the y value of the bottom of the closest top pipe) should be determined from the playing field. Write a piece of code which will determine the correct values. (Tip: use a for loop to go over all pipes in the playing field)
# Check which pipe (upper or lower) is the closest
for pipe in pipes:
# If the pipe is an upper pipe, and closer then the closest pipe stored, and at the right side of the pipe image
if pipe.pipe_type == PIPE_UPPER and pipe.rect.right < closest and pipe.rect.right > self.rect.left:
# Update the closest pipe position
closest = pipe.rect.right
bottom_y = pipe.rect.bottom
Next the horizontal and vertical distance from the bird to the closest pipe should be defined. Write a piece of code to determine the distance by using the closest, bottom_y and other variables.
# Get the horizontal distance of the bird to the pipe
horizontal_distance = closest - self.rect.centerx
# Get the vertical distance of the bird to the pipe
vertical_distance = (self.rect.centery) - (bottom_y + PIPE_GAP_SIZE / 2)
Use these values created in the previous assignment to determine the inputs for the neural network. Write a piece of code to generate the input values and make sure these values are at least 0.01. After that, return the inputs.
# Define the input values
inputs = [
((horizontal_distance / DISPLAY_W) * 0.99) + 0.01,
(((vertical_distance + Y_SHIFT) / NORMALIZER) * 0.99) + 0.01
]
# Return the input values
return inputs
After doing all the assignments in the previous tabs and putting the code together the following function has been written to determine the inputs for the neural network.
def get_inputs(self, pipes):
""" Function to get the input values for the first layer, input layer
:param pipes: list, containing sets of pipe objects
:return:
inputs: list, containing floats as input values for the neural network
"""
# Initialize the closest x-position of a pipe with a too big value
closest = DISPLAY_W * 2
# Initialize the y-position of a pipe
bottom_y = 0
# Check which pipe (upper or lower) is the closest
for pipe in pipes:
# If the pipe is an upper pipe, and closer then the closest pipe stored, and at the right side of the pipe image
if pipe.pipe_type == PIPE_UPPER and pipe.rect.right < closest and pipe.rect.right > self.rect.left:
# Update the closest pipe position
closest = pipe.rect.right
bottom_y = pipe.rect.bottom
# Get the horizontal distance of the bird to the pipe
horizontal_distance = closest - self.rect.centerx
# Get the vertical distance of the bird to the pipe
vertical_distance = (self.rect.centery) - (bottom_y + PIPE_GAP_SIZE / 2)
# Define the input values
inputs = [
((horizontal_distance / DISPLAY_W) * 0.99) + 0.01,
(((vertical_distance + Y_SHIFT) / NORMALIZER) * 0.99) + 0.01
]
# Return the input values
return inputs
For an output to be generated the input should be sent into the neural network. In this part you will achieve that by using a small piece of code.
In the bird.py file, decide when the bird will jump.
The bird should jump based on what the output of the neural network is. So you should get the inputs and then let them be processed by the neural network. The output value will then decide if the bird will jump or not.
def jump(self, pipes):
""" Function to decide if the bird should flap or not
:param pipes: list, containing sets of pipe objects
"""
# Get the input values for the neural network
inputs = self.get_inputs(pipes)
# Get the maximal output value of the neural network
val = self.nnet.get_max_value(inputs)
# Decide if the bird should flap
if val > JUMP_CHANCE:
# Flap the bird when the output value is higher than the set threshold
self.speed = BIRD_START_SPEED
Each time the birds are gone, we need to refresh the whole game and make a new generation of birds. We do this in order to optimize the bird performance, so every generation will fly further through the pipes. We make a new class called BirdCollection to keep track of our living birds.
Make the new class BirdCollection and an initialization function. In this function you should define the display, the list of birds and a function for creating a new generation of birds.
class BirdCollection:
""" Object to store information of the bird collection
"""
def __init__(self, gameDisplay):
""" Initialize the bird collection class
:param gameDisplay: pygame window size
"""
# Set the display width and height
self.gameDisplay = gameDisplay
# Store created birds
self.birds = []
# When the class is initialized, an new generation of birds will be created
self.create_new_generation()
In the class you just defined, we will again create a list of birds. Next to the list of birds we will append as many new birds as the defined generation size allows.
def create_new_generation(self):
""" Function to create a new generation of birds
"""
# Start with an empty bird list
self.birds = []
# Add as many birds as the defined population size
for i in range(0, GENERATION_SIZE):
# Extend the bird list with the new birds
self.birds.append(Bird(self.gameDisplay))
Lastly, we will need to keep updating the BirdCollection class by making an update function that returns the number of alive birds.
To make this update possible a function should be created. Create this function and define the starting conditions. The time movement and the list of pipes will be used as input.
For you to know which ones are still alive, you should check their state.
def update(self, dt, pipes):
""" Function to update the number of living birds
:param dt: integer, time movement in milliseconds
:param pipes: list, containing pipe information
:return:
num_alive: integer, number of living birds
"""
Write a piece of code to check how many birds are still alive. Use a for loop to go over all the birds. Then return the number of alive birds.
To check the state of the birds you can use the BIRD_ALIVE definition.
# Start the counter of alive birds at 0
num_alive = 0
# Check every bird in the bird list
for bird in self.birds:
# Update the position of the bird by adding the movement and pipe information
bird.update(dt, pipes)
# Check if the bird is living
if bird.state == BIRD_ALIVE:
# Count the living birds
num_alive += 1
# Return the number of living birds
return num_alive
After doing all the assignments in the previous tabs and putting the code together the following function has been written to update the number of alive birds.
def update(self, dt, pipes):
""" Function to update the number of living birds
:param dt: integer, time movement in milliseconds
:param pipes: list, containing pipe information
:return:
num_alive: integer, number of living birds
"""
# Start the counter of alive birds at 0
num_alive = 0
# Check every bird in the bird list
for bird in self.birds:
# Update the position of the bird by adding the movement and pipe information
bird.update(dt, pipes)
# Check if the bird is living
if bird.state == BIRD_ALIVE:
# Count the living birds
num_alive += 1
# Return the number of living birds
return num_alive
Changing the code in the nnet file does nothing from itself. We need to change the other files a bit to integrate the new code into the already existing files. Below, there are a few more assignments to make the integration complete.
In the update function of the bird.py file, add the jump function.
self.jump(pipes)
In the main.py file, import the class BirdCollection instead of Bird.
from bird import BirdCollection
In the main.py file, change bird = Bird(gameDisplay) to include the BirdCollection class.
birds = BirdCollection(gameDisplay)
In the main.py file, add the data label for the number of alive birds. The label function should be updated accordingly.
def update_data_labels(gameDisplay, dt, game_time, num_iterations, num_alive, font):
""" Function to visualize game information while running the game
:param gameDisplay: pygame window size
:param dt: integer, time movement in milliseconds
:param game_time: float, time in milliseconds the game is running
:param num_iterations: integer, number of iterated games
:param num_alive: integer, number of living birds
:param font: integer, the size of the text
"""
# Define the position of displaying the game information
y_pos = 10
x_pos = 10
gap = 20
# Show the number of frames per second
y_pos = update_label(round(1000 / dt, 2), 'FPS', font, x_pos, y_pos + gap, gameDisplay)
# Show the game time
y_pos = update_label(round(game_time / 1000, 2), 'Game time', font, x_pos, y_pos + gap, gameDisplay)
# Show the number of iterated games
y_pos = update_label(num_iterations, 'Iterations', font, x_pos, y_pos + gap, gameDisplay)
# Show number of living birds
y_pos = update_label(num_alive, 'Alive', font, x_pos, y_pos + gap, gameDisplay)
In the main.py file, add num_alive to the labels that should be updated while the program is running.
# Update the information text while running the game
update_data_labels(gameDisplay, dt, game_time, num_iterations, num_alive, label_font)
In the main.py file, update the event for-loop. Make it such that you cannot let the bird fly anymore and make sure when a key is down, that the program shuts down.
# Get every event from the queue
for event in pygame.event.get():
# Stop running the game if an event is quited
if event.type == pygame.QUIT:
running = False
# Stop running the game when a key is pressed
elif event.type == pygame.KEYDOWN:
running = False
In the main.py file, change the bird.update to the check of the number of alive birds. This number should be saved in a variable.
num_alive = birds.update(dt, pipes.pipes)
In the main.py file, change the if bird is dead sequence. The sequence should now happen when there are no birds alive anymore. Moreover, instead of creating a new bird, the program should create a whole generation.
# Check if there are any birds left
if num_alive == 0:
# Create a new set of pipes when there is not any bird left
pipes.create_new_set()
# Restart the game time
game_time = 0
# Create a new set of birds
birds.create_new_generation()
# Update the number of iterated games
num_iterations += 1
At the end of this tutorial you have created a neural network in python and changed the code of the game to function with this neural network. The neural network does not yet learn from itself, this will be the topic of the next tutorial.
All the code which you should have at the end of this tutorial can also be downloaded from GitHub
Why and how matrices are transposed is explained in tutorial 3 about matrices. ↩︎