¶ Introduction
Hello interested person and welcome to the Serpentine (Rule based) introduction. Here you can get a taste of what being a member of Serpentine is like and what kind of activities you can expect! Serpentine is an AI association that focusses on building AI knowledge through practical experience. The main way we gather that sweet EXP is by participating in competitions! In this guide you can find one of those competitions, Pommerman to be exact, to try out and the necessary resources to build a bot! Challenge our leaderbord and try to beat the previous scores!
You will also see some guides on the basics of AI. Depending on your level of skill you might want to start with a basic bot and slowly integrate AI techniques into your workflow later. We leave the specifics to you, but try to build something that works as soon as possible and improve from there. The shorter the feedback cycle the better!
Leaderboards
| Level | People who finished this level |
|---|---|
| Setup | Martin, Manuel, Erik, Max |
| Lesson 1 | Dik |
| Lesson 2 | BramH |
| Lesson 3 | Richard |
| Lesson 4 | Bas |
| Lesson 5 | |
| Lesson 6 | BramG, Jop |
| Lesson 7 | Thymen |
| Lesson 8 |
When possible their name link to their code based.
¶ The Challenge - Pommerman
Pommerman is a Python machine learning competition based on the game Bomberman. Python is the programming language of choice for most AI developers it is a great way to start learning about AI. This introduction tries to be as comprehensive as possible and therefore will assume no previous knowledge or setup. We have written guides for two types of agents for this challenge, a classical agent, and a learning agent.
The classical agent will explain how to build a bot based on hardcoded rules. It will greatly help you in getting an understanding of the game, the gym api, and python.
The learning agent will guide you in building a DQN (Deep Q-Network) for pommerman. Which is an implementation of a Q-learning algorithm. More information is provided in the lessons, or you can read the wikipedia page about it (Q-learning) for a quick introduction.
If you have any feedback, question or remarks on this page, the lessons or any educational matter, do contact the education committee.
¶ Setup
To get you started, make sure that you have everything ready. This means that you have installed Python, Pycharm and possibly Git. You can use the following guides to help you install all programs:
This tutorial has been checked for python versions 3.7 up to and including 3.9, other versions might not work.
Setup guides
With the above programs installed you are ready to install Pommerman and work through the lessons.
¶ Lessons
¶ Classical AI Agent
These lessons will guide you in building your own classical ai agent, even if you have limited python knowledge. From experience we have seen that gaining the understanding of how a game works by building a classical agent eases the process of creating a self learning agent.
Lessons for a classical AI agent
- Lesson 1 Game explanation and moving the bot
- Lesson 2 Movement checking, type hinting and documentation
- Lesson 3 Can we get to that location
- Lesson 4 Yes?, then lets move to that location
- Lesson 5 Suicide is not an option
- Lesson 6 Lets break stuff
- Lesson 7 Clear the way
- Lesson 8 PLANNED: pip install --upgrade myself
¶ Learning AI agents
The resources section has some links to RL implementations. There are papers available.
These lessons will guide you in developing a self learning agent for pommerman from scratch. A decent understanding of python is required. It is advised to first follow one of the two Cartpole tutorials.
Lessons for a learning AI agent
- Video Cartpole tutorial The hello world of learning ai within serpentine. A 4 part tutorial on implementing a dqn for cartpole. The concepts explained in these tutorials will be used in the following lessons. By sentdex
- Blog Cartpole tutorial by Greg Surma on towardsdatascience.com
- AI 1 It is alive!
- AI 2 PLANNED: I remember this!
- AI 3 PLANNED: Did I do good?
¶ Additional Resources
¶ Tutorials
If you are not yet familiar with Python this is not a problem, since this serie is meant for people with no experience. But if you first want to get some idea of what python is or a specific reminder on a topic, these materials can help you:
Python tutorials
- Whirlwind tour through python
A jupyter notebook series about a fast-paced introduction to essential components of the Python language for researchers and developers who are already familiar with programming in another language
- Basic Python syntax
A cheat sheet that will show you the basic syntax and datatypes from python
- Python tutorial
Explains the core concepts and syntax of Python (by w3schools)
- Python programming
A free step by step introduction to Python.
¶ Diving deeper
In case you are already familiar with coding and would like to try the challenge by yourself there are some additional resources that can be useful. This includes game information, an example of Reinforcement learning and examples of Pommerman bots.
Links
- Pommerman documentation
Documentation provided by the Pommerman developers - Machine learning papers implementing Pommerman
A list of papers with Pommerman implementations using the latest machine learning approaches - Pommerman agent by Papkov
RL based pommerman agent. - Pommerman agent by Guikarist
Some RL based pommerman agents. - Pommerman agent submission to conference
Submission to the 4th Multidisciplinary Conference on Reinforcement Learning and Decision Making (2019), second place Pommerman 2018 competition. Read paper for implementation. - Reinforcement learning with Cartpole
SA simple guide with python implementation in reinforcement learning