Building on last year's competition and the benchmarking events that preceded it, this event will be a forum for reinforcement learning researchers to rigorously compare the performance of their methods on a suite of challenging domains. The competition finals and workshop will take place during the Multidisciplinary Symposium on Reinforcement Learning, a part of the 2009 International Conference on Machine Learning (ICML'09) in Montreal, Canada. To encourage student participation, we will be awarding travel scholarships for student competitors.

In order to encourage even greater participation, our technical committee has been working hard to lower the bar for entry. The competition software is easy to install, and contains sample code so that you can get started with minimal effort. In addition to travel scholarships there are a number of exciting prizes for grabs. We are also including resources to make it easy for instructors to use the competition software as a part of a course on Reinforcement Learning, Machine Learning, or Artificial Intelligence.

Competition Overview

The competition structure of the competition is similar to previous years. The software contains a set of domains, and for each domain there is a sample agent to get you started. The domain consists of an environment with which your agent will interact, with the goal of maximizing the cumulative reward that it receives in a fixed number of steps.

Since the goal of the competition is to compare learning agents, we have put in place certain mechanisms to ensure that the final agents must be able to learn in order to win. This means that the environments that we provide you for training your agents are slightly different than the environments on which the agents will be tested. The optimal policy for a training environment will likely be far from optimal in the testing environment, and agents that are robust to variations in the environment will perform better. As each domain is fairly different, the mechanisms for altering the domains between training and testing will vary. In general, the dynamics will have parameters that are drawn from a secret distribution. For training, we will provide a set of sampled parameters, and then a different set of samples will be used for testing.

The competition is broken up into three phases. In the first phases, competitors build their agents, testing and training them on their local systems using the environments that are provided with the competition software. Next is a proving phase, which can be thought of as the regular season, in which competitors can test their agents a limited number of times, on a new set of environments. The proving phase results will be public, so that competitors can get an idea of how their agents stack up compared to the competition. In the final phase, each competitor will have a limited number of runs on the final competition environments. The results will be collected, kept secret, and then presented during the workshop at the Multidisciplinary Symposium on Reinforcement Learning at ICML'09.

Domains

This year's competition features the following domains:

• Acrobot: The goal is to balance a multi-jointed virtual gymnast.
• Adversarial Tetris: Last year's entries pushed the state of the art and we are excited to see how this years competitors stack up.
• Helicopter: Based on the helicopter simulator from Andrew Ng's group, the goal is to keep a helicopter hovering at stationarity.
• Infinite Mario: Based on everyone's favourite side-scrolling adventure game, the goal is to guide our hero through a complex level featuring enemies and traps.
• Octopus Arm: The goal is to guide the tip of a twisty, flexible arm to a specific location as quickly as possible.
• Polyathlon: Designed to encourage general-purpose algorithms, this domain features a set of randomly generated MDPs.

Check out the domains page for more information on each of these events.

Get Started!

Participating in the RL Competition has never been easier. Whether you are a course instructor looking for a fun, challenging project for your students, a researcher looking to try out your newest algorithm, or a hobbyist, the RL Competition is the event for you. Get started now by downloading the software, learning about the domains, and participating in the discussion forums.