retro-contest-sonic

A student implementation of the World Models paper with documentation.

Ongoing project.

TODO

CURRENTLY DOING

DONE

• β-VAE for the Visual model
• MDN-LSTM for the Memory model
• CMA-ES for the Controller model
• Training pipelines for the 3 models
• Human recordings to generate data
• MongoDB to store data
• LSTM and VAE trained "successfully"
• Multiprocessing of the evaluation of a set of parameters given by the CMA-ES
• Submit learnt agents

LONG TERM PLAN ?

• Cleaner code, more optimized and documented
• Game agnostic
• Continue training / testing better architectures
• Online training instead of using a database

How to launch the scripts

• Install the modules in the requirements.txt, pytorch 0.4 and mongoDB
• Buy or find the ROMs of Sonic The Hedgehog and install them with retro-gym.

Once you've done that, you will need to train the 3 components : python train_vae.py python train_lstm.py --folder=xxx python traincontroller.py --folder=xxx where xxx is the folder number created in savedmodels/ While training the VAE and the LSTM, pictures will be saved in a folder results/ Once you're done, you can use your best trained controller to play a random level using : python play_best --folder=xxx Dont forget to change the RENDER_TICK in const.py to 1, so you can see what's happening.

Resources

• My write-up on the code and concepts of this repository
• World Models paper
• Coded on Ubuntu 16.04, Python 3.5, PyTorch 0.4 with GPUs (some change have to be made in order to make it fully compatible to CPU as well, such as adding a map_location="cpu" when loading the model)

Differences with the official paper

• No temperature
• No flipping of the loss sign during training (to encourage exploration)
• β-VAE instead of VAE