Applying RL to Gameplay
The main purpose of this project is to make a player bot that can play 1) FIFA or any relevant game like basketball, tennis, golf with a feature of action recognition. 2) this part mainly focuses on the performance of the bot. We will train and try to increase the efficiency of the bot. 3) the interface of the game is not the main focus. Algorithms are. As a part of the project arcade games like Pudge wars and chess are automated. All the work is put together for easy accessibility as an android app. 4) sports/e-sports news generation. 5) AR/VR experience.
OpenAI gym for the environment for soccer. This bot will be trained with reinforcement learning algorithms like q learning. Neural networks are also used to process the data.
Basketball and tennis environments are not provided by openAI, so we need to program it manually.
For action recognition, (this is not possible with openAI setup; need to feed in the videos for which UI will be presented in the app) we use CNN or harr cascade to train the model.
The biggest part is the DotA2 bot. This is programmed in Lua, and already existing scripts are modified.
We use AR studio or similar software.
Existing vs purposed
This project will be adding environments to openAI. It will also set new benchmark standards to the existing bot performance. Fitting in the best algorithm for action recognizer since most of the existing work is on SVM classifier. We will try to use CNN based classification or harr cascades and calculate the performance. The sequence of steps is not summarized until now. Dataset will be written by us specifically for these games and then generate sports news. Till now all these games are not exposed to AR/VR.
DotA 2 installed
Lua, go installed
OpenAI gym, universe, govnc driver
Docker with Ubuntu running
Start: 20 August 2019
End: 21 April 2020
August — learn Lua, train FIFA bot, gather FIFA, basketball, tennis game videos and respective news articles
September — action recognition, look at the code of existing openAI env, building openAI environment for new games
October — start with news generation, try bots for Pudge wars and chess
November — build DotA 2 bot
December — build DotA 2 bot
January — build DotA 2 bot
February — build DotA 2 bot
March — adding AR to all projects
April — interface to put all together
Task 1 — Environments for openAI bot
Add a folder in envs named real_games and create a py file that uses pygame... In this case, we need to import spaces and proceed with render and other functions. Pyglet is another option. But we do not follow it.
Create real_games dir
Edit these files
Register these envs by adding to the registry. from gym.envs.registration import register
Task 2 — FIFA bot
Train it using one of the continuous reinforcement learning algorithms like DQN, DDPG, A3C, NAF, TRPO, PPO, TD3, SAC. Need to learn what Monte Carlo, policy gradient methods are... We also try to use TensorFlow or PyTorch as per convenience.
Task 3 — Action recognition and news generation
We built an activity recognition system to classify actions performed by Volleyball players during a match, by leveraging views from an on-court camera and available training data for 6 classes of actions (“serve”, “smash”, “block” etc). We used two approaches for the problem: One using hand-crafted Optical Flow and HoG features classified using an SVM, and another approach using a CNN on raw frames.
Task 4 — DotA2 arcade games
Task 5 — DotA2 real bot 10 on 10
Task 6 — AR/VR implementation
I am still working on this. I will walk you through all the specifications as soon as possible. Stay tuned.