Ml-agents: Building and Training on COLAB

[taesiri]

Hi Everyone,

I'd like to share my own setup for training ml-agents on COLAB. It is a fairly simple setup but it helped me save some time. I make changes to the environment I want locally and push the changes to Github. After that, I build the environment in the COLAB and train the model I want. I should mention that installing Unity on COLAB and building a project taking 5 up to 15 minutes.

Here is the link to my COLAB notebook

I used some scripts from unity3d-gitlab-ci-example for building the Unity project. For example, I copied and modified BuildCommand.cs into my project. The modification is in the GetEnabledScenes method and it's only 1 line. I only return the name of the scene I want to get build.

The whole setup is very simple at the moment. If anyone from ml-agent team is interested, I can make a PR with an appropriate BuildCommand file and a sample notebook.

[ervteng]

Hi @taesiri, thank you for sharing! This will be useful for others for sure. @unityjeffrey perhaps we can look into linking the tutorial in some way?

[xiaomaogy]

Hi @taesiri, this looks really nice! I've tried to walk through your notebook, but on the step of "Activate Unity" I got a few error messages saying "invalid option -- o" etc, so I wasn't able to proceed.

[taesiri]

@xiaomaogy, I think you haven't installed Unity yet. Either $UNITY_EXEC is pointing to the wrong path or you've forgotten to accept the License Agreement.

After executing the above cell, you have to accept the License Agreement by pressing y in the textbox which appeared bellow the cell.

That being said, It is better to activate Unity using the second method. For some reason passing the password in the command line is problematic.

[taesiri]

Here is a video demo of the notebook

[xiaomaogy]

@taesiri Thanks for the video, I will try again. I was really confused by "Upload ulf file in colab using the left side-bar.", because I wan't able to see the Files tab and switch to it, and the video helped me find it.

[xiaomaogy]

@taesiri
I was able to go through the whole process and I really like it. I was especially impressed by the convenience of this solution, everything is laid out nicely in just one place. The tensorboard, training console and the files can all be managed in one place (if we can somehow visualize the unity, that would be even better).

Regarding the setup part, aside from a little bit confusion in the license activation part(and the video helped a lot), all of the other steps are really smooth. Nice job!

One thing I'm curious about is you mentioned that this setup helps you save time. Could you please elaborate more on this if you don't mind?

[taesiri]

@xiaomaogy
I am glad you liked it. I prepared this notebook very quickly, and surely, we can add a better explanation for each cell to reduce confusion.

I also had some thoughts about visualizing Unity's output. I am pretty sure we can add it either through some sort of xvbf hack or a minor modification in the ml-agents.

About the time-saving part, I like to modify/create different environments, and training agents on them can be time-consuming (I only have two computers). With this setup, I can use COLAB to train more agents simultaneously. The training power part aside, I think it is really convenient to push your C# code changes to the GitHub and let the scripts do the compiling job for you. (After installing Unity in one COLAB session, you can reuse it until the session expires.) Surely one can make a Linux build locally and upload the binaries in COLAB (or in any other Linux servers), but it's time consuming compared to automatic builds. (I also have bad internet connection which makes upload process painful)

[xunyiljg]

Thank you for sharing. I have also successfully used colab.

[taesiri]

@xunyiljg I'm glad you liked it 😊.

Just a quick update:

• I was able to stream Unity's graphical output to Twitch directly from a COLAB notebook. In my early attempts xvbf gave me a headache so I switched to Xorg virtual desktop. My current notebook is a mess but I will share it after I cleaned it up.
• I also wrote a script to alter a scene before building the game. It comes handy to change few things like mark a Brain as trainable in the BroadcastHub directly in COLAB.
• I saw other people have some struggle running Unity in headless server with graphical output (without --no-graphics flag). For example #1786 marked as closed without any solution or workaround. I will write a tutorial for this later.
• The choice of Twitch is totally optional. I used ffmpeg to capture the screen. You can just save the video on Google Drive instead.
• The current setup is super inefficient! I will do some benchmark to find good parameters for broadcasting.

[taesiri]

Here is the Link to my updated notebook. It works with the latest beta version of ml-agents 0.11. I should mention that I am really happy with the changes made in this version.

• Now you can build any scene from the ml-agents repository and train it on COLAB. I used the input interactive widget and let the user choose which see he or she wants to build.

• You can either train in headless (no graphics) mode or stream the output to Twitch. You can get your stream key here.

• I used i3 for managing windows as its super lightweight and easy to work with. You can interact with it through the i3ipc python package.

Happy Coding!

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 14 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had activity in the last 28 days. If this issue is still valid, please ping a maintainer. Thank you for your contributions.

[vwxyzjn]

@taesiri It doesn't work anymore. See

I think list_of_scenes needs to be adjusted

# list_of_scenes = list(Path('/content/my-agents/UnitySDK/Assets/ML-Agents/Examples/').rglob('*.unity'))
list_of_scenes = list(Path('/content/my-agents/Project/Assets/ML-Agents/Examples/').rglob('*.unity'))

Then I can choose a scene, but then when I try to build it, I have another error

I feel like there should be a guide on how to build all example envs headlessly... What if I want to train everything in the server where no GUI is available? The current guides do not seem to cover these instructions.

[taesiri]

Hi @vwxyzjn.

I do not maintain that notebook anymore as there was little to no feedback from the community. I have a version of this notebook that works with my own fork of mlagents. You are more than welcome to copy my notebook and fix the issue you mentioned. The master branch is changing rapidly; I strongly suggest to switch to one of the stable release branches.

Some notes for you:

• Upgrade the version of Unity to 2019.3, Here is the URL for latest Linux version of Unity

• You'll probably run into more issues when compiling a project, I strongly suggest to take a look at this repository. For example, this issue will help you to solve a problem related to the sound card.

• Instead of TensorBoard in Colab, Try Tensorboard.dev. (Or you can use WandB)

Cheers,
M