Stable-baselines: Issue with code from "Getting Started"

[Pqformelpro]

I just installed stable_baselines and the dependencies and tried to run the code segment from the "Getting started" section in the documentation.

But when i try to run it using Anaconda im running in an AttributeError:

runfile('C:/Users/wagne_000/Anaconda3/envs/ai_project/lib/site-packages/stable_baselines/common/mpi_adam.py', wdir='C:/Users/wagne_000/Anaconda3/envs/ai_project/lib/site-packages/stable_baselines/common')
Traceback (most recent call last):

File "", line 1, in
runfile('C:/Users/wagne_000/Anaconda3/envs/ai_project/lib/site-packages/stable_baselines/common/mpi_adam.py', wdir='C:/Users/wagne_000/Anaconda3/envs/ai_project/lib/site-packages/stable_baselines/common')

File "C:\Users\wagne_000\Anaconda3\envs\ai_projectlib\site-packages\spyder_kernels\customize\spydercustomize.py", line 704, in runfile
execfile(filename, namespace)

File "C:\Users\wagne_000\Anaconda3\envs\ai_projectlib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "C:/Users/wagne_000/Anaconda3/envs/ai_project/lib/site-packages/stable_baselines/common/mpi_adam.py", line 5, in
import stable_baselines.common.tf_util as tf_utils

File "C:\Users\wagne_000\Anaconda3\envs\ai_projectlib\site-packages\stable_baselines__init__.py", line 7, in
from stable_baselines.ddpg import DDPG

File "C:\Users\wagne_000\Anaconda3\envs\ai_projectlib\site-packages\stable_baselines\ddpg__init__.py", line 1, in
from stable_baselines.ddpg.ddpg import DDPG

File "C:\Users\wagne_000\Anaconda3\envs\ai_projectlib\site-packages\stable_baselines\ddpg\ddpg.py", line 16, in
from stable_baselines.common.mpi_adam import MpiAdam

File "C:\Users\wagne_000\Anaconda3\envs\ai_projectlib\site-packages\stable_baselines\common\mpi_adam.py", line 5, in
import stable_baselines.common.tf_util as tf_utils

AttributeError: module 'stable_baselines' has no attribute 'common'

[araffin]

Hello,
It seems that you are running stable-baselines on a windows system... I'm afraid I cannot really help as I don't have any machine running on windows (and that continuous integration tests are using linux systems).
Did you installed all the required dependencies (not only the python ones)? Including openmpi?

I think @GBanlik and @lhorus installed it on windows (see https://github.com/hill-a/stable-baselines/issues/40), so I would ask them how they did it.

(Two possible alternative solutions: use google colab or use docker images)

[Pqformelpro]

I just now got it running without problems on my desktop pc by just installing everything normally.
I used the mpi from microsoft and executing the getting started code worked first try. The problem mentioned above is on my laptop and both run windows 10, so probably not a windows problem.

Im going to look into the problem on the laptop in more detail tomorrow and report back.

[Pqformelpro]

Double checked all needed dependencies and they were all installed and up-to-date on my Laptop. I created a new environment and setup everything again and its works fine, no problems at all. The only thing I remember what might have caused a problem was losing internet connection for about a minute while installing a package, but there were no errors and it installed fine afterwards. Dont remember which package it was though.

[Guilherme-B]

I apologize for the late reply.

Unfortunately I could not get it running, the solution, which isn't a solution at all, was indeed to use Colab, with its many limitations.

Did you successfully make it run, for instance, on a A2C learner, @Pqformelpro ?

[Pqformelpro]

I apologize for the late reply.

It's not even past 24 hours, so dont worry i got plenty of time :)

But Yeah everything is working fine for me. I am only using Anaconda and installing dependencies either through Anaconda or pip. I only get an error for atari-py when trying to install stable_baselines but I installed and hopefully resolved the problem through cloning this github https://github.com/j8lp/atari-py and installing it through pip. But as i want to use retro anyways i dont think this will be an issue if it still persists.

The code im currently running is:

import retro
import gym

from stable_baselines.common.policies import CnnPolicy
from stable_baselines.common.vec_env import DummyVecEnv
from stable_baselines import PPO2

import numpy as np

class Discretizer(gym.ActionWrapper):

    def __init__(self, env):
        super(Discretizer, self).__init__(env)

        buttons = ["B", "A", "MODE", "START", "UP", "DOWN", "LEFT", "RIGHT", "C", "Y", "X", "Z"]

        actions = [["LEFT"], ["RIGHT"], ["LEFT", "DOWN"], ["RIGHT", "DOWN"], ["DOWN"], ["DOWN", "B"], 
                         ["B"]]

        self._actions = []

        for action in actions:
            arr = np.array([False] * 12)
            for button in action:
                arr[buttons.index(button)] = True
            self._actions.append(arr)
        self.action_space = gym.spaces.Discrete(len(self._actions))

    def action(self, a):
        return self._actions[a].copy()

env = Discretizer(retro.make("SonicTheHedgehog-Genesis", "GreenHillZone.Act1"))
env = DummyVecEnv([lambda: env]) 

model = PPO2(CnnPolicy, env, verbose=1)
model.learn(total_timesteps=10000)

obs = env.reset()

while True:
    action, _states = model.predict(obs)
    obs, rewards, dones, info = env.step(action)
    env.render()

for the code i followed this tutorial
https://www.youtube.com/watch?v=PT9pQliUXDk&list=PLTWFMbPFsvz2LIR7thpuU738FcRQbR_8I

the only thing i cant get to run right now is multprocessing
but i just read something on the docs about issues when using windows and the recommendation of using docker so i will probably look into that next

[araffin]

@Pqformelpro Could you provide us with the precise steps you followed (and command lines you used) to install stable-baselines on windows? I would like to document that for the other potential users ;)

Can stable_baselines also be used for gym-retro? Exchanging gym for retro works and also renders the environment but im wondering if the algorithms will work aswell.

Yes, stable baselines can be used on any gym environment (given that the observation space and action space are supported). For retro, I would recommend pre-processing the data (cf what is done with atari: https://github.com/hill-a/stable-baselines/blob/master/stable_baselines/common/cmd_util.py#L18 and here: https://github.com/openai/baselines/blob/6e607efa905a5d5aedd8260afaecb5ad981d713c/baselines/common/cmd_util.py#L65)

[Pqformelpro]

@Pqformelpro Could you provide us with the precise steps you followed (and command lines you used) to install stable-baselines on windows? I would like to document that for the other potential users ;)

Yes sure. So far I've only got some bullet points noted on which steps I took, but as this is part of a school project and I will be documenting it further anyways I gladly share it here aswell. Expect this to be here this weekend, probably today.

Yes, stable baselines can be used on any gym environment (given that the observation space and action space are supported). For retro, I would recommend pre-processing the data (cf what is done with atari: https://github.com/hill-a/stable-baselines/blob/master/stable_baselines/common/cmd_util.py#L18 and here: https://github.com/openai/baselines/blob/6e607efa905a5d5aedd8260afaecb5ad981d713c/baselines/common/cmd_util.py#L65)

Ok, ty I will look into this.

[Pqformelpro]

Ok so here is how I setup stable_baselines on Windows.

First up I installed Anaconda and the corresponding IDE Spyder.

In the next step i created an environment, lets call it "stable_baselines", in Anaconda and chose Python 3.6 to be installed. Some packages like TensorFlow arent available on Python 3.7.

Then I installed some packages with the package manager from Anaconda Navigator.

• mpi4py
• zlip
• git (only if you dont have it in some other way already)

We can now open Anaconda Prompt and activate the created environment. In Windows this is done by typing

activate stable_baselines

Now i install gym-retro. Not sure if stable_baselines includes gym-retro, otherwise this step can be skipped.

pip install gym-retro

Next I installed atari-py as installing stable_baselines will throw an error when trying to build the wheel for it.

git clone https://github.com/j8lp/atari-py
cd atari-py
pip install -e .

To make mpi4py work we also need Microsoft MPI which you can get here:

https://www.microsoft.com/en-us/download/details.aspx?id=57467

Now everything should be setup and we can install stable_baselines. Keep in mind that this will still throw an error when building the atari-py wheel, but we can ignore that.

pip install stable_baselines

This was done on a Laptop and a desktop PC. Both of them use Windows 10. Also keep in mind that multiprocessing is not working. I tried it once and I got an error "TypeError: can't pickle GameData objects". Currently I'm happy that stable_baselines is working as it is and trying to work out while multiprocessing is not running is not on my list now.
This setup works for gym and gym-retro. I tried "CartPole-v1" and "SonicTheHedgehog-Genesis" and both ran just fine.

If anything is unclear please feel free to ask.

[araffin]

OK, thanks =)

Keep in mind that this will still throw an error when building the atari-py wheel, but we can ignore that.

If it throws an error, how did you installed it? Or was it only a warning? (Because normally when an error is thrown, it stops the whole process)

Also keep in mind that multiprocessing is not working

Yes, this was reported several times...

[Pqformelpro]

If it throws an error, how did you installed it? Or was it only a warning? (Because normally when an error is thrown, it stops the whole process)

only the installation for atari-py was stopped and rolled back, the rest of stable_baselines installed just fine and thats why I installed atari-py from this github https://github.com/j8lp/atari-py

[araffin]

I tried your instructions on my old computer (dual boot with windows 10) and it worked =)
Surprisingly, I did not have any problem with the multiprocessing...
Also, I would recommend installing stable_baselines from source to avoid the atari error (that made my installation failed the first time).
I will add the instructions to the repo soon ;)

[araffin]

done ;) (instructions are now in the doc)

[trinhvv]

Ok so here is how I setup stable_baselines on Windows.

First up I installed Anaconda and the corresponding IDE Spyder.

In the next step i created an environment, lets call it "stable_baselines", in Anaconda and chose Python 3.6 to be installed. Some packages like TensorFlow arent available on Python 3.7.

Then I installed some packages with the package manager from Anaconda Navigator.

• mpi4py
• zlip
• git (only if you dont have it in some other way already)

We can now open Anaconda Prompt and activate the created environment. In Windows this is done by typing

activate stable_baselines

Now i install gym-retro. Not sure if stable_baselines includes gym-retro, otherwise this step can be skipped.

pip install gym-retro

Next I installed atari-py as installing stable_baselines will throw an error when trying to build the wheel for it.

git clone https://github.com/j8lp/atari-py
cd atari-py
pip install -e .

To make mpi4py work we also need Microsoft MPI which you can get here:

https://www.microsoft.com/en-us/download/details.aspx?id=57467

Now everything should be setup and we can install stable_baselines. Keep in mind that this will still throw an error when building the atari-py wheel, but we can ignore that.

pip install stable_baselines

This was done on a Laptop and a desktop PC. Both of them use Windows 10. Also keep in mind that multiprocessing is not working. I tried it once and I got an error "TypeError: can't pickle GameData objects". Currently I'm happy that stable_baselines is working as it is and trying to work out while multiprocessing is not running is not on my list now.
This setup works for gym and gym-retro. I tried "CartPole-v1" and "SonicTheHedgehog-Genesis" and both ran just fine.

If anything is unclear please feel free to ask.

You save my day!