Baselines: Version of gym for a2c

[ahsteven]

Today I was able to get the baslines installed with gym and run some examples. From other posts, for installing mujoco-pi the following combination had to be used:

python 3.5.2
mujoco 131
mujoco_py 0.5.7
gym 0.9.3

This combination works for some things however my goal is to train a2c on atari games so when I run
(gym-py35) teves@teves:~$ python -m baselines.a2c.run_atari

I get the error:

Traceback (most recent call last):
File "/home/teves/anaconda3/envs/gym-py35/lib/python3.5/runpy.py", line 184, in _run_module_as_main
"__main__", mod_spec)
File "/home/teves/anaconda3/envs/gym-py35/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/teves/baselines/baselines/a2c/run_atari.py", line 4, in
from baselines.common.cmd_util import make_atari_env, atari_arg_parser
File "/home/teves/baselines/baselines/common/cmd_util.py", line 7, in
from gym.wrappers import FlattenDictWrapper
ImportError: cannot import name 'FlattenDictWrapper'

This seems like because I am running an older gym version. Is that correct? What versions of the above shoule I have installed to use a2c?

update: I was able to upgrade to gym 0.10.3 and was able to install mujoco using pip install mujoco-py==0.5.7 however I was not able to install it using: pip install -e '.[mujoco]'.

Then I was able to run a2c example but not acktr.

(gym) teves@teves:~$ python -m baselines.acktr.run_atari

WARNING:tensorflow:From /home/teves/baselines/baselines/a2c/utils.py:13: calling reduce_max (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From /home/teves/baselines/baselines/a2c/utils.py:15: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
Traceback (most recent call last):
  File "/home/teves/anaconda3/envs/gym/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/teves/anaconda3/envs/gym/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/teves/baselines/baselines/acktr/run_atari.py", line 21, in <module>
    main()
  File "/home/teves/baselines/baselines/acktr/run_atari.py", line 18, in main
    train(args.env, num_timesteps=args.num_timesteps, seed=args.seed, num_cpu=32)
  File "/home/teves/baselines/baselines/acktr/run_atari.py", line 12, in train
    learn(policy_fn, env, seed, total_timesteps=int(num_timesteps * 1.1), nprocs=num_cpu)
  File "/home/teves/baselines/baselines/acktr/acktr_disc.py", line 125, in learn
    model = make_model()
  File "/home/teves/baselines/baselines/acktr/acktr_disc.py", line 120, in <lambda>
    lrschedule=lrschedule)
  File "/home/teves/baselines/baselines/acktr/acktr_disc.py", line 64, in __init__
    update_stats_op = optim.compute_and_apply_stats(joint_fisher_loss, var_list=params)
  File "/home/teves/baselines/baselines/acktr/kfac.py", line 288, in compute_and_apply_stats
    stats = self.compute_stats(loss_sampled, var_list=varlist)
  File "/home/teves/baselines/baselines/acktr/kfac.py", line 298, in compute_stats
    factors = self.getFactors(gs, varlist)
  File "/home/teves/baselines/baselines/acktr/kfac.py", line 130, in getFactors
    factors = searchFactors(t, graph)
  File "/home/teves/baselines/baselines/acktr/kfac.py", line 120, in searchFactors
    if len(bTensor.get_shape()) > 0 and bTensor.get_shape()[0].value == None:
  File "/home/teves/anaconda3/envs/gym/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 484, in __len__
    raise ValueError("Cannot take the length of Shape with unknown rank.")
ValueError: Cannot take the length of Shape with unknown rank.

[ahsteven]

I was able to update to 10.3 and get A2C to work. However, ACKTR still is not working. After running:

python -m baselines.acktr.run_atari

I get:

Traceback (most recent call last):
  File "/home/teves/anaconda3/envs/gym/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/teves/anaconda3/envs/gym/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/teves/baselines/baselines/acktr/run_atari.py", line 21, in <module>
    main()
  File "/home/teves/baselines/baselines/acktr/run_atari.py", line 18, in main
    train(args.env, num_timesteps=args.num_timesteps, seed=args.seed, num_cpu=32)
  File "/home/teves/baselines/baselines/acktr/run_atari.py", line 12, in train
    learn(policy_fn, env, seed, total_timesteps=int(num_timesteps * 1.1), nprocs=num_cpu)
  File "/home/teves/baselines/baselines/acktr/acktr_disc.py", line 125, in learn
    model = make_model()
  File "/home/teves/baselines/baselines/acktr/acktr_disc.py", line 120, in <lambda>
    lrschedule=lrschedule)
  File "/home/teves/baselines/baselines/acktr/acktr_disc.py", line 64, in __init__
    update_stats_op = optim.compute_and_apply_stats(joint_fisher_loss, var_list=params)
  File "/home/teves/baselines/baselines/acktr/kfac.py", line 288, in compute_and_apply_stats
    stats = self.compute_stats(loss_sampled, var_list=varlist)
  File "/home/teves/baselines/baselines/acktr/kfac.py", line 298, in compute_stats
    factors = self.getFactors(gs, varlist)
  File "/home/teves/baselines/baselines/acktr/kfac.py", line 130, in getFactors
    factors = searchFactors(t, graph)
  File "/home/teves/baselines/baselines/acktr/kfac.py", line 120, in searchFactors
    if len(bTensor.get_shape()) > 0 and bTensor.get_shape()[0].value == None:
  File "/home/teves/anaconda3/envs/gym/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 484, in __len__
    raise ValueError("Cannot take the length of Shape with unknown rank.")
ValueError: Cannot take the length of Shape with unknown rank.

If this is working for anybody else please let me know your setup.

[DMTSource]

Has anyone been able to get around this issue for acktr?

[windyyc]

I add comment in kfac.py at lines 120 to avoid this error, but I don't know it whether or not brings some problems.
the file baselines/acktr/kfac.py

[zacwellmer]

upgrade your version of gym(I think 0.9.6 is latest verified at the time of writing this)

pip install git+git://github.com/openai/gym==0.9.6 --upgrade

[NishanthVAnand]

I am still facing this issue. Can anyone tell me how to fix this