Gym: TypeError: 'NoneType' object is not iterable

[boyliwensheng]

when I run in pycharm with this code.
import gym
env= gym.make('CartPole-v0')
env.reset()
env.render()

I got a error :TypeError: 'NoneType' object is not iterable. Can you tell me what's wrong with it

[mpSchrader]

Can you give more information, like version, full trace?

[lucko515]

Hello, I have the same error.

Here is the trace:

WARN: gym.spaces.Box autodetected dtype as <class 'numpy.float32'>. Please provide explicit dtype.
Exception ignored in: <bound method Viewer.__del__ of <gym.envs.classic_control.rendering.Viewer object at 0x7fa46c038400>>
Traceback (most recent call last):
  File "/home/luka/Documents/rl/gym/gym/envs/classic_control/rendering.py", line 143, in __del__
  File "/home/luka/Documents/rl/gym/gym/envs/classic_control/rendering.py", line 62, in close
  File "/home/luka/anaconda3/envs/rl/lib/python3.5/site-packages/pyglet/window/xlib/__init__.py", line 480, in close
  File "/home/luka/anaconda3/envs/rl/lib/python3.5/site-packages/pyglet/gl/xlib.py", line 345, in destroy
  File "/home/luka/anaconda3/envs/rl/lib/python3.5/site-packages/pyglet/gl/base.py", line 334, in destroy
  File "/home/luka/anaconda3/envs/rl/lib/python3.5/site-packages/pyglet/gl/xlib.py", line 335, in detach
  File "/home/luka/anaconda3/envs/rl/lib/python3.5/site-packages/pyglet/gl/lib.py", line 97, in errcheck
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 954, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 887, in _find_spec
TypeError: 'NoneType' object is not iterable

[Magic-wei]

I had the same error.

• my OS is ubuntu 16.04 LTS, and
• I install gym from source, master branch

I tested the codes like this:

#! /usr/bin/env python3
# -*- coding = utf-8 -*-

import gym
env = gym.make('CartPole-v0')
for i_episode in range(20):
    observation = env.reset()
    for t in range(100):
        env.render()
        print(observation)
        action = env.action_space.sample()
        observation, reward, done, info = env.step(action)
        if done:
            print("Episode finished after {} timesteps".format(t+1))
            break

And the trace is here ~ (I show some info before this error, it maybe helpful to figure out the problem)

[-0.04731956 -0.03730637 -0.0084249  -0.02162919]
[-0.04806568  0.15793539 -0.00885748 -0.31695832]
[-0.04490698  0.35318237 -0.01519665 -0.61242138]
[-0.03784333  0.15827606 -0.02744508 -0.32456331]
[-0.03467781  0.35377783 -0.03393634 -0.62577336]
[-0.02760225  0.15914563 -0.04645181 -0.34396854]
[-0.02441934 -0.03528578 -0.05333118 -0.06628795]
[-0.02512505  0.16055863 -0.05465694 -0.37530881]
[-0.02191388  0.3564126  -0.06216312 -0.68471221]
[-0.01478563  0.55234003 -0.07585736 -0.99629988]
[-0.00373883  0.35830996 -0.09578336 -0.72837245]
[ 0.00342737  0.55461632 -0.11035081 -1.04959954]
[ 0.0145197  0.7510154 -0.1313428 -1.3747836]
[ 0.02954     0.55775656 -0.15883847 -1.12589602]
[ 0.04069514  0.75456248 -0.18135639 -1.46389111]
Episode finished after 15 timesteps
[ 0.01058637  0.02918534  0.00631525  0.01827018]
[ 0.01117008  0.22421616  0.00668065 -0.27241354]
[ 0.0156544   0.41924215  0.00123238 -0.5629819 ]
[ 0.02403924  0.22410292 -0.01002725 -0.26991096]
[ 0.0285213   0.02912549 -0.01542547  0.01959252]
[ 0.02910381  0.22446522 -0.01503362 -0.27791713]
[ 0.03359312  0.41979838 -0.02059197 -0.57530348]
[ 0.04198908  0.22497106 -0.03209803 -0.28917806]
[ 0.04648851  0.42053567 -0.0378816  -0.59180908]
[ 0.05489922  0.61616691 -0.04971778 -0.89617994]
[ 0.06722256  0.8119264  -0.06764138 -1.20406733]
[ 0.08346108  1.00785437 -0.09172272 -1.51715862]
[ 0.10361817  1.20395833 -0.1220659  -1.83700673]
[ 0.12769734  1.40019978 -0.15880603 -2.16497693]
[ 0.15570133  1.20694772 -0.20210557 -1.92523942]
Episode finished after 15 timesteps
Exception ignored in: <bound method Viewer.__del__ of <gym.envs.classic_control.rendering.Viewer object at 0x7ff378b69048>>
Traceback (most recent call last):
  File "/home/wangwei/github/OpenAI/gym/gym/envs/classic_control/rendering.py", line 143, in __del__
  File "/home/wangwei/github/OpenAI/gym/gym/envs/classic_control/rendering.py", line 62, in close
  File "/usr/local/lib/python3.5/dist-packages/pyglet/window/xlib/__init__.py", line 480, in close
  File "/usr/local/lib/python3.5/dist-packages/pyglet/gl/xlib.py", line 345, in destroy
  File "/usr/local/lib/python3.5/dist-packages/pyglet/gl/base.py", line 334, in destroy
  File "/usr/local/lib/python3.5/dist-packages/pyglet/gl/xlib.py", line 335, in detach
  File "/usr/local/lib/python3.5/dist-packages/pyglet/gl/lib.py", line 97, in errcheck
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 954, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 887, in _find_spec
TypeError: 'NoneType' object is not iterable

[Magic-wei]

@boyliwensheng maybe it's better to add label _help wanted_ to this issue.

[sunjeet95]

Even I am getting the same exact error.

[Magic-wei]

@mpSchrader More info has been given here. Please help us if you have time. Thx.

[mpSchrader]

Hi guys,

For me it looks like it is a problem when Python is shutting down and an environment has been render previously.
My observation is that everything is working well as long as Python is not shut down. When I run the code as a script I get the error right away. On the other hand when I run the code in the Python shell, I don't get the code right away. The error occurs as soon as I exit the shell.

Overall the problem only occurs when you render the environment. From my point of view this is just a rendering problem.

I will try to find the bug. Nevertheless you should be good to go develop your own code and ignore the error for now!

Best,
Max

Sorry for the delay. I have been on vacation! :wink:

[mpSchrader]

Update:
The problem does not occur when you probperly close the env bevor shutting down Python.
Here the proper code:

import gym
env = gym.make('CartPole-v0')
for i_episode in range(20):
    observation = env.reset()
    for t in range(100):
        env.render()
        print(observation)
        action = env.action_space.sample()
        observation, reward, done, info = env.step(action)
        if done:
            print("Episode finished after {} timesteps".format(t+1))
            print((i_episode+1), "/", 20)
            break
env.close()

[mpSchrader]

Please confirm that this solves your problem! @boyliwensheng
If so please close the issue! :smile:

[Magic-wei]

@mpSchrader It works! Thank you for help!

[pglombardo]

This solved the issue for me as well. Thanks.

[HightMike]

Thank you

[YMMS]

Hi all, I still get the similar error. (My OS is Ubuntu 18.04)

My Code is as following:

# -*- coding:utf-8 -*-

import gym
from gym import wrappers, logger

logger.set_level(logger.INFO)
env = gym.make('CartPole-v0')
outdir = './random-agent-results'
env = wrappers.Monitor(env, directory=outdir, force=True)
for i_episode in range(20):
    observation = env.reset()
    for t in range(100):
        env.render()
        print(observation)
        action = env.action_space.sample()
        observation, reward, done, info = env.step(action)
        if done:
            print("Episode finished after {} timesteps".format(t+1))
            print((i_episode+1), "/", 20)
            break
env.close()

The error info is as following:

[ 0.00290468  0.1968751  -0.19102299 -1.04031627]
Episode finished after 42 timesteps
20 / 20
INFO: Finished writing results. You can upload them to the scoreboard via gym.upload('/home/yangming/eclipse-workspace/rl-gym/CartPole/random-agent-results')
Exception ignored in: <bound method Viewer.__del__ of <gym.envs.classic_control.rendering.Viewer object at 0x7f6b3a504198>>
Traceback (most recent call last):
  File "/home/yangming/Temp/gym/gym/envs/classic_control/rendering.py", line 143, in __del__
  File "/home/yangming/Temp/gym/gym/envs/classic_control/rendering.py", line 62, in close
  File "/home/yangming/.pyenv/versions/gym/lib/python3.5/site-packages/pyglet/window/xlib/__init__.py", line 480, in close
  File "/home/yangming/.pyenv/versions/gym/lib/python3.5/site-packages/pyglet/gl/xlib.py", line 345, in destroy
  File "/home/yangming/.pyenv/versions/gym/lib/python3.5/site-packages/pyglet/gl/base.py", line 334, in destroy
  File "/home/yangming/.pyenv/versions/gym/lib/python3.5/site-packages/pyglet/gl/xlib.py", line 335, in detach
  File "/home/yangming/.pyenv/versions/gym/lib/python3.5/site-packages/pyglet/gl/lib.py", line 97, in errcheck
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 954, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 887, in _find_spec
TypeError: 'NoneType' object is not iterable

When I try the code https://github.com/openai/gym/blob/master/examples/agents/random_agent.py (it has env.close()).

It still got similar error as below:

INFO: Starting new video recorder writing to /tmp/random-agent-results/openaigym.video.0.25721.video000008.mp4
INFO: Starting new video recorder writing to /tmp/random-agent-results/openaigym.video.0.25721.video000027.mp4
INFO: Starting new video recorder writing to /tmp/random-agent-results/openaigym.video.0.25721.video000064.mp4
INFO: Finished writing results. You can upload them to the scoreboard via gym.upload('/tmp/random-agent-results')
Exception ignored in: <bound method Viewer.__del__ of <gym.envs.classic_control.rendering.Viewer object at 0x7f1a786e4ac8>>
Traceback (most recent call last):
  File "/home/yangming/Temp/gym/gym/envs/classic_control/rendering.py", line 143, in __del__
  File "/home/yangming/Temp/gym/gym/envs/classic_control/rendering.py", line 62, in close
  File "/home/yangming/.pyenv/versions/gym/lib/python3.5/site-packages/pyglet/window/xlib/__init__.py", line 480, in close
  File "/home/yangming/.pyenv/versions/gym/lib/python3.5/site-packages/pyglet/gl/xlib.py", line 345, in destroy
  File "/home/yangming/.pyenv/versions/gym/lib/python3.5/site-packages/pyglet/gl/base.py", line 334, in destroy
  File "/home/yangming/.pyenv/versions/gym/lib/python3.5/site-packages/pyglet/gl/xlib.py", line 335, in detach
  File "/home/yangming/.pyenv/versions/gym/lib/python3.5/site-packages/pyglet/gl/lib.py", line 97, in errcheck
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 954, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 887, in _find_spec
TypeError: 'NoneType' object is not iterable

How could I avoid this problem? @mpSchrader, thank you.

[mpSchrader]

Hi @YMMS,

the problem result from the following lines:

env = gym.make('CartPole-v0')
...
env = wrappers.Monitor(env, directory=outdir, force=True)
...
env.close()

I will explain you why. First you create a regular CartPole environment, which you then use to create a wrapped environment, so you no have two environments. But in the end you only close the wrapped environment.
One solution for that could look as follows:

import gym
from gym import wrappers, logger

logger.set_level(logger.INFO)
env_to_wrap = gym.make('CartPole-v0')
outdir = './random-agent-results'
env = wrappers.Monitor(env_to_wrap, directory=outdir, force=True)
#env = wrappers.Monitor(env, force=True)
for i_episode in range(20):
    observation = env.reset()
    for t in range(100):
        env.render()
        print(observation)
        action = env.action_space.sample()
        observation, reward, done, info = env.step(action)
        if done:
            print("Episode finished after {} timesteps".format(t+1))
            print((i_episode+1), "/", 20)
            break
env.close()
env_to_wrap.close()

You should always close every environment you created.
I hope this helps! :-)

[YMMS]

@mpSchrader Thank you, it works

[bee-san]

Hey, I still get the NoneType error despite closing my environments

import gym
import numpy as np
from gym import wrappers


# instantiate a gym environment
# version 0 because that's the latest version available for this
env = gym.make('CartPole-v0')

done = False
count = 0

best_length = 0
episode_lengths = []
best_weights = np.zeros(4)
length = []

for i in range(100):
    # allows us to make an more intelligent decision about moving right or left
    # if positive move right, if negative move left
    new_weights = np.random.uniform(-1.0, 1.0, 4)
    length = []
    for j in range(100):
        # you must reset the environment before you use it
        observation = env.reset()
        done = False
        count = 0
        while not done:
            # render the environment visually
            #env.render()
            count += 1

            action = 1 if np.dot(observation, new_weights) > 0 else 0


            # we need to decide an action. either left or right
            # this api gives us a random action from its possible sample space
            # action= env.action_space.sample()

            # vector with 4 quantities, position, cart, angle of pole
            observation, reward, done, info = env.step(action)

            if done:
                break
        length.append(count)

    average_length = float(sum(length) / len(length))

    if average_length > best_length:
        best_length = average_length
        best_weights = new_weights

    episode_lengths.append(average_length)
    if i % 19 == 0:
        print('best length is ', best_length)    

done = False
count = 0
# records it
env_to_wrap = gym.make('CartPole-v0')
env = wrappers.Monitor(env_to_wrap, 'MovieFiles', force=True)
#observation = env.reset()

while not done:
    # render the environment visually
    #env.render()
    count += 1

    action = 1 if np.dot(observation, best_weights) > 0 else 0
    observation, reward, done, info = env.step(action)

print('with best weights game lasted ', count)

env.close()
env_to_wrap.close()

Any suggestions?

[PixnBits]

This occurs for me when running the example that uses the done flag in https://gym.openai.com/docs/
I can't find the repo/branch for that site but would like to open a PR to add the env.close line if someone knows where that is?

[sprakashdash]

@brandonskerritt It shows no error when you write:
observation = env.reset() before the second while not done: statement.

[nofreewill42]

@brandonskerritt

#################### You do:

env = gym.make('CartPole-v0')
env_to_wrap = gym.make('CartPole-v0')
env = wrappers.Monitor(env_to_wrap, 'MovieFiles', force=True)
env.close()
env_to_wrap.close()

#################### You should do:

env = gym.make('CartPole-v0')
env_to_wrap = gym.make('CartPole-v0')
env_wrapped = wrappers.Monitor(env_to_wrap, 'MovieFiles', force=True)
env.close()
env_to_wrap.close()
env_wrapped.close()

[christopherhesse]

Thanks for looking into this @mpSchrader!

[Abhimanyu8713]

Thanks it worked. @mpSchrader

[zhaoyucong]

@mpSchrader
Traceback (most recent call last):
File "VDSR.py", line 161, in
for w in weights:
TypeError: 'NoneType' object is not iterable

part code：
weights=weights_o.append(weights_w)

for w in weights:
    loss += tf.nn.l2_loss(w)*1e-4

I am not the reason,can you help me

[mpSchrader]

Hi @zhaoyucong,
This does not look like a gym issue to me. It looks like the weights you try to iterate over are not a list.
append is a inplace function and there for returns the type None. Your error there for is in the line:
weights=weights_o.append(weights_w)