Stable-baselines: AttributeError: 'NoneType' object has no attribute 'get'

[chyunf001]

**Hello, I got a issue when learning a custom environment. Please give me some advices, thank you!

from stable_baselines import A2C
from stable_baselines.common.cmd_util import make_vec_env
import numpy as np
# instantiate the env
np.random.seed(0)
env = gym.make('MyHEM-v0')
env = make_vec_env(lambda: env, n_envs = 1)
model = A2C('MlpPolicy', env, gamma=1.0, verbose=1)
model.learn(total_timesteps=5000)

**My error is**
---------------------------------------------------------------------------
 AttributeError: 'NoneType' object has no attribute 'get'

**My environment code is roughly like this**
---------------------------------------------------------------------------
import random
import json
import gym
from gym import spaces
import pandas as pd
import numpy as np

class HEMS01(gym.Env):
    metadata = {'render.modes': ['human']}

    def __init__(self,):
        super(HEMS01, self).__init__()
        self.appliances_number = 1
        self.electricity_cost = np.array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 12, 12, 5, 5, 5, 5, 10, 10, 10, 5, 5, 5])
        self.appliances_consumption = 1
        self.schedule_start =10
        self.schedule_stop = 20
        self.time_stamp = 0
        ##  Action of the format
        n_actions = 2
        self.action_space = spaces.Discrete(n_actions)
        # discribe by Discrete and Box space
        self.observation_space = spaces.Box(low = 0, high = 24,
                                            shape=(1,5), dtype = np.float32)
        self.done = False
        self.time_stamp = 0
        self.state_accumulation = 0
        self.episode_rewards = []
        self.history_actions = []

    def get_action_shape(self):
        return 1

    def reset(self):
        self.done = False
        self.time_stamp = 0
        self.state_accumulation = 0
        self.history_actions = []
        return self.get_obs()

    def get_obs_shape(self):
        return np.shape(self.get_obs())

    def get_obs(self):
        return [self.time_stamp,self.state_accumulation,self.schedule_start,
               self.usage_duration,self.schedule_stop]

    def reward(self, action):
          pass

    def seed(self, seed=None):
        self.np_random, seed = seeding.np_random(seed)
        return [seed]

    def step(self, action):
        pass

    def render(self, mode='human', close=False):
        pass

    def close(self):
        pass

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-16-b7dcb49ceb4c> in <module>
      8 env = make_vec_env(lambda: env, n_envs = 1)
      9 model = A2C('MlpPolicy', env, gamma=1.0, verbose=1)
---> 10 model.learn(total_timesteps=5000)
     11 #model.learn(total_timesteps=5000)

~\Anaconda3\envs\pytorch\lib\site-packages\stable_baselines\a2c\a2c.py in learn(self, total_timesteps, callback, log_interval, tb_log_name, reset_num_timesteps)
    259                 callback.on_rollout_start()
    260                 # true_reward is the reward without discount
--> 261                 rollout = self.runner.run(callback)
    262                 # unpack
    263                 obs, states, rewards, masks, actions, values, ep_infos, true_reward = rollout

~\Anaconda3\envs\pytorch\lib\site-packages\stable_baselines\common\runners.py in run(self, callback)
     46         self.callback = callback
     47         self.continue_training = True
---> 48         return self._run()
     49 
     50     @abstractmethod

~\Anaconda3\envs\pytorch\lib\site-packages\stable_baselines\a2c\a2c.py in _run(self)
    370 
    371             for info in infos:
--> 372                 maybe_ep_info = info.get('episode')
    373                 if maybe_ep_info is not None:
    374                     ep_infos.append(maybe_ep_info)

AttributeError: 'NoneType' object has no attribute 'get'

[araffin]

Hello,
Please fill up the issue template completely (especially please run the check_env() first).

[chyunf001]

Thanks for your reply. Actually, this version of environment has the same error as the original one. So I condiser this error may have something to do with the structure of the environment? And I have tried the check_env() according to your kind advice, the error is like this:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-5-0f6792f977ee> in <module>
----> 1 check_env(env)

~\Anaconda3\envs\pytorch\lib\site-packages\stable_baselines\common\env_checker.py in check_env(env, warn, skip_render_check)
    179         True by default (useful for the CI)
    180     """
--> 181     assert isinstance(env, gym.Env), ("You environment must inherit from gym.Env class "
    182                                       " cf https://github.com/openai/gym/blob/master/gym/core.py")
    183 

AssertionError: You environment must inherit from gym.Env class  cf https://github.com/openai/gym/blob/master/gym/core.py

Please give me more advice to fix it up. Thank you again! Best wishes!

[Miffyli]

You have to run the non-vectorized environment through check_env, i.e. without the make_vec_env.

[chyunf001]

Thanks for your reply. But actually I have used 'env = make_vec_env(lambda: env, n_envs = 1)' as it is shown at the beginning. And then use 'check_env(env)' to check the environment.

import sys
sys.path.append('C:\\Users\\yc331\\Desktop')
import HEMS_single
import gym
from stable_baselines.common.env_checker import check_env
from stable_baselines.common.cmd_util import make_vec_env
import numpy as np

env = gym.make('MyHEM-v0')
env = make_vec_env(lambda: env, n_envs = 1)
check_env(env)

Besides, I tried to register env in gym and adopt 'import' and 'gym.make' to use it. And my tensorflow version is 1.15.0. I wonder is it has something to do with the error?

```python

AssertionError Traceback (most recent call last)
in
9 env = gym.make('MyHEM-v0')
10 env = make_vec_env(lambda: env, n_envs = 1)
---> 11 check_env(env)

~\Anaconda3\envs\pytorch\lib\site-packages\stable_baselines\common\env_checker.py in check_env(env, warn, skip_render_check)
179 True by default (useful for the CI)
180 """
--> 181 assert isinstance(env, gym.Env), ("You environment must inherit from gym.Env class "
182 " cf https://github.com/openai/gym/blob/master/gym/core.py")
183

AssertionError: You environment must inherit from gym.Env class cf https://github.com/openai/gym/blob/master/gym/core.py

[Miffyli]

I meant to say you have to use check_env directly on the environment from gym.make, _without the make_vec_env_. I.e:

env = gym.make('MyHEM-v0')
check_env(env)

[chyunf001]

The issue has been solved perfectly with your guidance. As a beginner, it helps me know more about 'check_env()' and stable-baselines. I appreciate it very much for your kind help. Best wishes!