Gym: How to configure the cartpole environment such that it is not capped at 200?

[yxchng]

[tlbtlbtlb]

After creating the environment, you can change the limit:

    env.tags['wrapper_config.TimeLimit.max_episode_steps'] = 500

[hholst80]

You can also create your own environment. Look in envs/__init__.py how it is done.

register(
    id='CartPole-v0',
    entry_point='gym.envs.classic_control:CartPoleEnv',
    tags={'wrapper_config.TimeLimit.max_episode_steps': 200},
    reward_threshold=195.0,
)

register(
    id='CartPole-v1',
    entry_point='gym.envs.classic_control:CartPoleEnv',
    tags={'wrapper_config.TimeLimit.max_episode_steps': 500},
    reward_threshold=475.0,
)

You can define a new one in the same spirit:

register(
    id='CartPole-v2',
    entry_point='gym.envs.classic_control:CartPoleEnv',
    tags={'wrapper_config.TimeLimit.max_episode_steps': 5000},
    reward_threshold=4750.0,
)

[sanjaythakur]

Hi,
I was trying to raise the maximum steps per episode on Mountain Car environment.
I used this

env = gym.make('MountainCar-v0')
env.max_episode_steps = 500

But it still remain capped at 200.
I also tried creating a new register entry, but it gave me some 'UnregisteredEnv' error.
Can anyone give me some idea on how I should go about increasing the upper bound on episode size?
Thanks!!

[tlbtlbtlb]

You need to create a new env with a different name. For example:

gym.envs.register(
    id='MountainCarExtraLong-v0',
    entry_point='gym.envs.classic_control:MountainCarEnv',
    max_episode_steps=500,
    reward_threshold=-110.0,
)
env = gym.make('MountainCarExtraLong-v0')

[sanjaythakur]

Thanks, one of the ways worked. I edited '__init__.py' under 'gym/envs/' to increase the maximum allowed steps in an episode.
However, just to let you know, this particular way
env.tags['wrapper_config.TimeLimit.max_episode_steps'] = 500
after the environment creation line doesn't work. It gives

AttributeError: 'MountainCarEnv' object has no attribute 'tags'

So I checked the 'mountain_car.py' under 'gym/envs/classic_control/' and I really didn't find anything that limits the upper bound on the size of the episodes.

[hholst80]

It might be injected into the object by other means. Remember: this is Python. 😂

[sanjaythakur]

@hholst80 , Yes, I agree. But, I stopped looking for efficient ways, once I found an inefficient solution that worked.

[quantcompany]

With
env = gym.make("CartPole-v0")
what I did was simply:
env._max_episode_steps = 500

and it worked.