Gym: changing action and observation space size

[mohakbhardwaj]

I am trying to make an environment for graph search and need to be able to change the environment and observation space size as more nodes get added to the frontier. Is it possible using the current gym spaces? and is there any tutorial on how to implement your own spaces

[stevenschmatz]

Which action/observation space objects are you using?

• One option would be to directly set properties of the gym.Space subclass you're using. For example, if you're using a Box for your observation space, you could directly manipulate the space size by setting env.observation_space.low and env.observation_space.high values. Check out the source code for more details.
• Alternatively, you could directly create a new Space object and set it to be your observation space: env.observation_space = Box(low, high, shape). Doing this rapidly would probably be more costly than the other method.

Not sure what the other options are here, but would love to hear a second opinion. As for a tutorial, I couldn't find one, but you could take a look at the code for some built-in envs to get started!

[mohakbhardwaj]

I am using Box for the observation space and Discrete for my action space and both keep on changing as the size of my search frontier increases, so the second option might not be very feasible for me. I will try the first one and see the results. For Discrete space, would changing the env.action_space.n property be sufficient?

[stevenschmatz]

Yes, I think changing the env.action_space.n should be sufficient.

[theonlyjohnny]

Any note on whether or not this worked for you @mohakbhardwaj ?

[mohakbhardwaj]

@theonlyjohnny I was able to get this working using the advice from @stevenschmatz

[nnmsIsaac]

Close this issue, maybe?

[bionicles]

it would be great to clarify if Box can support shapes like (None, 3) etc -- where only one dimension is constrained

[shuruiz]

You can define you space like this:

from gym import spaces
space = spaces.Discrete(8) # Set with 8 elements {0, 1, 2, ..., 7}
x = space.sample()
assert space.contains(x)
assert space.n == 8

The space library enables your to define your own action/state space.

source: https://gym.openai.com/docs/

[Eriz11]

@mohakbhardwaj I'm developing an environment that has a changing action space, as your case. The implementation seems reasonable. However, I have doubts on how to then integrate it within the agents.

¿Could you please elaborate a bit more on how did you handle this within the agents? I mean, the last layer of a DQN model for example would need to change its outputs constantly. ¿Did you implement the agent yourself or did you use some library + wrapper?

Many thanks in advance,

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ppanteliadis]

Would it be possible then to have an observation space of varying y dimensions?
for example

After first iteration:

array([[0.],
           [0.],
           [0.]])

After second iteration:

array([[0., 0.],
           [0., 0.98],
           [0., 0.]])

And at the third:

array([[0., 0., 0],
           [0., 0.98],
           [0., 0.]])

In this example, in the second iteration the dimensions of the obs space changed from (3,1) to (3,2) and at the 3rd iteration they changed to (3,3) on the first row and remained (3,2) on the rest. Is this possible?