Baselines: Using Vectorized environement

[NourBesbes]

env = DummyVecEnv([make_env]) env = VecNormalize(env)

Why using vectorized environement ? It returns observation values instead of the values returned by the environement class ( step function )

Any ideas, please ?

[unixpickle]

The vectorized environment simply wraps one or more gym.Env environments and runs them simultaneously. PPO2 is implemented to use a vectorized environment. So, if you want to use a single gym.Env, you have to use something like DummyVecEnv that makes the environment appear to be vectorized.

[NourBesbes]

What about the vecnormalize function?
If I am using a simple gym environment.
Should I use it??

[unixpickle]

I think VecNormalize is useful for some MuJoCo tasks, but it's definitely not a requirement.

[araffin]

@NourBesbes You should not use VecNormalize on pixels because most of the cnn policies already normalize images (dividing by 255), unless you only want to normalize reward (in that case, you should pass ob=False)
However, VecNormalize can be really useful for everything that is not pixels (e.g. joint space or [x,y,z] coordinates). Also, you have to save the running average if you want to play with a trained agent afterwards.

Vecnormalize computes a running average to estimate mean and std for observations and rewards.