Gym: Mujoco agent not fully tracked

[rajcscw]

When I render the Half cheetah environment (also for other mujoco envs), I observe that camera is not moved automatically when the agent moves off the screen. Is this an issue or some manipulations to the camera to be done?

[R1ckF]

Hi,
I have exactly the same issue. Any new information available?

[pzhokhov]

Hi @rajcscw and @R1ckF! In mujoco env render, when you use mode 'human', Tab key allows you to move between viewpoints. One of the viewpoints is locked to the agent. Now if you are using rendering mode 'rgb_array' there is no good way to configure the viewpoint. Due to a recent bug https://github.com/openai/baselines/issues/549 this also applies to replaying models in baselines; will be fixed soon.

[pzhokhov]

should be fixed now

[rajcscw]

@pzhokhov Could you please tell me in which gym version it is fixed?

[rajcscw]

@pzhokhov So how is it fixed? When we render with "rgb_array" mode, will the agent be tracked automatically?

[pzhokhov]

Tracking the agent in the rgb_array mode is not actually fixed (what was fixed is correct usage of mode='human' downstream). As such, let me reopen this issue

[pzhokhov]

also, as a temporary patch, you can add camera_id=0 to the render argument here: https://github.com/openai/gym/blob/4ceff7dc096fa1c4fb56e68a1d6e34f4bff2870e/gym/envs/mujoco/mujoco_env.py#L105
(this will switch rendering rgb_mode rendering of mujoco envs to the tracking camera)

[rajcscw]

@pzhokhov When I add that argument, it throws Fixed camera id is outside valid range

[pzhokhov]

@rajcscw which environment are you using?

[rajcscw]

@pzhokhov swimmer

[rajcscw]

Any updates on this @pzhokhov? This will be really helpful!

[pzhokhov]

@rajcscw sorry for the delay. Looks like specifically for swimmer the mujoco xml file does not include tracking camera. Here's a branch in which I have added a tracking camera:
https://github.com/openai/gym/tree/issue1135
(specifically, here https://github.com/openai/gym/commit/82cae71acf11eaa0b994e397e8d476c7e8852f35)
It should also be used with camera_id=0 fix from the discussion above. Changing the rgb_mode rendering behaviour merits releasing new version of the environments (or at least adding a gym.make kwarg now that we support them that defaults to old behaviour). PR on that is coming.

[pzhokhov]

oops closed accidentally, sorry and reopened

[rajcscw]

@pzhokhov Sorry for the delay, that worked perfectly! I assume the rest of the mujoco envs works fine already, have not tried it though.

[pzhokhov]

here's a PR that adds an optional kwarg that enables tracking in the rgb mode: https://github.com/openai/gym/pull/1405

[pzhokhov]

should be fixed now; I actually changed the default behavior to tracking rgb camera. The old behaviour can be restored by passing rgb_rendering_tracking=False kwarg. Also, a new version of pypi package is released.