Ml-agents: Multiagent Bug in Gym-Unity 14.1

Created on 28 Feb 2020 · 9Comments · Source: Unity-Technologies/ml-agents

Hello,

There seems to be a bug with multiagent environments in gym-unity release 14.1.
Some of the example environments work fine (e.g. 3D ball) while some of them (e.g. WallJump) fail with this error:

"The environment does not have the expected amount of agents. "
gym_unity.envs.UnityGymException: The environment does not have the expected amount of agents. Some agents did not request decisions at the same time.

It is possible that the issue is originated in the Unity environment side, but I can't seem to find the reason behind it. Single Agent environments work fine.

I'm using the provided notebooks here.

Thanks,
Sahand

bug

Source

sahandrez

All 9 comments

Hi @sahandrez, the multi-agent gym currently doesn't support environments with multiple types of behaviors (WallJump has two), or on-demand decisions (e.g. Bouncer). This is due to how gym expects all data per step to be from the same observation/action space.

ervteng on 28 Feb 2020

Hi @ervteng,
I see. Thanks. I'm having another problem after fixing what you mentioned. Now in our custom environment, we are getting this error as soon as an agent is done in the environment:

File "\workspace\ml-agents\gym-unity\gym_unity\envs\__init__.py", line 531, in mark_agent_done
    gym_index = self._agent_id_to_gym_index.pop(agent_id)
KeyError: 7

Any idea what might have caused it? Thanks.

sahandrez on 28 Feb 2020

Hi @sahandrez, are the agents done at different times/do they use on demand decisions?

Could you post the Inspector for your agents? Would be really helpful for us to debug. Thanks!

ervteng on 2 Mar 2020

Hi @ervteng,
Thanks. We solved our problem. There was a bug in our environment which was not flagged in V0.13 but with the new multiagent system in V0.14 we figured it out.

sahandrez on 2 Mar 2020

👍1

Hello @sahandrez ,
Can you tell me how you fixed the problem? I am facing the same KeyError issue in mark_agent_done

batu on 9 Mar 2020

Hi @batu,
Sorry I no longer have access to the code. You can ask @ReinforcedMan for more info.

sahandrez on 9 Mar 2020

👍1

Hi @batu ,
Did you fix it? I got the same error when I use gym-unity-0.15.0. Looking forward to your reply. Thank you.

RealZST on 23 Mar 2020

Unfortunately no. But I have something that is good enough.

In my case, I believe it happened when my agent got out of bounds and didn't reset properly. It was going out of bound because my implementation was acting weird when I push the simulation speed very high.

Currently, it happens only every 10-100 episodes, so when it does I just catch it and... pass.

batu on 24 Mar 2020

For this error:

File "\workspace\ml-agents\gym-unity\gym_unity\envs\__init__.py", line 531, in mark_agent_done
    gym_index = self._agent_id_to_gym_index.pop(agent_id)
KeyError: 7

We found a bug in the gym wrapper which would appear if an Agent called EndEpisode multiple times in the same step. The fix for this is in https://github.com/Unity-Technologies/ml-agents/pull/3700, and we'll be doing a patch release next week that contains this fix (and a few others).