Ml-agents: Dynamic agent spawning / death of agent

[MDReptile]

I have a two part question. First is how should I go about dynamically spawning agents, in a simulation where agents can die and spawn dynamically?

Second is, how do I handle agent death? It seems calling Destroy() on the agents gameobjects brings up issues.

Is there a clean way to add/remove agents during training or runtime?

[kwea123]

To safely remove agents, I would create an empty object an add prefabs and agent script to it, this object then is responsible for destroying and spawning new agents. Also use DestroyImmediate() instead of Destroy().

As far as I know, there is no way to handle dynamic states for the moment, another way round this problem would be to define the number of states to be the maximum, i.e. if your dynamic spawning agents is the number of enemies and there are at most 10, then define the number of states to be that for 10 agents, and add additional information indicating whether the ith agent is present (for example [1,0,...0] indicating only the first enemy is present). In this way your number of states is fixed regardless of the number of agents present. Don't know if it will perform well though...

[MDReptile]

Hmm, so you mean have one agent control all the separate entities?

That would mean on reset, it would reset them all wouldn't it?

EDIT: ok I re-read, so you mean have an agent factory type thing, like to create and destroy agents? I'm not sure I follow you about the object responsible for destroying and spawning new agents... How would it work? Would it use agent.RemoveBrain() or something to stop an agent before destroy?

[kwea123]

Oh sorry I didn't see the plural. Yeah If you have multiple agents then set multiple objects. What I mean is that don't attach the script to an object that you're going to destroy.

[MDReptile]

Ok, so have a total number of agents present that control "dummies" of themselves... and dynamically spawn the dummies.

How do I "pause" the non-active agents?

EDIT: Would it hurt to just ignore their input while "paused"?

[MarcoMeter]

One approach would be to destroy everything except the Agent script. So the agent script would be the parent object and all the childs are destructible. While the agent is paused/freezed (whatever you want to call it). Just simply pass some dummy state data to the brain and do not execute any decisions.

[kwea123]

I would write a while(waiting to respawn...) loop in AgentReset() for the agents, so that they wait for an external condition to be met to respawn. This solution might not be optimal in terms of energy though..

[MDReptile]

I'll give that a shot, thanks for the help @kwea123 !

[kwea123]

Would it use agent.RemoveBrain() or something to stop an agent before destroy?

Normally you need to set done=true to stop the agent first, then in AgentOnDone() call AgentReset() which calls DestroyImmediate(your gameobject), then respawn with your gameobject = Instantiate(your prefab,...)

[MDReptile]

Well I want to suspend the agent while the rest of the agents play out the scenario, and after all agents die, or some time passes, restart the scenario (and regenerate the world... randomly spawn more agent dummies...), and then put the agents back to work at full capacity again. For now I'm testing just ignoring input and giving it blank state info.

[mmattar]

Closing this out due to inactivity, but also adding a reference to a new doc: https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Design-Agents.md#instantiating-an-agent-at-runtime

Please reopen the issue if you have any additional questions.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.