Ml-agents: Help with reward system

[dacalder]

Hi! How can I wait for the end of an animation to give a reward? I tried using an enum state system, but when I use a while state == OtherState.attacking, the game crashes. The reason I'm trying to do this is because I'm not sure how bad it would be to attribute a "done" reward to an action that did not lead to it.

Thanks!

[MarcoMeter]

If you use a while-loop inside of any frequently called functions (like Update() or FixedUpdate()), your game will crash.

There are a couple of approaches to solve that.

You can use animation events. https://docs.unity3d.com/Manual/AnimationEventsOnImportedClips.html
You could poll the current animation state in FixedUpdate(), but do not use any loops for that as FixedUpdate() is called frequently at fixed time steps.
And checkout state machine behaviors. https://docs.unity3d.com/ScriptReference/StateMachineBehaviour.html

[srcnalt]

How can I wait for the end of an animation to give a reward?

You can add events to your animation (source). From the method you use, you can get a reference of your agent and set the reward to the desired value.

Make sure to write a decision component to test your game on Heuristic mode before using it.

[dacalder]

Thank you for your response! I'm stuck in a similar problem. I am training an agent to shoot arrows, and I need to know if the arrow hits the moving target. Do you have any suggestions on how to wait to reward the shooting action until it hits/misses the objective?

Thanks!

[MarcoMeter]

@dacalder
This is a communication problem issue, which is related to how you setup your code basis.

But in general, the target has a reference to the agent and can trigger a function on the agent once the target got hit.
The job can get done by the arrow as well. Once you instantiate the arrow, simply pass a reference of the agent (who I guess instantiates the arrow) to the arrow, so that the arrow can trigger the function on the agent to make some reward logic happen.

[unityjeffrey]

Thanks for reaching out to us. Hopefully you were able to resolve your issue. We are closing this due to inactivity, but if you need additional assistance, feel free to reopen the issue.

[SestoAle]

I have a similar problem. I set on demand decisions and, when it comes the turn of the agent, i call RequestDecision() that calls CollectObservation() and then AgentAction(), but AgentAction() sends immediately the reward signal to the brain. how can I wait for the agent to finish his turn before giving the reward?

[unityjeffrey]

hi @SestoAle can you open a new issue?

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.