Ml-agents: Raycast Debugging

[Xiromtz]

Is your feature request related to a problem? Please describe.
This seems to be intentional, but when using the "RayPerceptionSensor" component, it does not show the debug gizmos unless when training.

Describe the solution you'd like
I want to see my raycasts in realtime without having to start a seperate training process, i.e. controlling my agent using heuristics. I don't quite understand why this isn't possible or I might have missed something.

Describe alternatives you've considered
I've even tried starting a training process while forcing the agent to heuristic mode. This still does not display any raycast debugging.

I might have missed something, but this seems to be intentional. It shouldn't be hard to add a simple checkbox in the debug gizmos inspector for the ray perception component.

Thanks in advance, this would be a great help.

[ervteng]

Hi @Xiromtz, you can see the raycasts during inference mode as well, but only in the scene view. But I was able to replicate that if you're using Heuristic the raycasts don't show. This seems like a bug, and I've logged it with internal tracking number MLA-622.

[Xiromtz]

Yeah, I forgot to mention that it also works in inference! But Inference obviously only works with a trained model, so I still can't test my implementation very well if I can't control the agent myself.
Thanks for the quick reply though!
I've never used this github feature; Should I close this thread, since you've logged the bug internally? Or will it be closed once the bug is fixed?

[ervteng]

Feel free to leave the issue open until the bug is fixed - we'll close it at that time. Thanks!

[Xiromtz]

Awesome, thanks for the help!
For anyone having the same problem and wanting to quickly test their raycasts via heuristic input:
A simple workaround is calling the heuristic function from the AgentAction function by replacing the action array.

[chriselion]

Just a followup on this - I just merged a https://github.com/Unity-Technologies/ml-agents/pull/3484 which (amongst other things) will draw the RayPerception gizmos whenever the Agent is selected (including non-play mode and using Heuristic behavior). It's on the master branch now and will be in the next release (in a few weeks).

This still doesn't run the sensors when using Heuristic behaviors (which is a larger discussion) but it should be enough to solve your original problem.