Ml-agents: Imitation Learning slows over time

[dracolytch]

While I was doing the work for my video tutorial on Imitation Learning, I found that in long training sessions the game would slow down, sometimes significantly. The scenario being used is pretty straightforward (I am using the Physics engine though). Any thoughts as to why the training might get slower over time?

Video demoing scenario is here: https://youtu.be/WzuVZuGT-rw
Code is here: https://github.com/dracolytch/ML-Simplest-Scenario

[vincentpierre]

Hi,
The slowdown happens because the experience buffer used to train the agent becomes very large and it becomes harder to sample efficiently from it. If you have a BCTeacherHelper component attached to your teacher in your environment, you can press C to clear the experience buffer. Another thing that could cause this is when the batch_size and batches_per_epoch parameters are too large, the amount of calculation needed to update the model (at every step) slows the communication down. I think you could decrease these two values and see if it solves your problem.
I hope this helps.

[superjayman]

If you press c to clear the experience buffer does that also clear the learning it has done so far?

[MarcoMeter]

@superjayman
The learned behavior is stored within the weights of the neural network. This is not getting erased/cleared/reset.

The learning algorithm samples training examples from the experience buffer. So if the buffer is large, then there is a potential that old or useless data is utilized during training and thus it is not of help for the training progress.

[mmattar]

Closing out this issue as it seems to be resolved. Do submit a new one if you have any additional questions.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.