Ml-agents: Comprehension Questions

[MentalGear]

Great blog posts & documentation, although what I'm still missing is:

1. What exactly is a step ? Is it a frame, or action that an agent takes ?

2. Is the sequence of Observations of importance when training the same models?
   For example, given

AddVectorObs(A)
AddVectorObs(B) 
AddVectorObs(C) 

changed to

AddVectorObs(C)
AddVectorObs(B) 
AddVectorObs(A) 

a. And then training the same model confuse previous training, or would it not be an issue ?
b. What about resuming training on a codebase that has added new AddVectorObs(), scalling it from for ex. 3 to 5 Observatons ?

[ervteng]

• A step corresponds to one action of the agent.
• Yes, the order matters - if you train a model with one order, those inputs are mapped in that order into the neural network. It will have to re-learn what each input means if you change the order.
• We currently don't support changing observations - it's still an active area of research as to how to do that effectively in deep RL. But definitely something we're looking into for the future.

[MentalGear]

Wouldn't it makse sense to provide a GUI, like with the action space, to put your observation variables in, for that if you refactor your code, you don't mess up the order.
Or even better: Let us assign them fixed names when defining them, like:

AddVectorObs(C, "velocity")

I mean, there must be a better way to anchor them than just hope not to mess up the observation order, which can quickly happen when coding.

[ervteng]

I completely agree - we'll definitely take this feedback into account in future updates of the UI. Thanks!

[awjuliani]

Hi all. Closing this issue due to inactivity. Feel free to re-open if you feel necessary.

[MentalGear]

@ervteng Was there a variable name for Observation Vars introduced ?

[ervteng]

Hi @MentalGear, we're refactoring how vector observations are being done completely - but no plans for names. We'll keep you posted!