Baselines: ACKTR with LSTM from A2C

[kikushah]

Hi,

Is there a way to use LSTM policy with ACKTR, instead of the cnn policy?

When I use the LSTM implementation in A2C with this ACKTR implementation the KFAC module throws an error. Below is a snapshot, Please check if I am doing something wrong.

Backend TkAgg is interactive backend. Turning interactive mode on.
get factor for step_model/fc1/w:0
get factor for step_model/fc1/b:0
get factor for step_model/fc2/w:0
get factor for step_model/fc2/b:0
get factor for step_model/lstm1/wx:0
gradientsSampled/AddN_40:0
['MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul']
1
get factor for step_model/lstm1/wh:0
gradientsSampled/AddN_41:0
['MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul', 'MatMul']
1
get factor for step_model/lstm1/b:0
gradientsSampled/AddN_39:0
[None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]

[lnpalmer]

KFAC isn't compatible with recurrent networks- it makes loss curvature approximations that are specific to the model architecture and haven't been extended to recurrent nets. That being said, you might be able to modify KfacOptimizer to not optimize the LSTM part and use a simpler optimizer for that part instead.