Stable-baselines: How to log the gradient and weights of the actor policy in TD3 using tensorboard?

[AvisekNaug]

Code snippet:

import tensorflow as tf
def Callback(_locals, _globals):
    self_ = _locals['self']
    tf.summary.histogram("Actor Network Weights Histogram", self_.policy_out.policy.??? )

I cannot understand from the

class FeedForwardPolicy(TD3Policy):

in FeedForwardPolicy(TD3Policy)

how to get the the trainable variables for the network, something like tf.trainable_variables() used to plot historgams in tensorboard.

[araffin]

Hello,

You should take a look at the code of TD3, not the policy, that's where we use tf.trainable_variables(). (I recommend you to take a look at PPO2 too, where we log more things using tensorboard)
You can get the name of the weights (scope + name) using model.get_parameter_list() (cf doc)

Since https://github.com/hill-a/stable-baselines/issues/409, we also added some documentation on how to log additional variables.

[AvisekNaug]

Thank you I will look into the details1

[yotamitai]

Hey,
How can I retrieve the actual values of the weights in each layer?

More specifically - I trained an agent and would now like to know the weight values of the second-to-last layer.
Using model.get_parameter_list() I was able to see the layers but couldn't find their weight values.
Is there a different method for obtaining these?

[Miffyli]

@yotamitai

Use get_parameters(), which is right next to get_parameter_list().

[yotamitai]

Thanks!

Is there a way to obtain the result of a specific layer for an observation input?
Given an observation, I'd like to know the output of a certain layer.

For instance, is there an argument I can pass to the predict() function that would limit the prediction to a specific layer instead of running through the whole model?

[Miffyli]

@yotamitai

You need to modify stable-baselines to achieve that, see #825.