Probability: Gradients does not exist for variables ['sin_cos_model_3/dense_flipout_12/kernel_posterior_untransformed_scale:0' etc ...

[gunshi]

Hi, i'm using the tf-nightly-2.0-preview build for tfp, along with tf-2.0.0b1.
I'm trying to use the DenseFlipout layers to code a bayesian nn, as in the following standalone code
[(https://colab.research.google.com/drive/11ohqORjeuxtd9AWC9A0Bqdd1pL1n6iMl)]

When i run the part where ti applies the gradients however, I get the warning :
"W0618 15:00:14.972359 139746404841280 optimizer_v2.py:982] Gradients does not exist for variables ['sin_cos_model_3/dense_flipout_12/kernel_posterior_untransformed_scale:0', 'sin_cos_model_3/dense_flipout_12/bias_posterior_loc:0', 'sin_cos_model_3/dense_flipout_13/kernel_posterior_untransformed_scale:0', 'sin_cos_model_3/dense_flipout_13/bias_posterior_loc:0', 'sin_cos_model_3/dense_flipout_14/kernel_posterior_untransformed_scale:0', 'sin_cos_model_3/dense_flipout_14/bias_posterior_loc:0', 'sin_cos_model_3/dense_flipout_15/kernel_posterior_untransformed_scale:0', 'sin_cos_model_3/dense_flipout_15/bias_posterior_loc:0'] when minimizing the loss."

and i also see that my loss doesn't decrease. Printing out the gradients shows that some of them are None.

Is this layer/model not supported currently in eager mode/tf2?
Any help/alternate ways to do it would be much appreciated!
Thanks
Gunshi

[SiegeLordEx]

The fix for that landed today, so this should be fixed in the nightly tomorrow.

[gunshi]

Cool, thanks!
Just as a follow up, (since I'm planning to add that on top of this eventually)
Is it supported to use bijectors/transformedDistributions (like affine flows) and define losses based on the transformed variables (like log_prob or kl_divergence with a prior) in tfp/tf2 eager mode yet? (I'm only now getting familiar with tf probability and not sure what all is supported in tf2)

[SiegeLordEx]

All the variational layers should work in TF2 now, and if you're using your own custom prior/posterior distributions then you'll just need to be careful to do this trick (and also see below) for your transformed parameters.

More generally, with any bijector/distribution if you re-create your distributions under the tf.GradientTape or inside the loss_fn passed to an optimizer, then things will already fine today, e.g.

transformed_scale = tf.Variable(0.)
with tf.GradientTape() as tape:
   dist = tfd.Normal(0., tf.nn.softplus(transformed_scale))
   log_prob = dist.log_prob(1.)
tape.gradient(log_prob, transformed_scale) # not None

What we're working on now (and what fixed the variational layers), is support for this kind of code:

transformed_scale = tf.Variable(0.)
dist = tfd.Normal(0., tfp.util.DeferredTensor(transformed_scale, tf.nn.softplus))
with tf.GradientTape() as tape:
   log_prob = dist.log_prob(1.)
tape.gradient(log_prob, transformed_scale) # not None

which can be considered nicer to read. Only a few distributions/bijectors have been converted to support this, but going forward they'll all support this eventually.

[ritchie46]

Is this the same problem?

Somehow I don't get gradients in my train loop. Below is a simple example.

import tensorflow as tf
import tensorflow_probability as tfp
tf.enable_eager_execution()

hidden_size = 32
output_size = 1
m = tf.keras.Sequential(
[
            tfp.layers.DenseLocalReparameterization(hidden_size, tf.nn.leaky_relu),
            tfp.layers.DenseLocalReparameterization(hidden_size, tf.nn.leaky_relu),
            tfp.layers.DenseLocalReparameterization(output_size)
        ]
)

If I run a gradient recording step two times, the second times doesn't show any gradients. A list with None types is returned.

for _ in range(2):
    with tf.GradientTape() as tape:
        loss_value = m(tf.ones((1, 2))) * 2
        print(tape.gradient(loss_value, m.trainable_variables))

This is not the case if we replace the model m by a 'standard' tensorflow model, i.e.

m = tf.keras.Sequential([tf.keras.layers.Dense(1)])

I am using tensorflow=1.13.1 and tensorflow-probability=0.6.0

[zhoudoao]

I have encountered the same problem when I used the DenseFlipout as a layer of a model in Tensorflow 2.0. The problems occurs in the second call of tape.gradient(loss, trainable_variables) which returns None for all gradients. Can I use VariableLayer and DenseVariable to implement the function of DenseFlipout and DenseReparameterization? As I know, VariableLayer and DenseVariable are compatible with Tensorflow 2.0 and DenseVariable is a higher level than DenseFlipout and DenseReparameterization.

[gunshi]

Yes I also observe a None in the second position(and onwards) of my gradients list.

[zhoudoao]

After updating the newest TF-nightly and TFP-nightly, this issue is solved. This issue is the same as https://github.com/tensorflow/probability/issues/409

[gunshi]

@zhoudoao @ritchie46 Does the model train properly for you after the upgrade? (the loss decreases etc )

[zhoudoao]

Yes, the loss decreases and accuracy increases. It looks like to converge. But I haven't tested how the gradients change in every iteration.

[ritchie46]

@gunshi Yes, gradients work fine after updating to the nightlies.

[cottrell]

What is the new tf2.0 nighlyt called? Old one doesn't work.

[cottrell]

Nvmd, I think the nightlies are missing some MacOS builds.

[kristofgiber]

Just for the record, I haven't seen this in the documentation, do I have to manually turn on eager execution for TFP to work with TF2?

@cottrell Im also on osx, what build is missing?

[brianwa84]

Eager is optional. Feel free to call tf.enable_v2_behavior() (recommended), or not.
The v2 escape hatch to ~v1 behavior is to put code in a @tf.function-wrapped python function a la:

@tf.function
def f():
  # Inside this function, we are in graph mode.
  return tf.zeros([1,2,3])

[kristofgiber]

@brianwa84 Thanks, that clears it up a bit. I'm only confused about how to apply eager execution on TFP. To me executing_eagerly() returns True by default in TF2, and indeed tf2 functions return tensor objects that include a numpy-array field which helps with debugging. But TFP functions such as distributions still return tensor objects without an evaluated numpy array, even in eager mode. Is there no way to read output value of tfd distributions eagerly?

[kristofgiber]

@brianwa84
I understand now that the @tf.function decorated functions and their child functions are taken out of eager execution so that's why those TFP functions (which I had inside my loss function called by a decorated function) didn't run eagerly.

Just one more observation, using

import tensorflow as tf

seems to get the exact same results and speed for me as using

import tensorflow.compat.v2 as tf
tf.enable_v2_behavior()

I ran my comparison for a VAE on MNIST, this included a couple of tfp functions such as MultivariateNormalDiag and Bernoulli.

So for this task it seems to be fine just to install tensorflow2-beta + TFP and import tensorflow as tf

[cottrell]

Nvmd, I think the nightlies are missing some MacOS builds.

tf-nightly-2.0-preview python 3.7

[TMullerSG]

Is this the same problem?

Somehow I don't get gradients in my train loop. Below is a simple example.

import tensorflow as tf
import tensorflow_probability as tfp
tf.enable_eager_execution()

hidden_size = 32
output_size = 1
m = tf.keras.Sequential(
[
            tfp.layers.DenseLocalReparameterization(hidden_size, tf.nn.leaky_relu),
            tfp.layers.DenseLocalReparameterization(hidden_size, tf.nn.leaky_relu),
            tfp.layers.DenseLocalReparameterization(output_size)
        ]
)

If I run a gradient recording step two times, the second times doesn't show any gradients. A list with None types is returned.

for _ in range(2):
    with tf.GradientTape() as tape:
        loss_value = m(tf.ones((1, 2))) * 2
        print(tape.gradient(loss_value, m.trainable_variables))

This is not the case if we replace the model m by a 'standard' tensorflow model, i.e.

m = tf.keras.Sequential([tf.keras.layers.Dense(1)])

I am using tensorflow=1.13.1 and tensorflow-probability=0.6.0

hi, did you solve the problem?

[nbro]

@SiegeLordEx, @brianwa84, @davmre @jvdillon, etc., you should re-open this issue! It's still not solved in the lastest night versions of TF (2.2) and TFP (0.11) See https://github.com/tensorflow/probability/issues/409#issuecomment-618505416.