Shap: GradientExplainer has no expected values: how to interpret it.

[1vecera]

I use GradientExplainer to explain a Recurrent Neural Net with a continuous target. I used KernelExplainer and switched to GradientExplainer as I expect it to be a better approximation of SHAP values.

However, it has no expected_values attr. The code is commented out.

In the documentation, it is usually written:

For models with a single output this returns a matrix of SHAP values (# samples x # features). Each row sums to the difference between the model output for that sample and the expected value of the model output (which is stored as expected_value attribute of the explainer). For models with vector outputs, this returns a list of such matrices, one for each output.

So I'm not sure how to interpret the values without the expectation. If a fix is required, can I somehow calculate it from the gradients?

Thanks, @slundberg ! Amazing package!

[kdmcdrm]

I had this issue as well, I ended up pulling the code that calculates the expected value for deep_tf.py. I put the following in gradient.py

        if (hasattr(self.data, '__call__')):
            self.expected_value = None
        else:
            if self.data[0].shape[0] > 5000:
                warnings.warn("You have provided over 5k background samples! For better performance consider using smaller random sample.")
            if not tf.executing_eagerly():
                self.expected_value = self.run(self.model_output, self.model_inputs, self.data).mean(0)
            else:
                if type(self.model)is tuple:
                    sel.fModel(cnn.inputs, cnn.get_layer(theNameYouWant).outputs)
                self.expected_value = tf.reduce_mean(self.model(self.data), 0)

It make my force plots have nearly the correct prediction, they're still off by about 0.01, but I can live with that.

[RichardScottOZ]

Thanks, will give that a shot.

[kingwongf]

if (hasattr(self.data, '__call__')):
            self.expected_value = None
        else:
            if self.data[0].shape[0] > 5000:
                warnings.warn("You have provided over 5k background samples! For better performance consider using smaller random sample.")
            if not tf.executing_eagerly():
                self.expected_value = self.run(self.model_output, self.model_inputs, self.data).mean(0)
            else:
                if type(self.model)is tuple:
                    sel.fModel(cnn.inputs, cnn.get_layer(theNameYouWant).outputs)
                self.expected_value = tf.reduce_mean(self.model(self.data), 0)

Under which function you added the piece of code? GradientExplainer()'s or _TFGradientExplainer()'s __init__ ?