Shap: Error using DeepExplainer and KernelExplainer for NN model of TF 2.0

[smallfade]

The model I trained is a 3-layer feedforward neural network with tabular data.
The DeepExplainer could be initialized but error when calling shap_values at second line:

e = shap.DeepExplainer(model, background)
shap_values = e.shap_values(x_test[1:5])

TypeError: in converted code:
    relative to /home/small/miniconda3/envs/scML/lib/python3.7/site-packages:

    shap/explainers/deep/deep_tf.py:239 grad_graph  *
        out = self.model(shap_rAnD)
    tensorflow_core/python/keras/engine/base_layer.py:847 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    tensorflow_core/python/keras/engine/network.py:708 call
        convert_kwargs_to_constants=base_layer_utils.call_context().saving)
    tensorflow_core/python/keras/engine/network.py:860 _run_internal_graph
        output_tensors = layer(computed_tensors, **kwargs)
    tensorflow_core/python/keras/engine/base_layer.py:847 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    tensorflow_core/python/keras/layers/core.py:1056 call
        outputs = gen_math_ops.mat_mul(inputs, self.kernel)
    tensorflow_core/python/ops/gen_math_ops.py:6136 mat_mul
        name=name)
    tensorflow_core/python/framework/op_def_library.py:536 _apply_op_helper
        repr(values), type(values).__name__, err))

    TypeError: Expected float32 passed to parameter 'b' of op 'MatMul', got <tf.Variable 'dense_1/kernel:0' shape=(13642, 128) dtype=float32> of type 'ResourceVariable' instead. Error: custom_record_gradient() takes 4 positional arguments but 5 were given



md5-f1a42b94e2b4491f3415be4b6caee9c3



TypeError: in converted code:
    relative to /home/small/miniconda3/envs/scML/lib/python3.7/site-packages/tensorflow_core/python:

    keras/engine/data_adapter.py:279 permutation
        indices = math_ops.range(num_samples, dtype=dtypes.int64)
    ops/math_ops.py:1417 range
        start = cast(start, dtype=dtype, name="start")
    util/dispatch.py:180 wrapper
        return target(*args, **kwargs)
    ops/math_ops.py:704 cast
        x = gen_math_ops.cast(x, base_type, name=name)
    ops/gen_math_ops.py:2225 cast
        "Cast", _inputs_flat, _attrs, _result, name)

    TypeError: custom_record_gradient() takes 4 positional arguments but 5 were given

[Hahny]

Have you found the solution to this problem? I am running into the same issue

[williamcaicedo]

I solved a similar issue by adding these lines before getting the Shapley Values (I upgraded to TF 2.1 as well):
import tensorflow
from tensorflow.compat.v1.keras.backend import get_session tensorflow.compat.v1.disable_v2_behavior()

[smallfade]

I solved a similar issue by adding these lines before getting the Shapley Values (I upgraded to TF 2.1 as well):
import tensorflow
from tensorflow.compat.v1.keras.backend import get_session tensorflow.compat.v1.disable_v2_behavior()

Thanks for your tip. But this doesn't solve my problem. I guess these lines would work if the model is built in v1 and used in v2, but my model was built entirely with tf.keras in v2.

[williamcaicedo]

Actually I built my model using TF 2.1

[smallfade]

Actually I built my model using TF 2.1

Then maybe it's a different cause.

[Hahny]

@smallfade It did not work for me either.

[towhid355]

@williamcaicedo It did not work for me. I am using tensorflow==2.0.0 , shap==0.35.0

[slundberg]

Sorry I have been out sick. Could someone share a small self contained example to debug this?

[towhid355]

@slundberg Stay home. Stay safe.
plz check.

Version:

tensorflow==2.0.0
shap==0.35.0

tf.keras model summary:

Layer (type)                 Output Shape              Param #
=================================================================
input_layer (Reshape)        (None, 2, 13, 1, 1)       0
_________________________________________________________________
conv_lstm_1 (ConvLSTM2D)     (None, 2, 13, 1, 64)      166656
_________________________________________________________________
dropout_1 (Dropout)          (None, 2, 13, 1, 64)      0
_________________________________________________________________
conv_lstm_2 (ConvLSTM2D)     (None, 13, 1, 32)         61568
_________________________________________________________________
dropout_2 (Dropout)          (None, 13, 1, 32)         0
_________________________________________________________________
flatten (Flatten)            (None, 416)               0
_________________________________________________________________
dense (Dense)                (None, 100)               41700
_________________________________________________________________
output_layer (Dense)         (None, 1)                 101
=================================================================
Total params: 270,025
Trainable params: 270,025
Non-trainable params: 0

shap code :

data = tf.convert_to_tensor(data,  dtype=tf.float32)
test= tf.convert_to_tensor(test[:10],  dtype=tf.float32)
explainer = shap.DeepExplainer(model, data)
shap_values = explainer.shap_values(test, check_additivity=True)

Error:

  shap\explainers\deep\deep_tf.py:239 grad_graph  *
        out = self.model(shap_rAnD)
    tensorflow_core\python\keras\engine\base_layer.py:847 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    tensorflow_core\python\keras\engine\sequential.py:256 call
        return super(Sequential, self).call(inputs, training=training, mask=mask)
    tensorflow_core\python\keras\engine\network.py:708 call
        convert_kwargs_to_constants=base_layer_utils.call_context().saving)
    tensorflow_core\python\keras\engine\network.py:860 _run_internal_graph
        output_tensors = layer(computed_tensors, **kwargs)
    tensorflow_core\python\keras\engine\base_layer.py:847 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    tensorflow_core\python\keras\layers\core.py:471 call
        (array_ops.shape(inputs)[0],) + self.target_shape)
    tensorflow_core\python\ops\array_ops.py:813 _slice_helper
        name=name)
    tensorflow_core\python\ops\array_ops.py:979 strided_slice
        shrink_axis_mask=shrink_axis_mask)
    tensorflow_core\python\ops\gen_array_ops.py:10403 strided_slice
        "StridedSlice", _inputs_flat, _attrs, _result, name)

    TypeError: custom_record_gradient() takes 4 positional arguments but 5 were given

[karkinissan]

Getting the exact same error as @towhid355 . Any update on this?

[amilton300]

Had the same issue. Solved it by updating tensorflow from 2.0.0 to the newest version.

[a-agmon]

Was also solved for me by updating tf to 2.2