Shap: KernelExplainer, Tensorflow / Keras, and multiple inputs

[grofte]

When you have multiple input nodes in Keras you pass them as a list. My inputs are one group of numerical features and then a number of features that get passed to an embedding layer.
DeepExplainer handles a list as an input fine but when I try to pass a list to KernelExplainer I get

AssertionError: Unknown type passed as data object: <class 'list'>

That doesn't seem completely model agnostic ; ]

[slundberg]

haha, well it is agnostic to any function that works on a vector :) ...I will add this as a todo, since I think having multiple inputs as a list might be a nice interface to provide. (you could of course write a simple wrapper that packed and unpacked things into a single vector, but that is a pain)

[grofte]

Somehow my models always end up having multiple inputs and outputs so I need to wrap the models anyway =)

Maybe it's better to just have a section in the tutorial about how to go about it. I can send you a link to an example when I'm happy with it.

Edit: For my sake you can close this issue

[chillburg]

Somehow my models always end up having multiple inputs and outputs so I need to wrap the models anyway =)

Maybe it's better to just have a section in the tutorial about how to go about it. I can send you a link to an example when I'm happy with it.

Edit: For my sake you can close this issue

Dear @grofte ,

Can you please give a link or explain how to pass several inputs to a KernelExplainer? Would be very helpful)

Also, is it possible to use multiple inputs in DeepExplainer? I have seen an exaple in this thread but it gives an error even though the full code and data is included. I am also trying to explain a model with both numerical and embedded-categorical inputs (Wide&Deep), and can't make SHAP work unfortunately.

[chillburg]

Update: I managed to solve this problem with a slice() layer. Basically, you concatenate your data into one chunk, and then slice it back inside the model. Problem solved and SHAP works fine!

input = Input(shape=(data.shape[1], ))

x1 = Lambda( lambda x: slice(x, (0, 0 ), (-1, 11)))(input) # deep continuous variables (11)
x2 = Lambda( lambda x: slice(x, (0, 11), (-1, 1)))(input) # embedding (1)

embed1 = Embedding(input_dim = sub_grade_unique+2, output_dim = 8)(x2)
embed1 = Reshape(target_shape=(8,))(embed1)

concat = keras.layers.concatenate([x1, embed1])

Big thanks to @grofte for suggesting it!