Captum: Small change needed to make binary classification work

[rmcavoy]

I am trying to use Obstruction on a binary classification task. When I tried to perform the attribution, _verify_select_column (from _select_targets from _run_forward) errors out when pred_label=1 because it tries to use the value of the label to select the value of the output return output[(slice(None), *target)] . If _verify select_column instead first checks if output.shape[1] == 1: return output[(slice(None), slice(None))], then it successfully creates an obstruction map.

Is there something else I am missing to get this to work?

[bilalsal]

Hi @rmcavoy, thank you very much for bringing this up!
In case of a unique scalar output, Captum does not expect the user to specify a target, it will automatically default to the behavior you described.
The documentation of the attribute() method mentions the need to leave the target parameter as None when it is uniquely identifiable by Captum.
To avoid confusion, we will include a more informative error message you received.

Hope this helps

[vivekmig]

Hi @rmcavoy , as @bilalsal explained, this error is expected in your case, since a target should only be passed when needing to select a scalar output from multiple outputs per example. In classification tasks where there is an output per class, then it is correct to pass to any attribution method the target argument with the appropriate label.

In your case, since you output only one output for your binary classification task, no target should be passed, since there is only one scalar output per example. Note that this always attributes with respect to the positive class (class corresponding to higher output score) and doesn't depend on the target label.

Does simply not passing target or passing target=None resolve your issue?

[rmcavoy]

Yes, it did. It would be good to update the error message as passing target=None is not an intuitive option. As an aside, the name "target" there doesn't really correspond to its use as an index. Might be more intuitive if it was called "target_index" or you could just leave it the way it is and update the error message.

[vivekmig]

Makes sense, thanks for the feedback! It's difficult to make changes to the API and maintain backward compatibility, but we'll update the error to be more intuitive.