Keras: training multi-label classifier with partial nan

[lxkain]

Hi,

I am training a multi-label classifier that has some of the components set to nan regularly. For example, one of the target y may look like [0, 1, 1, nan, 0, nan, 1]. The input x does not have any nans in it. I am aware that I could impute the missing value using some strategy, but it seems like what I really want is to modify the loss function so it doesn't consider those values during fitting.

I have tried the following:

   def mse_nan(y_true, y_pred):  # return a scalar for each data-point
        index = ~K.tf.is_nan(y_true)
        y_true = y_true[index]
        y_pred = y_pred[index]
        return K.mean((y_true - y_pred) ** 2)

unfortunately, I am getting the following error:

 File "learning.py", line 124, in mse_nan
    y_true = y_true[index]
  File "/Users/kain/miniconda3/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 472, in _SliceHelper
    end.append(s + 1)
  File "/Users/kain/miniconda3/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 820, in binary_op_wrapper
    y = ops.convert_to_tensor(y, dtype=x.dtype.base_dtype, name="y")
  File "/Users/kain/miniconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 639, in convert_to_tensor
    as_ref=False)
  File "/Users/kain/miniconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 704, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/Users/kain/miniconda3/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 113, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/Users/kain/miniconda3/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 102, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/Users/kain/miniconda3/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 370, in make_tensor_proto
    _AssertCompatible(values, dtype)
  File "/Users/kain/miniconda3/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
    (dtype.name, repr(mismatch), type(mismatch).__name__))
TypeError: Expected bool, got 1 of type 'int' instead.

How can I do this? My last layer is sigmoidal, by the way. Any input is greatly appreciated!

[iezepov]

Hi, @lxkain

Try this:

def mse_nan(y_true, y_pred):
    index = ~K.tf.is_nan(y_true)
    y_true = K.tf.boolean_mask(y_true, index)
    y_pred = K.tf.boolean_mask(y_pred, index)
    return K.mean((y_true - y_pred) ** 2)

Reference: https://www.tensorflow.org/api_docs/python/tf/boolean_mask

[lxkain]

@iezepov , more thanks than you can imagine! Really super grateful.