Addons: Check which image ops are compatible with tf.data.Dataset

[facaiy]

Follow up https://github.com/tensorflow/addons/issues/243.

Here is a check list:

• [x] dense_image_warp
• [ ] sparse_image_warp
• [x] interpolate_bilinear
• [x] euclidean_distance_transform
• [x] adjust_hsv_in_yiq
• [x] random_hsv_in_yiq
• [x] mean_filter2d
• [x] median_filter2d
• [x] angles_to_projective_transforms
• [x] matrices_to_flat_transforms
• [x] rotate
• [x] transform

Like what we did in https://github.com/tensorflow/addons/pull/330, feel free to open a pull request for each op:

1. Add test for unknown rank image tensor.
2. Fix it if possible. Otherwise, please notice users that the op is not compatible with unknown rank tensor (shape=None) in its document.

[satishjasthi]

Hello,
Can anyone help me how to test whether a tfa op is compatible with tf.Dataset.
I would like to check for

• dense_image_warp
• interpolate_bilinear
  My question might sound silly, I'm a newbie please help me.

[seanpmorgan]

Hi @satishjasthi welcome to addons! So this colab notebook shows what the issue was (has since been fixed for that op):
https://colab.research.google.com/drive/1ZDhnGrorvSf04wzS-0utztZ6Nnzusvl7

Basically, if an image op can't be used in a tf.dataset mapping it's likely because we're not handling an unknown shape correctly. tf.dataset runs as a subgraph so the dynamic shape is not known until the graph is ran.

A good test to see if it's compatible can be see in previous pull requests that adressed this:
https://github.com/tensorflow/addons/pull/343/files#diff-7137321db62bba0af8193f6f6213fd46R243

Yan added an image util which can be used to infer shape:
https://github.com/tensorflow/addons/blob/master/tensorflow_addons/image/utils.py#L29

Let us know if you run into any issues, and thanks for the help!

[mels630]

regarding the last two ops:

• angles_to_projective_transforms maps a scalar (or batch of N scalars) to an output of dimension (N, 8) representing the equivalent projective transform. It additionally requires image_width and image_height (as scalars or tensors of same dimension as the input angles) to compute the transform.
• matrices_to_flat_transforms similarly maps (N, 3, 3) inputs to (N, 8) outputs (and the inverse flat_transforms_to_matrices from (N, 8) to (N, 3, 3)). For these I don't think there is any sense of image dimensions; only the batch size is not specified.

I don't think there is a sense in which these ops would be suitable for images of unknown dimension as they don't act directly on image data but rather define geometric operations to apply to the image data.

Should we make this issue as done, or is there something to do to make this more transparent to the user?

[WindQAQ]

Hi @mels630, you are right. I tend to close this issue after sparse_image_warp gets merged. https://github.com/tensorflow/addons/pull/446. Thanks for pointing it out!

[us]

Is there any pending to be added or this issue can be closed?

[bhack]

sparse_image_warp was not merged. So check https://github.com/tensorflow/io/issues/839#issuecomment-607362644