Dali: Return more than two tensors using the DALIIterator of the TF plugin

[wellescastro]

Hi!

I'm trying to use a TFRecordPipeline with the tensorflow plugin. The thing is that I have two different image batches plus the labels.

For the pipeline, I have the following graph definition:
`

def define_graph(self):

    inputs = self.input()

    modality1 = self.decode(inputs["modality1"])
    modality2 = self.decode(inputs["modality2"])

    label = self.decode_grayscale(inputs["label"])

    crop_pos_x = self.uniform()
    crop_pos_y = self.uniform()
    modality1 = self.cmnp(modality1, crop_pos_x = crop_pos_x,
                       crop_pos_y = crop_pos_y)

    modality2 = self.cmnp(modality2, crop_pos_x = crop_pos_x,
                crop_pos_y = crop_pos_y)

    return modality1, modality2, label

`
If I just call the pipeline.build() and then .run(), it works properly outside Tensorflow. However, when I try to integrate with a Tensorflow graph using the _DALIIterator()_ I can only get the first two tensors since it was designed to return images and labels (in the description of the DALIIterator method it is said that it "Returns 2 TensorFlow tensors with data and label."). Is there a workaround to make this work? Maybe some dali op that make it possible to concatenate the two augmented images and them return them as a single image in the define_graph function.

[JanuszL]

Hi,
This is a very good questions. We have not yet considered such use case you described.
Regarding TF it requires that operators have predefined number of inputs and outputs so it is not so straightforward to make your idea working. Concatenation of images may work, but only if two images are with the same size unless we pad with 0 smaller one.
We will check if it possible for example to use queue based approach to return variable number of output tensors for each iteration.
Tracked as DALI-241.

[JanuszL]

265 should address this problem.