Cntk: Reader that supports multiple input nodes?

[LowerySJ]

Hello, there don't seem to be any examples with readers that support multiple input nodes. Is this the case? Does the current release of CNTK support this, even if there are no examples? Are there any plans for this kind of feature?

Thank you and have a good summer.

[zpbappi]

Just wondering if you have the same number of samples for all the input nodes. If that is the case, why not combine them together in a single input so that CNTKTextFormatReader can understand. Then you can split the input into different slices and design the network as you want using BrainScript or NDL. #668 explains what I am trying to say.

[cmarschner]

Hi,

the CNTKTextFormatReader supports multiple input nodes. See here: https://github.com/Microsoft/CNTK/wiki/CNTKTextFormat-Reader
You can see an example e.g. here, https://github.com/Microsoft/CNTK/blob/master/Tests/EndToEndTests/Text/SequenceClassification/Config/seqcla.cntk

Clemens

From: Stephen J. Lowery [mailto:[email protected]]
Sent: Thursday, July 7, 2016 12:01 AM
To: Microsoft/CNTK [email protected]
Subject: [Microsoft/CNTK] Reader that supports multiple input nodes? (#650)

Hello, there don't seem to be any examples with readers that support multiple input nodes. Is this the case? Does the current release of CNTK support this, even if there are no examples? Are there any plans for this kind of feature?

Thank you and have a good summer.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/Microsoft/CNTK/issues/650, or mute the threadhttps://github.com/notifications/unsubscribe/ALtRQVFZcYn1ZMqbvcBdriTkWqZZ8CaDks5qTCWBgaJpZM4JGjAO.

[LowerySJ]

Hi, guys. Thanks for your help.

For the reference of others, I'm writing out my problem more clearly:

My items (the entities that I'm trying to classify) are represented by a set of images. Each image is from a distinct channel (RGB + others). It's looking like my easiest solution will be to modify ImageReader to take in the multiple images through OpenCV, then stack them into a single cv::mat object and pass it into the CNTK neural network at a single input node. From that single input node, I'll divide the data as needed using slice().

If that doesn't work, I'll consult CNTKTextFormatReader for help with altering ImageReader to accommodate these requirements.

I'll write out my final sol'n later.

Regards

[eldakms]

If your sets are of predefined size, you can also use composition of image deserializers, each deserializer providing its own input. For more information please see here: https://github.com/Microsoft/CNTK/wiki/Deserializers-and-Transforms
In the same way as the config there combines HTK and MLF Deserializers, you can combine several ImageDeserializer s.

[LowerySJ]

Hi, eldakms, thanks for your help. An attempt is attached. I'm just trying to read in the same image twice, sum the pixels of the two identical image and divide their values by 2 (yielding the original image) and put the original image through a simple network from the CIFAR-10 examples.

This attempt executes fine with a single image reader using the new syntax, with an error of 40% -- network error is not important now. Also, I had to modify Brainscript macros (Shared.bs) to make this. _Also: Github made me rename the files as .txts_

_But with 2 image reader deserializers this simple attempt crashes CNTK_. I will try debugging it tonight.

Is this new feature (multiple deserializers) implemented for ImageReader? An example would be greatly appreciated.

Thanks and regards

01_ConvBS4.txt
Shared.bs.txt

[LowerySJ]

Ok, so ImageReader won't even let me rename the input nodes to anything other than "feature" and "labels". When I try to run a simple BS network with a single ImageReader input, but with input nodes renamed to "feature1" and "labels1", I get

` [CALL STACK]

Microsoft::MSR::CNTK::DataReader:: GetMinibatch
- Microsoft::MSR::CNTK::DataReaderHelpers::GetMinibatchIntoNetwork
- Microsoft::MSR::CNTK::SGD:: TrainOneEpoch
- Microsoft::MSR::CNTK::SGD:: TrainOrAdaptModel
- Microsoft::MSR::CNTK::SGD:: Train
- DoTrainMicrosoft::MSR::CNTK::ConfigParameters,float
- DoCommands
- wmainOldCNTKConfig
- wmain1
- wmain
- __tmainCRTStartup
- BaseThreadInitThunk
- RtlUserThreadStart

EXCEPTION occurred: Could not map input 'features1' to the reader. Reader outputs only ["features", "labels"]. `

I'm trying to do this because distinct input node names are necessary for multiple input nodes, of course.

Looks like I'm going to have to modify ImageReader... D'oh!

I'm closing this issue and opening one that is specifically about CNTK's current lack of support for composition of ImageReaders for multiple input nodes. Hopefully this is in good form and helps people to better search for and resolve this issue.

[LowerySJ]

Continued at #694

[dongyu888]

In the config file’s reader section, you should rename feature to feature1 and label to label1 to match what’s inside your model file.