Keras: Multiple instance learning

[xypan1232]

Hi all,

i want to implement a multiple instance learning using keras. I first separate the data into bags, each bag has different number of instances. Could I ask is it possible to consider a bag as a batch, and create an objective function which take the argmax value of predicted probability for the batch with the true label for this bag. any suggestion about how to implement it? thanks

Please make sure that the boxes below are checked before you submit your issue. If your issue is an implementation question, please ask your question on StackOverflow or join the Keras Slack channel and ask there instead of filing a GitHub issue.

Thank you!

• [ ] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

• [ ] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

• [ ] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

• [ ] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

[unrealwill]

Hello,

I'm not sure I understand exactly what you are trying to do, but you can probably use the
TimeDistributed Wrapper https://keras.io/layers/wrappers/#timedistributed
Then take the argmax along the time dimension.

For handling the different number of instances, you will have to implement some kind of masking.

[xypan1232]

Thanks a lot.
I want to implement a multiple instance learning with cnn, each bag have several instances and only have one label.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[bw4sz]

@xypan1232 did you make progress on this? What was your use case?

[zxhuang97]

This can be done by TimeDistributed and GlobalMaxPooling

[jonnor]

@Killer97 can you provide or refer to an example of this? I could not get it plugged together correctly...

[zxhuang97]

@Killer97 can you provide or refer to an example of this? I could not get it plugged together correctly...

Say you have a backbone model works for an input of (16,112,112,3) and you want to use it under Multiple instance learning. You just need to wrap it with TimeDistributed and use some kind of 'selector' to pick the salient result. Here's an example when there're 32 instances in each bag.

input = Input(shape=(32,16,112,112,3))
x = BatchNormalization()(input)
x = TimeDistributed(backbone)(x)
x = GlobalMaxPool1D()(x)
model = Model(input,x)

[adelabdelli]

@killer97 can you provide or refer to an example of this? I could not get it plugged together correctly...

Say you have a backbone model works for an input of (16,112,112,3) and you want to use it under Multiple instance learning. You just need to wrap it with TimeDistributed and use some kind of 'selector' to pick the salient result. Here's an example when there're 32 instances in each bag.

input = Input(shape=(32,16,112,112,3))
x = BatchNormalization()(input)
x = TimeDistributed(backbone)(x)
x = GlobalMaxPool1D()(x)
model = Model(input,x)

What is the number 16 role here ? or what it represent ?