Keras: 'fit_generator' with multiple inputs

[tperol]

The fit method allows to train the model with multiple inputs by passing a dictionary.

How could I train a model using fit_generator and multiple inputs. It seems that for now, this method only accepts a tuple (X_train, y_train) ? Thanks !

[braingineer]

Hi @tperol ,

fit_generator ultimately calls train_on_batch which allows for x to be a dictionary.

Also, it could be a list, in which casex is expected to map 1:1 to the inputs defined in Model(input=[in1, ...], ...)

[tperol]

thanks !

[fchollet]

Note that x could also be a list (one input array per input). Both are allowed.

[randhawp]

Hi - getting this Exception: The model expects 2 input arrays, but only received one array. Found: array with shape (10, 3, 32, 32)

How to pass two inputs with fit_generator

code snippet below

model1 = Sequential()
model1.add(Convolution2D(32, 4, 4, activation='tanh', border_mode='same',input_shape=(3, img_width, img_height)))
......
model2 = Sequential()
model2.add(Convolution2D(32, 3, 3, activation='tanh', border_mode='same',input_shape=(3, img_width, img_height)))
...

model = Sequential()
model.add(Merge([model1,model2],mode='concat'))
model.add(Flatten())
model.add(Dense(128, activation='tanh', W_constraint=maxnorm(3)))
......
sgd = SGD(lr=0.03, momentum=0.03, decay=0.001, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

train_datagen = ImageDataGenerator( horizontal_flip=True)

test_datagen = ImageDataGenerator(horizontal_flip=True)

train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=epoch,
class_mode='categorical')

validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=epoch,
class_mode='categorical')

filepath="weights-improvement-{epoch:02d}-{val_acc:.2f}.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True,mode='max')
callbacks_list = [checkpoint,checkpoint]

model.fit_generator(
train_generator,
samples_per_epoch=nb_train_samples,
nb_epoch=nb_epoch,
validation_data=validation_generator,
nb_val_samples=nb_validation_samples,
callbacks=callbacks_list)

[joelthchao]

Note that x could also be a list (one input array per input). Both are allowed.

Is it possible to extends to any iterable object such as tuple?
In training.py, it forces multiple inputs to be a list.

if isinstance(x, list):
    batch_size = x[0].shape[0]

[Duke-ding]

I also face this problem ?How to pass two inputs with fit_generator.

[Ricocotam]

There's at least an easy way, it's to extend the class :

class MultipleInputData(ImageDataGenerator):
    """docstring for MultipleOutputData."""
    def __init__(self, get_multiple, *args, **kwargs):
        super(MultipleOutputData, self).__init__(*args, **kwargs)
        self.get_multiple = get_multiple

    def random_transform(self, x, seed=None):
        X = get_multiple(x)
        X = list(map(super(MultipleOutputData, self).random_transform), X, [seed]*len(X))
        return X

I guess it should work, I didn't try though. Ofc the "get_multiple" can be coded in the random_transform.

[lseongjoo]

Is it possible to extends to any iterable object such as tuple?
In training.py, it forces multiple inputs to be a list.

@joelthchao I was looking for the cause of a problem regarding multi input fit_generator. My generator yields inputs as a tuple. Changing it to list resolves the problem. For a pythonist, a person would assume that making it as list or tuple would be almost identical.