Keras: predict_generator function does not support samples with different lengths

[Shuailong]

Hi, I am doing a sequence labelling task(like PosTagging) using Keras + theano. The model I use is BiLSTM. For training, I did not do padding. Instead, I just used the batch size of 1, and did not specify the input_length parameter. It works well for training and evaluation(for both I use the generator version to deal with the memory problem). However, when I want to do prediction, I have the following errors:

Predicting...
Traceback (most recent call last):
  File "predict.py", line 81, in <module>
    main()
  File "predict.py", line 55, in main
    val_samples=samples)
  File "/home/user/.local/lib/python2.7/site-packages/keras/models.py", line 1001, in predict_generator
    pickle_safe=pickle_safe)
  File "/home/user/.local/lib/python2.7/site-packages/keras/engine/training.py", line 1749, in predict_generator
    all_outs[i][processed_samples:(processed_samples + nb_samples)] = out
ValueError: could not broadcast input array from shape (1,42,1310) into shape (1,10,1310)

I think the reason is the first sentence have a length of 42 words, while the second sentence have a length of 10 words.

My question is, why it works well for training and evaluation, but does not work for prediction?

The code I think relevant is here

    print('\nPredicting...')

    samples = 1 # only 1 work for now
    prob = model.predict_generator(data_generator(X_test, X_test_feats, y_test, tag_size, shuffle=False),
                                   val_samples=samples)
    result = prob.argmax(axis=-1)

[Shuailong]

Could you kindly have a look at this problem? @fchollet

[seanmullane]

This still seems to be a problem in Keras using Tensorflow 1.12.0.

When I run model.predict_generator I get results only if each X returned by the generator has the same length. Otherwise it throws this error:

ValueError: all the input array dimensions except for the concatenation axis must match exactly