Keras: lstm_1_input to have 3 dimensions, but got array with shape (2, 1)

[damianoporta]

Hello,
i have this code: https://gist.github.com/anonymous/03801941c2ccb77242faa21f793daf28
It is working perfecly but i have a problem if i need to change the timesteps of specific sentences.
I read that shapes of X and Y must be the same (right?), then i tried with:

X = np.array([
    [  # Sentence 1
        [1, 0, 0, 0, 0, 0],  # Character 1
        [1, 0, 0, 0, 0, 0],  # Character 2
        [1, 0, 0, 0, 0, 0],  # Character 3
        [1, 0, 0, 0, 0, 0],  # Character 4
    ],
    [  # Sentence 2
        [0, 0, 0, 0, 0, 1],  # Character 1
        [0, 0, 0, 0, 0, 1],  # Character 2
        [0, 0, 0, 0, 0, 1],  # Character 3
    ],
])

Y = np.array([
    [  # Sentence 1
        [1, 0, 0, 0, 0, 0],  # Character 1
        [1, 0, 0, 0, 0, 0],  # Character 2
        [1, 0, 0, 0, 0, 0],  # Character 3
        [1, 0, 0, 0, 0, 0],  # Character 4
    ],
    [  # Sentence 2
        [0, 0, 0, 0, 0, 1],  # Character 1
        [0, 0, 0, 0, 0, 1],  # Character 2
        [0, 0, 0, 0, 0, 1],  # Character 3
    ],
])

As you can see i have added ONE timestep inside the first sentence in X and Y. ( Character 4)
I am using batch_size = 1 so it should select on sentence at a time (right?), but it seems not possible.

The error is:

Traceback (most recent call last):
  File "/home/damiano/corrector.py", line 40, in <module>
    model.fit(X, Y, batch_size=1, epochs=250)
  File "/usr/local/lib/python3.5/dist-packages/keras/models.py", line 863, in fit
    initial_epoch=initial_epoch)
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 1358, in fit
    batch_size=batch_size)
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 1234, in _standardize_user_data
    exception_prefix='input')
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 128, in _standardize_input_data
    str(array.shape))
ValueError: Error when checking input: expected lstm_1_input to have 3 dimensions, but got array with shape (2, 1)

How should i train these sequences?

[td2014]

@damianoporta. If you look at the shapes of each via numpy shape, are you getting the dimensions you expect? This might help with running it down. Thanks

[damianoporta]

@td2014 I get (2,) that is correct of course but should keras not convert each array on the X/Y lists into np,array somehow?
Something like:

X = np.array([
    np.array([  # Sentence 1
        [1, 0, 0, 0, 0, 0],  # Character 1
        [1, 0, 0, 0, 0, 0],  # Character 2
        [1, 0, 0, 0, 0, 0],  # Character 3
        [1, 0, 0, 0, 0, 0],  # Character 4
    ]),
    np.array([  # Sentence 2
        [0, 0, 0, 0, 0, 1],  # Character 1
        [0, 0, 0, 0, 0, 1],  # Character 2
        [0, 0, 0, 0, 0, 1],  # Character 3
    ]),
])

However does it mean that all the sequences must have the same length for input/output?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[quintinali]

I encountered the same problem. Could you please tell me how did you solve it? @damianoporta Thanks.