Keras: Something goes wrong when i use functional api

[ssdf93]

It is OK, when i run the following code
``

model = Sequential()
print(dicSize,length,embedding.shape)
model.add(Embedding(dicSize, length, dropout=0.5,input_length=maxlen,mask_zero=True,weights=[embedding]))
model.add(GRU(length*2,return_sequences=True, dropout_W=0.5, dropout_U=0.1))
model.add(Dropout(0.3))
model.add(LSTM(length, dropout_W=0.5, dropout_U=0.1))
model.add(Dropout(0.3))
model.add(Dense(200))
model.add(PReLU())
model.add(Dropout(0.5))
model.add(Dense(5))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='sgd',metrics=['accuracy'])
model.fit(train_features,train_y,batch_size=128,nb_epoch=3,validation_split=0.3)
ans=model.predict(test_features,batch_size=128)

``

But goes wrong when i change it to functional api

`

inputs=Input(shape=(maxlen,))
embeds=Embedding(dicSize,length,input_length=maxlen,mask_zero=True,weights=[embedding])(inputs)
grus=GRU(length*2,return_sequences=True, dropout_W=0.5, dropout_U=0.1)(embeds)
lstms=LSTM(length, dropout_W=0.5, dropout_U=0.1)(grus)
denses=Dense(200,dropout=0.3,activation='relu')(lstms)
outputs=Dense(5,activation='softmax')(denses)
model=Model(input=inputs,output=outputs)
model.compile(loss='categorical_crossentropy', optimizer='sgd',metrics=['accuracy'])
model.fit(train_features,train_y,batch_size=128,nb_epoch=3,validation_split=0.3)

`

`

Traceback (most recent call last):
File "/Users/victor/Desktop/kaggle_for_fun/Sentiment Analysis on Movie Reviews/lstm_functional.py", line 155, in
embedding_methods(train_features,test_features,train_y,train,test)
File "/Users/victor/Desktop/kaggle_for_fun/Sentiment Analysis on Movie Reviews/lstm_functional.py", line 86, in embedding_methods
embeds=Embedding(dicSize,length,input_length=maxlen,mask_zero=True,weights=[embedding])(inputs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras/engine/topology.py", line 485, in call
self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras/engine/topology.py", line 543, in add_inbound_node
Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras/engine/topology.py", line 148, in create_node
output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras/layers/embeddings.py", line 133, in call
out = K.gather(W, x)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras/backend/theano_backend.py", line 162, in gather
return reference[indices]
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/tensor/var.py", line 503, in getitem
return self.take(args[axis], axis)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/tensor/var.py", line 535, in take
return theano.tensor.subtensor.take(self, indices, axis, mode)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/tensor/subtensor.py", line 2392, in take
return take(a, indices.flatten(), axis, mode).reshape(shape, ndim)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/tensor/subtensor.py", line 2370, in take
return advanced_subtensor1(a, indices)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/gof/op.py", line 611, in call
node = self.make_node(_inputs, *_kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/theano/tensor/subtensor.py", line 1687, in make_node
raise TypeError('index must be integers')
TypeError: index must be integers
`

[joelthchao]

It's interesting that Sequential and fucntional model have different behaviors on the same layer.

# Input need to specify dtype, that's why you got TypeError
inputs = Input(shape=(maxlen,)), dtype='int32')
# ...
# Also Dense has no argument dropout
denses = Dense(200, activation='relu')(lstms)

[braingineer]

I was going to comment and say it was baffling as to why it's happening, but I tracked the issue.

It's actually a slightly unfair comparison. If you start with the Embedding layer as the first layer in the functional approach (and necessarily passing it the relevant shape information), then it would have worked.

Embedding defaults to int32 dtype explicitly (it sets it in its constructor). Sequential defaults to the dtype of its first layer. Input defaults to the abstract Layer's default: float32.