Keras: Variable length RepeatVector

[ptresende]

I'm trying to implement a seq2seq LSTM autoencoder based on the example provided here:

from keras.layers import Input, LSTM, RepeatVector
from keras.models import Model

inputs = Input(shape=(timesteps, input_dim))
encoded = LSTM(latent_dim)(inputs)

decoded = RepeatVector(timesteps)(encoded)
decoded = LSTM(input_dim, return_sequences=True)(decoded)

sequence_autoencoder = Model(inputs, decoded)
encoder = Model(inputs, encoded)

I managed to get this working with zero-padded inputs, but have since found that Input layers can be initialized with None shapes, allowing for variable length inputs. However, I haven't found a way to have the RepeatVector layer dynamically adjust it's "repetition rate" based on the length of the current input.

Is there a way of accomplishing this?

[duolinwang]

Hi,
I have the exact same problem! Have you found a way to deal with it?

[wimthiels]

same problem here. I tried to route the variable length directly, but this doesn't work unfortunately

decoded2 = RepeatVector(encoder_inputs.shape[1])(state_c)

[carmelrabinov]

I was facing the same problem. managed to solve it with this:

def repeat_vector(args):
        layer_to_repeat = args[0]
        sequence_layer = args[1]
        return RepeatVector(K.shape(sequence_layer)[1])(layer_to_repeat)

reconstructed_seq = Lambda(repeat_vector, output_shape=(None, size_of_vector)) ([vector, seq])

[nanonaren]

That works! Seems like black magic. I tried to do it with TimeDistributed... but couldn't make it work. Thanks.

[mshtelma]

@nanonaren Can you please post the whole example with the solution of @carmelrabinov ?
I am trying to integrate it into VAE example with LSTMs, but it is not working (

@carmelrabinov As far as I understand size_of_vector in output_shape=(None, size_of_vector) means the initial size of input multiplied by the length of the sequence, so this is the repeated vector ? In this case it will be also dynamic if each batch has different sequence length.

[carmelrabinov]

@mshtelma Note that the real output_shape of the layer is (batch_size, feature_size, timestep_size) so in this case:

1. batch_size will be computed automatically (so we specify None)
2. size_of_vector = feature_size
3. timestep_size will be computed automatically (depend on length of each batch)

[AshimaChawla]

@ptresende, did you manage to solve this, as even in my case I have variable input lengths and repeat vector with None value throws an error.

[y2kpr]

I have been wrestling with this the past week and @carmelrabinov's solution works! Just to clarify something I was confused on, the parameters [vector, seq] sent to the lambda are decoded and inputs respectively from the question. So to connect it back to the question, the arguments should have been [decoded, inputs] (please correct me if I am wrong here).

[regCode]

I think the arguments are actually [encoded, inputs] so they can match the original code decoded = RepeatVector(timesteps)(encoded). The repeat_vector function only creates a new RepeatVector layer for each case. (similarly correct me if I'm wrong)

[LuCeHe]

@carmelrabinov , I've got a slightly different problem:

d = 2

c = K.random_normal((1, d), dtype=None)
m = Input(shape=(None, 32))

def repeat_vector(args):
   layer_to_repeat = args[0]
   reference_layer = args[1]
   return RepeatVector(K.shape(reference_layer)[0])(layer_to_repeat)

c_batch = Lambda(repeat_vector) ([c, m])

I want c to be an internal state whose initial value can be learned. I want c_batch to be of shape (batchSize, d), but what comes out from repeat_vector is a tensor of shape (1, batchSize, d). When I do right after:

c_batch = Lambda(lambda x: K.squeeze(x, 0))(c_batch)

I get:

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/scratch/celottil/work/MAC_variants/Tests.py", line 497, in <module>
    test_RepeatNoneTimes()
  File "/scratch/celottil/work/MAC_variants/Tests.py", line 479, in test_RepeatNoneTimes
    c_batch = Lambda(lambda x: K.squeeze(x, 0))(c_batch)   #Reshape((-1,1,d,), name='predictions')(c_batch)
  File "/scratch/celottil/.conda/envs/home-dialogue/lib/python3.6/site-packages/keras/engine/base_layer.py", line 474, in __call__
    output_shape = self.compute_output_shape(input_shape)
  File "/scratch/celottil/.conda/envs/home-dialogue/lib/python3.6/site-packages/keras/layers/core.py", line 651, in compute_output_shape
    x = K.placeholder(shape=input_shape)
  File "/scratch/celottil/.conda/envs/home-dialogue/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 517, in placeholder
    x = tf.placeholder(dtype, shape=shape, name=name)
  File "/scratch/celottil/.conda/envs/home-dialogue/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1734, in placeholder
    return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
  File "/scratch/celottil/.conda/envs/home-dialogue/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4922, in placeholder
    shape = _execute.make_shape(shape, "shape")
  File "/scratch/celottil/.conda/envs/home-dialogue/lib/python3.6/site-packages/tensorflow/python/eager/execute.py", line 143, in make_shape
    raise TypeError("Error converting %s to a TensorShape: %s." % (arg_name, e))
TypeError: Error converting shape to a TensorShape: int() argument must be a string, a bytes-like object or a number, not 'Tensor'. 

[LuCeHe]

import keras.backend as K

Le jeu. 21 mars 2019 à 11:35, Vincenzo Bazzucchi [e-mail]
a écrit :

What is K in the snippets?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/keras-team/keras/issues/7949#issuecomment-475279563,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJKmElfO1WTqDD5t3bI3wpWF1PXb_oyyks5vY6aqgaJpZM4PfekV
.

[benam2]

I faced with the same problem here. when I tried @carmelrabinov approach I received this error:
ValueError: Dimensions must be equal, but are 64 and 2000 for 'decoder_lstm/MatMul' (op: 'MatMul') with input shapes: [?,64], [2000,2000].

And this is my structure of model:

inputs = Input(shape=(None, self.EMBED_SIZE), name="input")
encoded_1 = Bidirectional(LSTM(self.LATENT_SIZE), merge_mode="sum", name="encoder_lstm")(inputs)
# decoded = RepeatVector(name="repeater")(encoded_1)
decoded = Lambda(self.repeat_vector, output_shape=(None, self.EMBED_SIZE))([encoded_1, inputs])
decoded = Bidirectional(LSTM(self.EMBED_SIZE, return_sequences=True), merge_mode="sum", name="decoder_lstm")(decoded)
autoencoder = Model(inputs, decoded)

Any idea how to resolve it?

[benam2]

@regCode Could you look at my question above and let me know is there any way I can fix this, please?

[flow2k]

@mshtelma Note that the real output_shape of the layer is (batch_size, feature_size, timestep_size) so in this case:

1. batch_size will be computed automatically (so we specify None)
2. size_of_vector = feature_size
3. timestep_size will be computed automatically (depend on length of each batch)

@carmelrabinov your code assumes the lengths of vectors (number of timesteps) in each batch is the same, correct?

[David-Jianguang-Ran]

Same problem here! Just wanted to post what end up working for me since the backend.shape() method didn't.

`

def repeat_encoding(input_tensors):
    """
    repeats encoding based on input shape of the model
    :param input_tensors: [  tensor : encoding ,  tensor : sequential_input]
    :return: tensor : encoding repeated input-shape[1]times
    """
    sequential_input = input_tensors[1]
    to_be_repeated = K.expand_dims(input_tensors[0],axis=1)

    # set the one matrix to shape [ batch_size , sequence_length_based on input, 1]
    one_matrix = K.ones_like(sequential_input[:,:,:1])

    # do a mat mul
    return K.batch_dot(one_matrix,to_be_repeated)

# then just call it with a list of tensors 
repeated = Lambda(repeat_encoding,name="repeat_vector_dynamic")([encoding,sequence_input])

`

[shairoz]

Been struggling with the same issue for some time, my solution was:

# shape = [K.shape(inp)[k] for k in range(2)]
# emb = Lambda(lambda x: K.repeat(x,shape[1]),output_shape=(None,EmbSize))(vec)

While this worked, I was unable to save the model due to:

{TypeError}can't pickle _thread.RLock objects

using @carmelrabinov solution solved the issue

Keras : 2.2.4
TF: 1.14.0