Tensorflow-examples: bidirectional RNN states never updated?

Created on 28 Apr 2016 · 6Comments · Source: aymericdamien/TensorFlow-Examples

Hopefully this is not stupid. In the Tensorflow tutorial, the states of the RNN are updated after each time-step and used as the states for the next time-step

output, state = lstm(current_batch_of_words, state)

In yours the state of the bidirectional RNN seems to just be discarded. Is this the case, or am I missing something? If so, why?

Also, a small detail; you use outputs[-1] for

return tf.matmul(outputs[-1], _weights['out']) + _biases['out']

which corresponds according to the Tensorflow API

(outputs, output_state_fw, output_state_bw) = outputs

So shouldn't this be

return tf.matmul(outputs[0], _weights['out']) + _biases['out']

Instead?

Source

nrf171

All 6 comments

The old version of Tensorflow didn't returned the states, so output[-1] is fine (see https://github.com/tensorflow/tensorflow/blob/r0.7/tensorflow/python/ops/rnn.py#L323).
However in newer version, it returns a tuple instead, so as you said, you need to change
outputs = rnn.bidirectionnal_rnn(... by outputs, _, _ = rnn.bidirectionnal_rnn(...

aymericdamien on 28 Apr 2016

👍1

@aymericdamien Speaking of BDLSTM example, is there any particular reason for n_hidden to be 128? I'm hesitant to open an issue for this question since it's likely a triviality, but the number puzzles me.

zandaqo on 2 Jun 2016

I am also interested in the problem. maybe base on experience? @zandaqo

coderLong on 3 Jun 2016

👍1

There is no particular reason, it depends on your data. A recommendation is to use multiple of 32 as it may speedup a little the computation when using float32.

aymericdamien on 3 Jun 2016

👍1

https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3_NeuralNetworks/bidirectional_rnn.py

ImportError: This module is deprecated. Use tf.nn.rnn_* instead.