Value Error problem with multicell Dimensions must

Question

I am working with python 3.6.5 and tensorflow 1.8.0 Nr of neurons are 10 at the moment, input in this example is 3

I have already build a recurrent neuronal network and now wanted to improve it. I need some help!

Here is a little excerpt of the code to reproduce my error: You can also replace BasicRNN by LSTM or GRU to get the other messages.

import numpy      as np
import tensorflow as tf  
batch_size = 10
nr_inputs = 3  
nr_outputs  = 4   
nr_steps = 4             
nr_layers  = 2
def  mini_batch ( Xdata, ydata, batch_size ) :  
    global  global_counter
    result      = None
    Xbatch      = np.zeros( shape=[batch_size, nr_steps,  nr_inputs],  dtype = np.float32 )
    ybatch      = np.zeros( shape=[batch_size, nr_outputs],            dtype = np.float32 )
    return  Xbatch, ybatch
X = tf.placeholder( tf.float32, [ None, nr_steps,    nr_inputs ] )
y = tf.placeholder( tf.float32, [ None, nr_outputs ]             )
neurons = tf.contrib.rnn.BasicRNNCell(num_units = 10)
neurons = tf.contrib.rnn.MultiRNNCell( [neurons] * nr_layers, state_is_tuple = True )
X_train = np.zeros( shape=[1000, nr_steps,  nr_inputs],  dtype = np.float32 )
y_train = np.zeros( shape=[1000, nr_outputs],            dtype = np.float32 )
X_test      = np.zeros( shape=[1000,  nr_steps,  nr_inputs],  dtype = np.float32 )
y_test      = np.zeros( shape=[1000,  nr_outputs],            dtype = np.float32 )
rnn_outputs, rnn_states = tf.nn.dynamic_rnn( neurons, X, dtype=tf.float32 )
logits = tf.contrib.layers.fully_connected( inputs        = rnn_states, num_outputs = nr_outputs, activation_fn = None )
xentropy    = tf.nn.sigmoid_cross_entropy_with_logits( labels = y,  logits = logits )
loss        = tf.reduce_mean( xentropy )
optimizer   = tf.train.AdamOptimizer( learning_rate = 0.01 )
training_op = optimizer.minimize( loss )
init         = tf.global_variables_initializer()
with tf.Session() as sess :
    init.run()
    global_counter = 0
    for epoch in range(100) :
        for iteration in range( 4) :
            X_batch, y_batch = mini_batch ( X_train, y_train, batch_size )
            sess.run( training_op, feed_dict={ X : X_batch,  y : y_batch } ) 
        loss_train = loss.eval( feed_dict={ X : X_batch,  y : y_batch } )
        loss_test  = loss.eval( feed_dict={ X : X_test,   y : y_test  } )
    sess.close()

I was trying this neurons = tf.contrib.rnn.MultiRNNCell([neurons]*nr_layers, state_ist_tuple = True)

and received the error

ValueError: Dimensions must be equal, but are 20 and 13 for 'rnn/.../MatMul1'(op 'MatMul') with input shapes [?,20], [13, 10] for a tf.contrib.rnn.BasicRNNCell(num_units = nr_neurons)

with input shapes [?,20], [13, 20] for a tf.contrib.rnn.GRUCell(num_units = nr_neurons)

and

with input shapes [?,20], [13, 40] for a tf.contrib.rnn.BasicLSTMCell(num_units = nr_neurons, state_is_tuple = True)

is there an error in the MatMul_1? Has anyone ever had similar problems? Thank you so much!

Answer 1

Instead of using the BasicRNNCell instance multiple times,one instance per RNN layer should be created - for example in this way:

neurons = [tf.contrib.rnn.BasicRNNCell(num_units=10) for _ in range(nr_layers)]
neurons = tf.contrib.rnn.MultiRNNCell( neurons, state_is_tuple = True )

In addition, there are other mistakes on your codes.rnn_states is a tuple containing cell state and hidden state, and its shape is ((None,10),(None,10)). I assume you want to use hidden state,replace it:

logits = tf.contrib.layers.fully_connected( inputs = rnn_states[1], num_outputs = nr_outputs, activation_fn = None )

That's OK!