Keras reports TypeError: unsupported operand type(

Question

I'm a beginner in Keras and just write a toy example. It reports a TypeError. The code and error are as follows:

Code:

inputs = keras.Input(shape=(3, ))

cell = keras.layers.SimpleRNNCell(units=5, activation='softmax')
label = keras.layers.RNN(cell)(inputs)

model = keras.models.Model(inputs=inputs, outputs=label)
model.compile(optimizer='rmsprop',
              loss='mae',
              metrics=['acc'])

data = np.array([[1, 2, 3], [3, 4, 5]])
labels = np.array([1, 2])
model.fit(x=data, y=labels)

Error:

Traceback (most recent call last):
    File "/Users/david/Documents/code/python/Tensorflow/test.py", line 27, in <module>
        run()
    File "/Users/david/Documents/code/python/Tensorflow/test.py", line 21, in run
        label = keras.layers.RNN(cell)(inputs)
    File "/Users/david/anaconda3/lib/python3.6/site-packages/tensorflow/python/keras/layers/recurrent.py", line 619, in __call__
...
    File "/Users/david/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py", line 473, in __call__
        scale /= max(1., (fan_in + fan_out) / 2.)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

So how can I deal with it?

Answer 1

The input to a RNN layer would have a shape of (num_timesteps, num_features), i.e. each sample consists of num_timesteps timesteps where each timestep is a vector of length num_features. Further, the number of timesteps (i.e. num_timesteps) could be variable or unknown (i.e. None) but the number of features (i.e. num_features) should be fixed and specified from the beginning. Therefore, you need to change the shape of Input layer to be consistent with the RNN layer. For example:

inputs = keras.Input(shape=(None, 3))  # variable number of timesteps each with length 3
inputs = keras.Input(shape=(4, 3))     # 4 timesteps each with length 3
inputs = keras.Input(shape=(4, None))  # this is WRONG! you can't do this. Number of features must be fixed

Then, you also need to change the shape of input data (i.e. data) as well to be consistent with the input shape you have specified (i.e. it must have a shape of (num_samples, num_timesteps, num_features)).

As a side note, you could define the RNN layer more simply by using the SimpleRNN layer directly:

label = keras.layers.SimpleRNN(units=5, activation='softmax')(inputs)

Answer 2

I think @today's answer is very clear. However, not complete. The key thing here is that, if your input doesn't contain num_features, you have to make a Embedding layer next to the input.

So if you use:

inputs = keras.Input(shape=(3,))
embedding = Embedding(voc_size, embed_dim, ..)
X = embedding(inputs)

it also works.