Keras error: expected dense_input_1 to have 3 dime

Question

I am trying out a simple model in Keras, which I want to take as input a matrix of size 5x3. In the below example, this is specified by using input_shape=(5, 3) when adding the first dense layer.

from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.optimizers import Adam
import numpy as np


model = Sequential()
model.add(Dense(32, input_shape=(5, 3)))
model.add(Activation('relu'))
model.add(Dense(32))
model.add(Activation('relu'))
model.add(Dense(4))


adam = Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)
model.compile(loss='mean_squared_error', optimizer=adam)


x = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12], [13, 14, 15]])


y = model.predict(x)

However, when I run the code, the model.predict() function gives the following error:

ValueError: Error when checking : expected dense_input_1 to have 3 dimensions, but got array with shape (5, 3)

But I don't understand the error. The shape of x is (5, 3), and this is exactly what I have told the first dense layer to expect as input. Why is it therefore expecting three dimensions? It seems that this may be something to do with the batch size, but I thought that input_shape is only referring to the shape of the network, and is nothing to do with the batch size...

Answer 1

When you add a Keras layer using the add() method from the Sequential API, the parameter input_shape only cares about the shape of your input data, regardless of the batch_size. Therefore in your case you are correct in stating to your model that you want an input shape of (5, 3) by specifying the argument input_shape = (5, 3).
HOWEVER, Keras always expect you to provide your input data in batches, even if you choose the batch size to be 1. This means that you need to add an extra dimension to your input in order to make it three-dimensional with the first dimension being the batch_size. You can do this like this:

x = x[None,:,:]

I think this should solve your problem

Answer 2

The problem lies here:

model.add(Dense(32, input_shape=(5, 3)))

it should be:

model.add(Dense(32, input_shape=(3,)))

This first example dimension is not included in input_shape. Also because it's actually dependent on batch_size set during network fitting. If you want to specify you could try:

model.add(Dense(32, batch_input_shape=(5, 3)))

EDIT:

From your comment I understood that you want your input to have shape=(5,3) in this case you need to:

1. reshape your x by setting:

   x = x.reshape((1, 5, 3))
   

   where first dimension comes from examples.

2. You need to flatten your model at some stage. It's because without it you'll be passing a 2d input through your network. I advice you to do the following:

   model = Sequential()
   model.add(Dense(32, input_shape=(5, 3)))
   model.add(Activation('relu'))
   model.add(Dense(32))
   model.add(Activation('relu'))
   model.add(Flatten())
   model.add(Dense(4))