ValueError: Error when checking input: expected ls

Question

I am struggling with the LSTM input_shape thing. Here I made a simple LSTM network that should be trained, to double the Input.

from keras.models import Sequential
from keras.layers import LSTM, Dense
import numpy as np

X = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
y = np.array([2, 4, 6, 8, 10, 12, 14, 16, 18, 20])

data_dim = 1
timesteps = 8

model = Sequential()
model.add(LSTM(32, return_sequences=True, input_shape=(timesteps, data_dim)))
model.add(LSTM(32, return_sequences=True))
model.add(Dense(10, activation='softmax'))

model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])

model.fit(X,y, batch_size=10, epochs=1000)

But there comes always this error message: ValueError: Error when checking input: expected lstm_1_input to have 3 dimensions, but got array with shape (10, 1) What am I doing wrong? Can someone explain me the input_shape thing. Kind regards. Niklas

Answer 1

There are a number of things that are wrong with your code.

1) You want to have a regression problem. At the last layer, softmax will squash numbers to range 0 and 1. You need a linear activation.

2) Consequently, the loss function should be mean_square_error.

3) The shape of your target y dictates that the size of the output layer at each time step should be 1 and not 10.

4) Shape of input and output arrays for an LSTM layer should be (batch_size, time_step, dim).

5) Time steps defined in LSTM layer and that of the input data should be the same.

I incorporated these changes in your code:

from keras.models import Sequential
from keras.layers import LSTM, Dense
import numpy as np

X = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
y = np.array([2, 4, 6, 8, 10, 12, 14, 16, 18, 20])

X = X.reshape(1,10,1)
y = y.reshape(1,10,1)

data_dim = 1
timesteps = 10

model = Sequential()
model.add(LSTM(32, return_sequences=True, input_shape=(timesteps, data_dim)))
model.add(LSTM(32, return_sequences=True))
model.add(Dense(1, activation='linear'))

print(model.summary())

model.compile(loss='mean_squared_error', optimizer='rmsprop', metrics=['accuracy'])

model.fit(X,y, batch_size=1, epochs=1000)