I understand the stateful LSTM prediction example in Keras on a single sequence. That example has one sequence of 50k observations.
My questions:
- What if you want to train multiple sequences of 50k observations? Say one that starts/ends at different values and has a slightly different behavior?
- How to modify the example to increase the prediction time step?
- Are LSTMs even any good for that sort of thing?
Fully replicable example with 3 mean-reverting time series and predicting 20 steps out.
# generate random data
import statsmodels.api as sm
import numpy as np
import pandas as pd
cfg_t_total = 25000
cfg_t_step = 20
cfg_batch_size = 100
np.random.seed(12345)
arparams = np.array([.75, -.25])
maparams = np.array([.65, .35])
ar = np.r_[1, -arparams] # add zero-lag and negate
ma = np.r_[1, maparams] # add zero-lag
y0 = sm.tsa.arma_generate_sample(ar, ma, cfg_t_total)
y1 = sm.tsa.arma_generate_sample(ar, ma, cfg_t_total)
y2 = sm.tsa.arma_generate_sample(ar, ma, cfg_t_total)
df=pd.DataFrame({'a':y0,'b':y1,'c':y2})
df.head(100).plot()
df.head(5)
# create training data format
X = df.unstack()
y = X.groupby(level=0).shift(-cfg_t_step)
idx_keep = ~(y.isnull())
X = X.ix[idx_keep]
y = y.ix[idx_keep]
from keras.models import Sequential
from keras.layers import Dense, LSTM
# LSTM taken from https://github.com/fchollet/keras/blob/master/examples/stateful_lstm.py
# how to do this...?!
print('Creating Model')
model = Sequential()
model.add(LSTM(50,
batch_input_shape=(cfg_batch_size, cfg_t_step, 1),
return_sequences=True,
stateful=True))
model.add(LSTM(50,
batch_input_shape=(cfg_batch_size, cfg_t_step, 1),
return_sequences=False,
stateful=True))
model.add(Dense(1))
model.compile(loss='mse', optimizer='rmsprop')
model.fit(X, y, batch_size=cfg_batch_size, verbose=2, validation_split=0.25, nb_epoch=1, shuffle=False)