I have a tensor that needs to predict the next element in a sequence with a tensorflow LSTM/RNN, while taking into account the previous 5 elements. What should I feed into X and Y?
From 1 2 3 4 5, I want to predict 6
Suppose my input sequence X is:
X = 1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
...
Would my Y be:
Y = 2 3 4 5 6
7 8 9 10 11
12 13 14 15 16
... ?
Or should I feed it:
X = 1 2 3 4 5
2 3 4 5 6
3 4 5 6 7
....
Would my Y be:
Y = 6
7
8
... ?
Or does TensorFlow do this automatically?
I am using the first approach now, inspired by a tutorial, with:
x = tf.placeholder(tf.int32, [None, num_steps], name='input_placeholder')
y = tf.placeholder(tf.int32, [None, num_steps], name='labels_placeholder')
rnn_outputs = tf.reshape(rnn_outputs, [-1, state_size])
y_reshaped = tf.reshape(y, [-1])
logits = tf.matmul(rnn_outputs, W) + b
predictions = tf.nn.softmax(logits)
total_loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=y_reshaped))
If I ask for a prediction (in the actual code the time steps is 16 and number of classes is 14313, sorry for that):
prevSlice = np.array([[1, 2 , 3 , 4, 5, 6 ,7, 8, 9 ,10, 11, 12, 13, 14, 15, 16]], dtype=np.string_)
feed_dict={g['x']: prevSlice}
preds, state = sess.run([g['preds'],g['final_state']], feed_dict)
I get 15 predictions too many. Or how should I interpret these? I don't need predictions for the next 16 slices, just for the 1 next.