I have tried to model by attached three layer of ConvLSTM but when I set in the first ConvLSTM return_sequence = False program won't run.
See the model summary Model summary
The model run after I set return_sequence = True in first ConvLSTM layer but if I set return_sequence = False program won't run
The second part in last layer of ConvLSTM when return_sequence=False the output dimension change from 5 to 4 with timestep dimension disappear, why this happened?
If return_sequence is true, it means the LSTM layer will return the full sequence of the output not only the final output. So the input of next layer is still time sequence, it also means the next layer MUST be RNN to handle time sequence.
If return_sequence is false, it means the LSTM layer will ONLY return the final output, which is not a time sequence any more. So the dimension will be reduced one. For your example, it will be changed from 5 to 4. And because the input for next layer is not time sequence anymore, the next layer MUST NOT be RNN any more.
When you set the parameter return_sequence = False, only one vector is returned after that, which does not satisfy the input dimension requirements of the next layer. This causes the program not to run. When you select true, a sequence is returned so that your timestep dimension does not disappear.
