Differents outputs from predictions using Tensorfl

Question

I am caught in a problem here when I try to take the predictions from my training model. The scenario is: I train a neural network model to learn and classify pictures using Tensorflow. When I train in Gcloud, it returns a different results that when I train locally. Even using the same OS, libraries and code, it returns differents outputs.

Some questions came on my mind:

The data I am talking about is the pictures I am using to train.

I) Considering that you are in the same machine, every train of same data will be different somehow?

II) The act of train in different machines with same data makes the predictions returns different outputs?

Thanks!

Answer 1

Training neural networks is a stochastic process and you will get slightly different results each time you train. For example, the order in which you feed the data will affect the final learned weights. It depends on how you configure your model in TensorFlow but usually the input is shuffled so that each run processes the images in a different order.

Also is your model actually converging?