I have a feed forward network with batch normalization layers in it. the network has to be run multiple times before one backpropogation can be applied. It is used to compute:
a. vanilla logits (vl)
b. logits with noisy inputs (nl)
difference between vl and nl is also part of the cost function
I cant understand how and when to run tf.GraphKeys.UPDATE_OPS. If i run it typically by grouping it with optimizer i think the moving averages will be updated with mean and standard deviation of noisy batches since that is the last run of the network but i need the updates based on vanilla (no noise) run of the network. I can't change the order of different runs of the network as noise is computed using the output of vanilla run of the network
