Conv2D transpose output shape using formula

Question

I get [-1,256,256,3] as the output shape using the transpose layers shown below. I print the output shape. My question is specifically about the height and width which are both 256. The channels seem to be the number of filters from the last transpose layer in my code.

I assumed rather simplistically that the formula is this. I read other threads.

H = (H1 - 1)*stride + HF - 2*padding

But when I calculate I don't seem to get that output. I think I may be missing the padding calculation How much padding is added by 'SAME' ?

My code is this.

    linear = tf.layers.dense(z, 512 * 8 * 8)
    linear  = tf.contrib.layers.batch_norm(linear, is_training=is_training,decay=0.88)
    conv = tf.reshape(linear, (-1, 128, 128, 1))
    out = tf.layers.conv2d_transpose(conv, 64,kernel_size=4,strides=2, padding='SAME')
    out = tf.layers.dropout(out, keep_prob)
    out = tf.contrib.layers.batch_norm(out, is_training=is_training,decay=0.88)
    out = tf.nn.leaky_relu(out)
    out = tf.layers.conv2d_transpose(out, 128,kernel_size=4,strides=1, padding='SAME')
    out = tf.layers.dropout(out, keep_prob)
    out = tf.contrib.layers.batch_norm(out, is_training=is_training,decay=0.88)
    out = tf.layers.conv2d_transpose(out, 3,kernel_size=4,strides=1, padding='SAME')
    print( out.get_shape())

Answer 1

Regarding 'SAME' padding, the Convolution documentation offers some detailed explanations (further details in those notes). Especially, when using 'SAME' padding, the output shape is defined so:

# for `tf.layers.conv2d` with `SAME` padding:
out_height = ceil(float(in_height) / float(strides[1]))
out_width  = ceil(float(in_width) / float(strides[2]))

In this case, the output shape depends only on the input shape and stride. The padding size is computed from there to fill this shape requirement (while, with 'VALID' padding, it's the output shape which depends on the padding size)

Now for transposed convolutions... As this operation is the backward counterpart of a normal convolution (its gradient), it means that the output shape of a normal convolution corresponds to the input shape to its counterpart transposed operation. In other words, while the output shape of tf.layers.conv2d() is divided by the stride, the output shape of tf.layers.conv2d_transpose() is multiplied by it:

# for `tf.layers.conv2d_transpose()` with `SAME` padding:
out_height = in_height * strides[1]
out_width  = in_width * strides[2]

But once again, the padding size is calculated to obtain this output shape, not the other way around (for SAME padding). Since the normal relation between those values (i.e. the relation you found) is:

# for `tf.layers.conv2d_transpose()` with given padding:
out_height = strides[1] * (in_height - 1) + kernel_size[0] - 2 * padding_height
out_width  = strides[2] * (in_width - 1) + kernel_size[1] - 2 * padding_width

Rearranging the equations we get

padding_height = [strides[1] * (in_height - 1) + kernel_size[0] - out_height] / 2
padding_width  = [[strides[2] * (in_width - 1) + kernel_size[1] - out_width] / 2

note: if e.g. 2 * padding_height is an odd number, then padding_height_top = floor(padding_height); and padding_height_bottom = ceil(padding_height) (same for resp. padding_width, padding_width_left and padding_width_right)

Replacing out_height and out_width with their expressions, and using your values (for the 1st transposed convolution):

padding = [2 * (128 - 1) + 4 - (128 * 2)] / 2 = 1

You thus have a padding of 1 added on every side of your data, in order to obtain the output dim out_dim = in_dim * stride = strides * (in_dim - 1) + kernel_size - 2 * padding = 256

Answer 2

I drew a diagram for myself based on @Aldream's answer. Maybe useful to visualize. Hope I got that right. But I have to research how and where this padding is applied to get the final shape.