First the broad questions:
- Is it possible to build a conditional graph with tensorflow?
- If yes, do the automated gradient-calcuation and the implemented optimizers work with it?
- Can I access the shape of a tensor and turn it into an integer for using it with an "if" condition and a "for i in range()" loop?
My actual use case is that I want to do a 1D convolution with variable tensor length. For this I first need an if statement that only executes the convolution if the length is greater then one. Then I have a for loop that goes through the tensor for the convolution. The problem is that this code:
for i in range(tf.shape(tensor)[0]):
doesn't work because the range operator needs an integer. Can I turn this somehow into an integer?
In the end I want to train this model with adagrad, either with the automatic differentiation or the already implemented optimiser
Edit:
this is the 1D convolution which will later be the first of two Layers in my Model. The type errors are behind each version of the for loop that triggers one
import tensorflow as tf
import numpy as np
def convolve(s, Tl, Tr, b):
if (tf.shape(s)[0] == 1):
return s
sum = 0
# for i in range(tf.shape(s)[0] - 1): # error: TypeError: range() integer end argument expected, got Tensor
# for i in range(s._shape._dims[0]._value - 1): # error: TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
for i in range(s.get_shape().as_list()[0] - 1): # error: TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
sum += tf.tanh(tf.matmul(Tl,s[i]) + tf.matmul(Tr, s[i+1]) + b)
return sum
ModelSize = 3
# tensor to be convolved
s = tf.placeholder("float", shape = [None, ModelSize])
# initialise weights
Tl = tf.Variable(tf.random_normal(shape=[ModelSize,ModelSize], stddev = 0.1 ))
Tr = tf.Variable(tf.random_normal(shape=[ModelSize,ModelSize], stddev = 0.1 ))
b = tf.Variable(tf.random_normal(shape=[ModelSize], stddev = 0.1 ))
#convolution
s_convolved = convolve(s, Tl, Tr, b)
# initialise variables.
init = tf.initialize_all_variables()
# run graph
sess = tf.Session()
sess.run(init)
# test data
s_dataLong = np.random.random((2,5,ModelSize))
s_dataShort = np.random.random((2,1,ModelSize))
for s_dataPart in s_dataLong:
print sess.run(s_convolved, feed_dict = {s : s_dataPart})
for s_dataPart in s_dataShort:
print sess.run(s_convolved, feed_dict = {s : s_dataPart})