Convert Sequential to Functional in Keras

I have a keras code written in Sequential style. But I am trying to switch Functional mode because I want to use merge function. But I faced an error below when declaring Model(x, out). What is wrong in my Functional API code?

# Sequential, this is working
# out_size==16, seq_len==1
model = Sequential()
model.add(LSTM(128, 
               input_shape=(seq_len, input_dim),
               activation='tanh', 
               return_sequences=True))
model.add(TimeDistributed(Dense(out_size, activation='softmax')))

# Functional API
x = Input((seq_len, input_dim))
lstm = LSTM(128, return_sequences=True, activation='tanh')(x)
td = TimeDistributed(Dense(out_size, activation='softmax'))(lstm)
out = merge([td, Input((seq_len, out_size))], mode='mul')
model = Model(input=x, output=out) # error below

RuntimeError: Graph disconnected: cannot obtain value for tensor Tensor("input_40:0", shape=(?, 1, 16), dtype=float32) at layer "input_40". The following previous layers were accessed without issue: ['input_39', 'lstm_37']

Updated

Thank you @Marcin Możejko. I finally did it.

x = Input((seq_len, input_dim))
lstm = LSTM(128, return_sequences=True, activation='tanh')(x)
td = TimeDistributed(Dense(out_size, activation='softmax'))(lstm)
second_input = Input((seq_len, out_size)) # object instanciated and hold as a var.
out = merge([td, second_input], mode='mul')
model = Model(input=[x, second_input], output=out) # second input provided to model.compile(...)

# then I add two inputs
model.fit([trainX, filter], trainY, ...)

One may notice that reference of an object created by a Input((seq_len, out_size)) call is accesible only from merge funciton call evironment. Moreover - it's not add to a Model definition - what makes graph disconnected. What you need to do is:

x = Input((seq_len, input_dim))
lstm = LSTM(128, return_sequences=True, activation='tanh')(x)
td = TimeDistributed(Dense(out_size, activation='softmax'))(lstm)
second_input = Input((seq_len, out_size)) # object instanciated and hold as a var.
out = merge([td, second_input], mode='mul')
model = Model(input=[x, second_input], output=out) # second input provided to model