Tensorflow on simple linear regression

问题:

I am a beginner in machine learning and tensorflow. In the first step trying the tensorflow, I tried a simple multivariate linear regression. However, it seems the model stuck at a local minimum. Here is my code.

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

def weight_variable(shape):
    initial = tf.truncated_normal(shape, stddev=1)
    return tf.Variable(initial)

# dataset
xx = np.random.randint(0,1000,[1000,3])/1000.
yy = xx[:,0] * 2 + xx[:,1] * 1.4 + xx[:,2] * 3

# model
x = tf.placeholder(tf.float32, shape=[None, 3])
y_ = tf.placeholder(tf.float32, shape=[None])
W1 = weight_variable([3, 1])
y = tf.matmul(x, W1)

# training and cost function
cost_function = tf.reduce_mean(tf.square(y - y_))
train_function = tf.train.AdamOptimizer(1e-2).minimize(cost_function)

# create a session
sess = tf.Session()

# train
sess.run(tf.initialize_all_variables())
for i in range(10000):
    sess.run(train_function, feed_dict={x:xx, y_:yy})
    if i % 1000 == 0:
        print(sess.run(cost_function, feed_dict={x:xx, y_:yy}))

The output is:

14.8449
2.20154
2.18375
2.18366
2.18366
2.18366
2.18366
2.18366
2.18366

The output value (yy) is ranging from 0 to 6, so having mean square error 2.18 is considerably large, knowing that there is no noise added to the dataset. I also tried GradientDescentOptimizer with learning rate 0.1 and 1e-2, but it does not improve the results much.

Is there anything wrong with my implementation?

回答1:

This is because y is not the same shape as y_. y is of shape (1000, 1) and y_ is of shape (1000). So when you subtract them, you're inadvertently creating a 2-D matrix.

To fix it change your cost function to:

cost_function = tf.reduce_mean(tf.square(tf.squeeze(y) - y_))

回答2:

As mentioned in another answer, u have to use

predictions = tf.add(b, tf.matmul(x, w))
error = tf.reduce_mean(tf.square(y - predictions))

And as you are saying that, you are a Tensorflow beginner, you can look at the example here:-

https://medium.com/@saxenarohan97/intro-to-tensorflow-solving-a-simple-regression-problem-e87b42fd4845