Multi variable gradient descent in matlab

问题:

I'm doing gradient descent in matlab for mutiple variables, and the code is not getting the expected thetas I got with the normal eq. that are: theta = 1.0e+05 * 3.4041 1.1063 -0.0665 With the Normal eq. I have implemented.

And with the GDM the results I get are: theta = 1.0e+05 * 2.6618 -2.6718 -0.5954 And I don't understand why is this, maybe some one can help me and tell me where is the mistake in the code.

Code:

function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters)

m = length(y); % number of training examples
J_history = zeros(num_iters, 1);
thetas = size(theta,1);
features = size(X,2)

mu = mean(X);
sigma = std(X);
mu_size = size(mu);
sigma_size = size(sigma);

%for all iterations
for iter = 1:num_iters

tempo = [];

result = [];

theta_temp = [];

%for all the thetas    
for t = 1:thetas
    %all the examples
    for examples = 1:m
       tempo(examples) = ((theta' * X(examples, :)') - y(examples)) * X(m,t)
    end

    result(t) = sum(tempo)
    tempo = 0;

end

%theta temp, store the temp 
for c = 1:thetas

    theta_temp(c) = theta(c) - alpha * (1/m) * result(c)
end

%simultaneous update
for j = 1:thetas

    theta(j) = theta_temp(j)

end

% Save the cost J in every iteration    
J_history(iter) = computeCostMulti(X, y, theta);

end

theta
end

Thanks.

EDIT: Data.

  X =
    1.0000    0.1300   -0.2237
    1.0000   -0.5042   -0.2237
    1.0000    0.5025   -0.2237
    1.0000   -0.7357   -1.5378
    1.0000    1.2575    1.0904
    1.0000   -0.0197    1.0904
    1.0000   -0.5872   -0.2237
    1.0000   -0.7219   -0.2237
    1.0000   -0.7810   -0.2237
    1.0000   -0.6376   -0.2237
    1.0000   -0.0764    1.0904
    1.0000   -0.0009   -0.2237
    1.0000   -0.1393   -0.2237
    1.0000    3.1173    2.4045
    1.0000   -0.9220   -0.2237
    1.0000    0.3766    1.0904
    1.0000   -0.8565   -1.5378
    1.0000   -0.9622   -0.2237
    1.0000    0.7655    1.0904
    1.0000    1.2965    1.0904
    1.0000   -0.2940   -0.2237
    1.0000   -0.1418   -1.5378
    1.0000   -0.4992   -0.2237
    1.0000   -0.0487    1.0904
    1.0000    2.3774   -0.2237
    1.0000   -1.1334   -0.2237
    1.0000   -0.6829   -0.2237
    1.0000    0.6610   -0.2237
    1.0000    0.2508   -0.2237
    1.0000    0.8007   -0.2237
    1.0000   -0.2034   -1.5378
    1.0000   -1.2592   -2.8519
    1.0000    0.0495    1.0904
    1.0000    1.4299   -0.2237
    1.0000   -0.2387    1.0904
    1.0000   -0.7093   -0.2237
    1.0000   -0.9584   -0.2237
    1.0000    0.1652    1.0904
    1.0000    2.7864    1.0904
    1.0000    0.2030    1.0904
    1.0000   -0.4237   -1.5378
    1.0000    0.2986   -0.2237
    1.0000    0.7126    1.0904
    1.0000   -1.0075   -0.2237
    1.0000   -1.4454   -1.5378
    1.0000   -0.1871    1.0904
    1.0000   -1.0037   -0.2237

y =
      399900
      329900
      369000
      232000
      539900
      299900
      314900
      198999
      212000
      242500
      239999
      347000
      329999
      699900
      259900
      449900
      299900
      199900
      499998
      599000
      252900
      255000
      242900
      259900
      573900
      249900
      464500
      469000
      475000
      299900
      349900
      169900
      314900
      579900
      285900
      249900
      229900
      345000
      549000
      287000
      368500
      329900
      314000
      299000
      179900
      299900
      239500

Full dataset.

回答1:

The line where you calculate tempo is wrong. It should be

tempo(examples) = ((theta' * X(examples, :)') - y(examples)) * X(examples,t)

Also try using matrix operations in MATLAB. Your code will be faster and it will also be easier to understand. For example, you can replace your nested loop with

E = X * theta - y;
for t = 1:thetas
    result(t) = sum(E.*X(:,t));
end

You can replace your subsequent two loop for updating theta into one line

theta = theta - alpha * (1/m) * result';