I'm trying to implement a simple gradient descent for linear regression.
It works normally if I compute the gradient manually (by using the analytical expression), but now i was trying to implement it with autograd from the mxnet module.
This is the code
from mxnet import autograd, np, npx
npx.set_np()
def main():
# learning algorithm parameters
nr_epochs = 1000
alpha = 0.01
# read data, insert column of ones (to include bias with other parameters)
data = pd.read_csv("dataset.txt", header=0, index_col=None, sep="\s+")
data.insert(0, "x_0", 1, True) # insert column of "1"s as x_0
m = data.shape[0] # number of samples
n = data.shape[1] - 1 # number of features
X = data.iloc[:, 0:n].values # array with x values
Y = data.iloc[:, -1].values # array with y values
theta = np.zeros(n) # initial parameters array
theta.attach_grad()
theta, J = GradientDescent(X, Y, theta, alpha, nr_epochs)
#-------------------#
# loss function #
#-------------------#
def LossFunction(X, Y, theta):
m = X.shape[0] # number of training samples
loss = 0
for i in range(X.shape[0]):
loss = loss + (1 / (2 * m)) * (H(X[i, :], theta) - Y[i]) ** 2
return loss
#----------------#
# hypothesis #
#----------------#
def H(x, theta):
return np.dot(x, theta)
#----------------------#
# gradient descent #
#----------------------#
def GradientDescent(X, Y, theta, alpha, nr_epochs):
m = X.shape[0]
n = X.shape[1]
grad = np.zeros(n)
Loss = np.zeros(nr_epochs)
for epoch in range(nr_epochs):
with autograd.record():
Loss[epoch] = LossFunction(X, Y, theta)
Loss[epoch].backward()
for j in range(n):
theta[j] = theta[j] - alpha * theta.grad[j]
return theta, Loss
if __name__ == "__main__":
main()
The problem is that I get an error when the code is computing the hypothesis, in the dot product between X and theta
return np.dot(x, theta)
The error message says: Argument a must have NDArray type, but got [ 1. -5.05358]
So, I'm assuming there must be some incompatibility between the types of x and theta. I checked them, and I got:
X -> <class 'numpy.ndarray'>
theta -> <class 'mxnet.numpy.ndarray'>
theta is created with np.zeros, so it is a mxnet array, while X is converted from a dataset with the .values method...is this the source of the problem? Thanks!