In order to compute the optimal theta e.g. in logistic regression, I have to create a costFunction (the function to be minimized) which is then passed to fminunc in order to obtain the optimal theta.
Also, if the gradient of costFunction can be computed, I set the 'GradObj' option to 'on' using
options = optimset('GradObj','on');
and code the costFunction so that it returns, as a second output argument, the gradient value g of X.
Then I give
[theta, cost] = fminunc(@(t)(costFunction(t, X, y)), initial_theta, options);
where X is the data matrix and y the response. How can I implement the above in R?
Take a look at the optim function. It can do unconstrained minimization using method = 'L-BFGS-B' and you can specify an analytical function to compute the gradient as well
EDIT. As Ben has pointed out correctly, fminunc does unconstrained optimization, which can also be achieved using the optim function choosing Nelder-Mead or BFGS. Moreover, I also noticed from the documentation of fminunc that it does large-scale optimization using trust region methods. There is an R package trust that I believe does the same thing. I would recommend taking a look at the optimization task view of R.
In the R, you can use the function nlminb in the R to do constrained optimization!
nlminb(start, objective, gradient = NULL, hessian = NULL, ..., scale = 1, control = list(), lower = -Inf, upper = Inf)
The start is a vector include all initial value of parameters. objective is the cost function or any other function that you want to minimize.
