Sparse Group lasso by CVX MATLAB package

Question

Does anyone know how can I implement sparse group lasso using CVX, convex optimization package in MATLAB?

I don't know how can I describe the formula as CVX prototype.

Answer 1

I believe CVX cannot easily handle the group norm in SGL, unless you hard-code the group norm for each dataset. If u are using Matlab, you can use SLEP toolbox.

Answer 2

Do you have to use CVX? Inria has a sparse modeling package called Spams written in Matlab, R, and Python. If you want a group lasso regularizer look at the documentation in the proximal toolbox under mexproximalFlat. There are some examples as well. I use the python spams package quite a bit.

Answer 3

I found something here

I decided to share it!!

Answer 4

(correction: both support different group sizes. the example from nfs supports different group sizes by using additional constraints.)

Refer to this webpage for an example given by nfs: http://ask.cvxr.com/t/formulating-sparse-group-lasso-in-cvx/793/4
However this example seems not allow different group sizes. You may refer to the following example (The formula used is Eq.3 in Simon, Noah, and Robert Tibshirani. "Standardization and the group lasso penalty." Statistica Sinica 22.3 (2012): 983.)

% Refer to Eq. (3) in /Simon, Noah, and Robert Tibshirani. 
%    "Standardization and the group lasso penalty." 
%    Statistica Sinica 22.3 (2012): 983./

% Note that group LASSO allows different group sizes
N = 64; m = 3;   
rho = [2; 4; 6];  % group sizes
n = sum(rho); % num of total parameters
X = rand(N,n);   % X = [X1, X2, ..., X_m]
y = rand(N,1);
lambda = 1;

IndexM = [1, 2; 3, 6; 7, 12];  % indexes of elements in each group
cvx_begin
    % w = [beta1'; beta2'; ...; beta_m']
    variable w(n)    
    expression ws(m)
    for i = 1:m
        ws(i) = norm(w(IndexM(i,1):IndexM(i,2)),2);
    end

    minimize( norm(y-X*w, 2) +  lambda*(sqrt(rho)' * ws) )
cvx_end

% get beta_i, i.e. i-th beta corresponding to i-th group
% e.g. 
i = 2;
beta_i = w(IndexM(i,1):IndexM(i,2));