“Linear dependence in the dictionary” exception in

Question

I'm using sklearns OrthogonalMatchingPursuit to get a sparse coding of a signal using a dictionary learned by a KSVD algorithm. However, during the fit I get the following RuntimeWarning:

/usr/local/lib/python2.7/dist-packages/sklearn/linear_model/omp.py:391: RuntimeWarning:  Orthogonal matching pursuit ended prematurely due to linear
dependence in the dictionary. The requested precision might not have been met.

  copy_X=copy_X, return_path=return_path)

In those cases the results are indeed not satisfactory. I don't get the point of this warning as it is common in sparse coding to have an overcomplete dictionary an thus also linear dependency within it. That should not be an issue for OMP. In fact, the warning is also raised if the dictionary is a square matrix.

Might this Warning also point to other issues in the application?

Answer 1

The problem was in the data vector y in

omp = OrthogonalMatchingPursuit(n_nonzero_coefs=target_sparsity)
omp.fit(D, y)

It contained numbers with very small magnitude. When I normalize y as well as D the fit works with the expected accuracy.