Quick and Hassle-free Installation & Usage of IT++

Question

IT++ is a popular GPL library (proposed to be LGPL but no signs as of now) for scientific computation especially in Telecommunications domain. This was recommended to me by my colleagues.

It looks hard to install this piece of library on my Linux/Windows system. And the installation guide seems hard.

I have a lot more to do other than spending time on this installation. Can anyone save me some time with a few quick installation and if possible, some hello, world! type instructions?

Answer 1

TL;DR

 $ wget https://sourceforge.net/projects/itpp/files/latest -O itpp.tar.bz2
 $ tar -xjvf itpp.tar.bz2 && cd itpp-*
 $ sudo apt-get install liblapack-dev libblas-dev libfftw3-dev
                 # Note: This is basic and recommended. MKL/ATLAS etc are
                 # platform-specialized, can give slightly better perf.

 $ mkdir build1 build2

 $ cd build1                   # DYNAMIC (*.so file)  
 $ cmake ..  
 $ make  
 $ make install  

 $ cd bulid2                   # STATIC (*.a file)
 $ cmake .. -DITPP_SHARED_LIB=off
 $ make
 $ make install

 # Note: The library installation is complete and is ready for both
 # static and dynamic compilations. But the subsequent run of a 
 # *static compilation* is relatively more difficult, see details below.

INTRO

IT++, is a powerful numerical computing library popular among researchers working in communication theory, providing matrix related constructs. While other matrix libraries relatively recent Eigen etc are gaining more popularity, it remains in use for its beautiful and elegant interface usually compared to MATLAB. It also provides interfacing to MATLAB via Mex routines. I use it all the time.

INSTALLATION - Prologue

Unfortunately, even though the installation instructions appear sufficiently complete, it is very painful for beginners as I find among my own colleagues. The reason I think is that it is not beginner-friendly. I sincerely hope this post will bridge the gap.

I think the main source of confusion in the installation is due to its dependencies of BLAS, LAPACK, and FFTW, being mentioned along-side of the optional, vendor-specific versions of BLAS/LAPACK/FFTW such as MKL, ATLAS, ACML libraries adding to the confusion of what actually is to be done.

The fact is, a majority of those confusing bits are optional!

I would recommend to first try the below quick instructions and ignore everything else till then.

INSTALLATION INSTRUCTIONS ON UBUNTU

1. Download IT++ archive file
   $ wget https://sourceforge.net/projects/itpp/files/latest -O itpp.tar.bz2

2. Extract and change directory
   $ tar -xjvf itpp.tar.bz2 && cd itpp-*

3. Create new sub-directories "build1" and "build2" (we will cd to them in next steps) $ mkdir build1 build2

4. Make sure you have LAPACK, BLAS, FFTW libraries. This should not affect later stages btw.
   $ sudo apt-get install liblapack-dev libblas-dev libfftw3-dev

Here an important caveat: A more efficient option is often said to be to use your processor's vendor-specific library such as Intel's MKL, AMD's ACML, ATLAS. But that efficiency comes at a cost -- you need to spend significant time to register (for MKL), download, install, and finally adjust linker switches to include them every time they are compiled and run.

5. The main installation steps:
   Please do both steps below. There is no harm, plus I personally think IT++ should have installing both as the default option. Simply because, you never know when do you require a static compilation in future. This is why almost every other library installs both versions on your pc.

   A. As a shared library (*.so files)

     $ cd build1  
     $ cmake ..  
     $ make  
     $ make install  
   

   B. As a static library (*.a files)

     $ cd bulid2
     $ cmake .. -DITPP_SHARED_LIB=off
     $ make
     $ make install
   

During cmake, you might experience the below errors, but they never affected later stages till successful running of individual IT++ programs. So please ignore them. In future, later versions might declare/correct them as bugs. Current version is 4.3.1.

     CMake Warning at CMakeLists.txt:192 (message):
        BLAS library not found.

     CMake Warning at CMakeLists.txt:196 (message):
        LAPACK library not found.

     CMake Warning at CMakeLists.txt:200 (message):
        FFT library not found.

Test your installation with Hello,World!

Consider the below sample program testITPP.cpp

  #include<iostream>
  using namespace std;
  #include<itpp/itbase.h>
  using namespace itpp;

  int main()
  {
    vec x(100); // vec == Vec<double>
    double y;
    x=linspace(1,100,100);
    y=sum(x);
    cout<<"Hello, World! \n The sum of first 100 integers is: "<<y<<endl;
  }

Now the main important thing during compilation is to use -litpp switch all the time, after all the files.

$ g++ testITPP.cpp -litpp
$./a.out 
Hello, World! 
The sum of first 100 integers is: 5050

If ./a.out produces errors, please run sudo ldconfig once to refresh library cache.

---

---

EXTRA: Struggles with Static Compilation:

Note that the earlier compilation uses shared-libraries by default and produces a binary that depends on too many other binaries on your PC. I.e., the above binary a.out cannot run on another unix system with no IT++ installed for e.g. This often creates problems, especially during submissions to university clusters and servers which allow users to perform heavy and long-duration computations.

A work-around is to use a static compilation, getting rid of all local dependencies. But this is usually difficult for beginners. With IT++ under MKL kind of libraries, it can really be frustrating to just know how to static-compile. A good summary of how to do plus all the bugs that I notice as of today are as below.

i. Static compilation with Basic BLAS/LAPACK/FFTW:

This is the simplest case. Two particularly important changes are to be noted. One -litpp becomes -litpp_static. Two, the new compilation switch -static. The first is a result of IT++'s strange choice to name binaries as libitpp.so and libitpp_static.a.

Also, unlike dynamic compilation, it may be compulsary to include all libraries other than C++ standard ones (i.e. -lc and -lm) for static compilation. This mainly includes -lblas -llapack -lfftw -lpthread and -ldl.

As a result, one of the following might work in your case:

$ g++ -static testITPP.cpp -litpp_static
 OR
$ g++ -static testITPP.cpp -litpp_static -llapack -lblas -lpthread
 OR
$ g++ sctest_PCCestimates.cpp -litpp_static \
       -llapack -lblas -lgomp -lpthread \
       -lgfortran -static -Wl,--allow-multiple-definition 

ii. Static compilation with MKL's BLAS/LAPACK/FFTW:

While I am yet to see how faster it is to use MKL's vendor-specific libraries, the static compilation can be really painful. The initial installation of MKL itself took more than an hour to me. Even though the exact instructions won't fit for other vendor-libraries, it should get you ready for the actual challenges you are about to face.

Most of the below are not understood by the error messages that you encounter. As a result they will be very annoying and painful-to-resolve. I am trying to list them all below. Hopefully they will save a ton of your valuable time.

1. Circular dependency during compilation: The first few libraries may need to be surrounded by -Wl,--start-group .... -Wl,--end-group.

2. With vendor-specific blas/lapack/fftw, you may need to search where their respective libblas.a liblapack.a libfftw.a or their equivalents such as libmkl_core.a in your vendor-specific library. The directory location must be supplied using the switch -L/path/to/statics/

3. A segmentation fault may be encountered, even after a successful compilation. The reason for this annoying problem seems to be that the libpthread.a is not entirely included. Then you need force appropriate action, by surrounding the switch -lpthread as -Wl,--whole-archive -lpthread -Wl,--no-whole-archive

Finally the version that worked to me on Ubuntu 16.04 is this. Note that there is none of -llapack -lblas -lfftw etc, as they are bundled under differently named static binaries.

$ g++ -static \  
        -L/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64_lin/  
       testITPP.cpp  
        -Wl,--start-group \
        -litpp_static \
        -lmkl_intel_lp64 \
        -lmkl_sequential \
        -lmkl_core  -Wl,--end-group \
        -Wl,--whole-archive -lpthread -Wl,--no-whole-archive \
        -ldl

Finally, inspite of you having a static binary successfully generated in above, you might get a warning saying warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking might appear but can be ignored as apparently there is no solution ever! This is related to a universally known issue that we need not care.

Hoping that someone else will write a guide for Windows/Visual-Studio installation.