Haskell dynamic library

Question

http://www.vex.net/~trebla/haskell/so.xhtml describes how to compile shared library.

About compiling command:

ghc -O2 -dynamic -shared -fPIC -o libEval.so Eval.hs hsbracket.c -lHSrts-ghc7.6.3

it says:

(Could you omit -dynamic to request static libraries of other packages? Not really, they were not generated with -fPIC. In particular it is illegal on x86_64.)

Why is it so? What should one do to compile shared library without libHS* dependencies?

Answer 1

Since Kaiko contacted me privately for an answer, might as well post it here...

Short version

By omitting -dynamic you would be trying to take all the static .a libs and link them into one massive .so file. Those .a libs were themselves built without -fPIC. All code that ends up on a .so file must be built with -fPIC (at least on ELF x86-64). Thus the linking would fail in this case because -fPIC is required but the libs were not built with -fPIC.

Long version

There are a few things that vary between different ways of building static and dynamic libraries:

1. Is it built as a .a archive or as a .so (or .dll/.dynlib) object?
2. Is it built with -fPIC, position independent code or not?
3. Are external symbols expected to be in the same DSO or an external DSO?

In principle, lots of different combinations of these things make sense but in practice only a few are used.

On Linux (ELF), there are two standard approaches to building libraries, fully static and fully dynamic. In the fully static approach the answer to question 1,2,3 above are: .a archive, no -fPIC, same DSO. In the fully dynamic approach the answers are: .so lib, -fPIC, external DSO.

Now what you want to do is different. You want all libraries to be built as .a files, but with -fPIC and external symbols expected to be in the same DSO. This would then let you link all those libraries together into one huge shared library. So the crucial difference is the use of -fPIC, since on ELF (specifically x86_64) code that ends up in a shared lib must be built with -fPIC.

By contrast, on Windows, GHC can do exactly what you want, to link all the Haskell libs (including the RTS etc) into one massive shared lib (.dll). This is because on Windows (unlike ELF), position independent code does not matter. So on Windows, one is able to take the static libraries and link them into a big shared library.

In principle this should also be possible on Linux, if all of the Haskell libraries were built statically but with -fPIC. This is not the default, and that is the immediate reason why you cannot omit -dynamic in this case on Linux.

So in principle, one could try rebuilding ghc and the core libraries from source using the -fPIC flag and then see if it then works to omit -dynamic and link everything into one huge shared lib.

Answer 2

Yes, compiling with -fPIC helps. Here is how to do that.

ghc-7.8.4/mk/build.mk:

 SRC_HC_OPTS     = -H64m -O 
 EXTRA_HC_OPTS   = -fPIC    
 SRC_CC_OPTS     = -fPIC -O 
 GhcStage1HcOpts = -fasm -O0
 GhcStage2HcOpts = -fasm -O0
 GhcLibHcOpts    = -fasm -O2
 GhcLibWays      = v dyn    
 DYNAMIC_GHC_PROGRAMS = YES 
 DYNAMIC_BY_DEFAULT   = NO  
 SplitObjs            = NO  
 HADDOCK_DOCS         = NO  
 BUILD_DOCBOOK_HTML   = NO  
 BUILD_DOCBOOK_PS     = NO  
 BUILD_DOCBOOK_PDF    = NO  

While you compile ghc:

export EXTRA_CONFIGURE_OPTS="--disable-library-profiling --enable-shared"

To build cabal packages with -fPIC use:

cabal install --enable-shared --ghc-option=-fPIC text

Test file foo.hs (Data.Text is used to see if cabal packages also work):

import Foreign.C as C 
import Data.Text as T
import Data.Text.Foreign as T 
foreign export ccall   len :: CString -> IO CInt 
len t = C.peekCString t >>= return . CInt . fromIntegral . T.length . T.pack  
main = return ()

Dynamic build:

ghc -dynamic --make foo.hs

Dynamic mixed with static build (not sure if pthread is needed but it illustrates how to add dynamic linking):

ghc -fPIC -shared --make -o libfoo.so \
 -optl-Wl,-Bstatic -lHSrts -lCffi \
   -lHSbase-4.7.0.2 -lHSinteger-gmp-0.5.1.0 -lHSghc-prim-0.3.1.0 \
-optl-Wl,-Bdynamic -lpthread foo.hs