Error installing PyLucene JCC on OSX

Question

I am having trouble installing PyLucene JCC on Mac OSX.

I dowloaded the latest version of JCC from apache lucene site and fallowed the instruction to install it using the command

python setup.py build

But I am getting the following error while installing

ld: internal error: atom not found in symbolIndex(__ZN7JNIEnv_13CallIntMethodEP8_jobjectP10_jmethodIDz) for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'c++' failed with exit status 1

I have tried following the instruction given on the official site of adding the flag "-framework", "Python" to the LFLAGS value when installing on mac OSX or using the export CFLAGS=-Qunused-arguments and export CPPFLAGS=-Qunused-arguments before runnig the install command to ignore the warnings while build. But none of them are working.

The full trace of error is

found JAVAHOME = /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home
found JAVAFRAMEWORKS = /System/Library/Frameworks/JavaVM.framework
Loading source files for package org.apache.jcc...
doc/serialized-form.html...
Building index for all the packages and classes...
Generating javadoc/overview-tree.html...
Generating javadoc/index-all.html...
Generating javadoc/deprecated-list.html...
Building index for all classes...
Generating javadoc/allclasses-frame.html...
Generating javadoc/allclasses-noframe.html...
Generating javadoc/index.html...
Generating javadoc/help-doc.html...
running build
running build_py
writing /Users/harshsingh/Documents/Codes/IR/jcc/jcc/config.py
copying jcc/config.py -> build/lib.macosx-10.10-intel-2.7/jcc
copying jcc/classes/org/apache/jcc/PythonVM.class -> build/lib.macosx-10.10-intel-2.7/jcc/classes/org/apache/jcc
copying jcc/classes/org/apache/jcc/PythonException.class -> build/lib.macosx-10.10-intel-2.7/jcc/classes/org/apache/jcc
running build_ext
building 'jcc' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -    DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -dynamiclib -D_jcc_lib -DJCC_VER="2.    21" -I/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/jdk1.7.0_45.    jdk/Contents/Home/include/darwin -I_jcc -Ijcc/sources -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c jcc/sources/jcc.cpp -    o build/temp.macosx-10.10-intel-2.7/jcc/sources/jcc.o -DPYTHON -fno-strict-aliasing -Wno-write-strings
clang: warning: argument unused during compilation: '-dynamiclib'
jcc/sources/jcc.cpp:197:16: warning: implicit conversion loses integer precision: 'long' to 'int'
      [-Wshorten-64-to-32]
    int hash = PyObject_Hash(arg);
        ~~~~   ^~~~~~~~~~~~~~~~~~
1 warning generated.
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -    DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -dynamiclib -D_jcc_lib -DJCC_VER="2.    21" -I/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/jdk1.7.0_45.    jdk/Contents/Home/include/darwin -I_jcc -Ijcc/sources -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c jcc/sources/JCCEnv.    cpp -o build/temp.macosx-10.10-intel-2.7/jcc/sources/JCCEnv.o -DPYTHON -fno-strict-aliasing -Wno-write-strings
clang: warning: argument unused during compilation: '-dynamiclib'
c++ -Wl,-x -dynamiclib -undefined dynamic_lookup build/temp.macosx-10.10-intel-2.7/jcc/sources/jcc.o build/temp.macosx-10.10-intel-2.7/jcc/sources/JCCEnv.    o -o build/lib.macosx-10.10-intel-2.7/libjcc.dylib -L/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib -ljava -    L/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/server -ljvm -Wl,-rpath -Wl,/Library/Java/JavaVirtualMachines/jdk1.7.0_45.    jdk/Contents/Home/jre/lib -Wl,-rpath -Wl,/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/server -Wl,-S -install_name @rpath/libjcc.    dylib -current_version 2.21 -compatibility_version 2.21
ld: internal error: atom not found in symbolIndex(__ZN7JNIEnv_13CallIntMethodEP8_jobjectP10_jmethodIDz) for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'c++' failed with exit status 1    

I am really lost now and any help will be really great.

Answer 1

If rebuilding from source still ends up giving the same errors, you could probably try the following approach.

Seems like there is a bug with running the clang++ command (below) with the -x flag, on OSX 10.9+ (referenced here)

cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -    DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -dynamiclib -D_jcc_lib -DJCC_VER="2.    21" -I/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/jdk1.7.0_45.    jdk/Contents/Home/include/darwin -I_jcc -Ijcc/sources -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c jcc/sources/JCCEnv.    cpp -o build/temp.macosx-10.10-intel-2.7/jcc/sources/JCCEnv.o -DPYTHON -fno-strict-aliasing -Wno-write-strings
clang: warning: argument unused during compilation: '-dynamiclib'
c++ -Wl,-x -dynamiclib -undefined dynamic_lookup build/temp.macosx-10.10-intel-2.7/jcc/sources/jcc.o build/temp.macosx-10.10-intel-2.7/jcc/sources/JCCEnv.    o -o build/lib.macosx-10.10-intel-2.7/libjcc.dylib -L/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib -ljava -    L/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/server -ljvm -Wl,-rpath -Wl,/Library/Java/JavaVirtualMachines/jdk1.7.0_45.    jdk/Contents/Home/jre/lib -Wl,-rpath -Wl,/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/server -Wl,-S -install_name @rpath/libjcc.    dylib -current_version 2.21 -compatibility_version 2.21

Run the clang++ command again without the -x flag and then run:

python setup.py build

Answer 2

Same problem here - from pyLucene-dev:

You must ensure that the compiler used to build Python is the same as the compiler you're using. If you did not build Python then it's likely you're htting a clang/gcc mismatch and you need to rebuild Python from sources, using the same compiler as you're using to build JCC.

Answer 3

The clang problem is fixed as given here. However, its not pushed in the PyLucene latest(as of May 2018) version(v6.5). However, you can still get the next build(v7.2) of Pylucene from this link. Using Pylucene 7.2.0 fixed the issue for me.