Issue in installing pysqlcipher3

问题:

So even though pip install pysqlcipher3 worked returned Successfully installed pysqlcipher3, it did not.

From my research, it seems that its a common issue for Windows user trying to install Sqlcipher.

Attempt #1

First, I try to download the pysqlcipher3 from github and build and install it manually via cmd (using python setup.py build and python seetup.py install).

Upon executing python3 setup.py build I got some error (missing few requirements, not having OPENSSL_CONF in environment variable...etc.) but I fix most of them.

However, I end up stuck with this error:

fatal error C1083: Cannot open include file: 'sqlcipher/sqlite3.h': No such file or directory.

Attempt #2

I read somewhere (I think) that I have to install the SQLCipher before installing the PySqlCipher3.

Upon checking, I found out that I have to either pay to get a prebuild executable or build it myself.

So I tried to build it myself following this tutorial.

After installing all requirements, and following the steps I reach another dead end. In the before-last step (step number 6), when executing nmake /f Makefile.msc I got this error:

fatal error C1083: Cannot open include file: 'stdio.h': No such file or directory

---

My question are:

1. I read that I need libsqlcipher which might be the issue. Where can I find it (link to executable preferred).
2. How can I solve the issue in my first attempt?
3. How can I solve the issue in my second attempt?

My goal: Is to have pysqlcipher3 installed and working.

---

EDIT 1 - solved Attempt #2 just to reach another issue

So it seems that I was missing some C++ features and tool in Visual Studio. So via Visual Studio Community installer, I download and install basic C++ stuff and it fixed the error.

I got another error which I fix by copying OpenSSL-Win64\include folder to C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include.

However, now I am stuck with this error:

sqlite3.c(77701): warning C4389: '!=': signed/unsigned mismatch
    lib.exe /NOLOGO /MACHINE:x64 /OUT:libsqlite3.lib sqlite3.lo 
    echo EXPORTS > sqlcipher.def
    dumpbin /all libsqlite3.lib  | tclsh .\tool\replace.tcl include "^\s+1 _?(sqlite3(?:session|changeset|changegroup|rebaser)?_[^@]*)(?:@\d+)?$" \1  | sort >> sqlcipher.def
    link.exe /DEBUG   /NOLOGO /MACHINE:x64  /DLL /DEF:sqlcipher.def /OUT:sqlcipher.dll sqlite3.lo sqlite3res.lo  
   Creating library sqlcipher.lib and object sqlcipher.exp
sqlite3.lo : error LNK2019: unresolved external symbol RAND_bytes referenced in function sqlcipher_openssl_random
sqlite3.lo : error LNK2019: unresolved external symbol RAND_add referenced in function sqlcipher_openssl_add_random
sqlite3.lo : error LNK2019: unresolved external symbol OBJ_nid2sn referenced in function sqlcipher_openssl_get_cipher
sqlite3.lo : error LNK2019: unresolved external symbol EVP_MD_size referenced in function sqlcipher_openssl_get_hmac_sz
sqlite3.lo : error LNK2019: unresolved external symbol EVP_CIPHER_nid referenced in function sqlcipher_openssl_get_cipher
sqlite3.lo : error LNK2019: unresolved external symbol EVP_CIPHER_block_size referenced in function sqlcipher_openssl_get_block_sz
sqlite3.lo : error LNK2019: unresolved external symbol EVP_CIPHER_key_length referenced in function sqlcipher_openssl_get_key_sz
sqlite3.lo : error LNK2019: unresolved external symbol EVP_CIPHER_iv_length referenced in function sqlcipher_openssl_get_iv_sz
sqlite3.lo : error LNK2019: unresolved external symbol EVP_CipherInit_ex referenced in function sqlcipher_openssl_cipher
sqlite3.lo : error LNK2019: unresolved external symbol EVP_CipherUpdate referenced in function sqlcipher_openssl_cipher
sqlite3.lo : error LNK2019: unresolved external symbol EVP_CipherFinal_ex referenced in function sqlcipher_openssl_cipher
sqlite3.lo : error LNK2019: unresolved external symbol EVP_CIPHER_CTX_new referenced in function sqlcipher_openssl_cipher
sqlite3.lo : error LNK2019: unresolved external symbol EVP_CIPHER_CTX_free referenced in function sqlcipher_openssl_cipher
sqlite3.lo : error LNK2019: unresolved external symbol EVP_CIPHER_CTX_set_padding referenced in function sqlcipher_openssl_cipher
sqlite3.lo : error LNK2019: unresolved external symbol EVP_sha1 referenced in function sqlcipher_openssl_get_hmac_sz
sqlite3.lo : error LNK2019: unresolved external symbol EVP_sha256 referenced in function sqlcipher_openssl_get_hmac_sz
sqlite3.lo : error LNK2019: unresolved external symbol EVP_sha512 referenced in function sqlcipher_openssl_get_hmac_sz
sqlite3.lo : error LNK2019: unresolved external symbol EVP_get_cipherbyname referenced in function sqlcipher_openssl_activate
sqlite3.lo : error LNK2019: unresolved external symbol PKCS5_PBKDF2_HMAC referenced in function sqlcipher_openssl_kdf
sqlite3.lo : error LNK2019: unresolved external symbol HMAC_CTX_new referenced in function sqlcipher_openssl_hmac
sqlite3.lo : error LNK2019: unresolved external symbol HMAC_CTX_free referenced in function sqlcipher_openssl_hmac
sqlite3.lo : error LNK2019: unresolved external symbol HMAC_Init_ex referenced in function sqlcipher_openssl_hmac
sqlite3.lo : error LNK2019: unresolved external symbol HMAC_Update referenced in function sqlcipher_openssl_hmac
sqlite3.lo : error LNK2019: unresolved external symbol HMAC_Final referenced in function sqlcipher_openssl_hmac
sqlcipher.dll : fatal error LNK1120: 24 unresolved externals

回答1:

First of all, "Successfully installed pysqlcipher3" is misleading (at least up to the current latest commit fd1b547407bcb7198107fe3c458105286a060b0d on the Github repository). I have just wasted some of my own time, and thought I should make it clear again.

The misleading "Successfully installed pysqlcipher3" appears on Linux systems too. In case you have a Ubuntu/Debian system handy, the following would get you going. Basically you need to install the SQLCipher libraries and development files before the pip3 command.

$ sudo apt install sqlcipher libsqlcipher0 libsqlcipher-dev
$ sudo -H pip3 install pysqlcipher3

$ python3 -c 'import pysqlcipher3; print(pysqlcipher3.__path__)'
['/usr/local/lib/python3.7/dist-packages/pysqlcipher3']

Since you are on Windows, unresolved external symbol RAND_bytes (first line on your compiler/linker error) basically means that your linker on Windows can't find the necessary OpenSSL libraries.

Have look at the posts here, here, and here. They have solutions to this exact OpenSSL linking issue. The basic steps towards resolution are the following:

1. Download and install the required 32/64-bit version of OpenSSL from here
2. Make Visual Studio to use the OpenSSL include directory e.g. C:\OpenSSL-Win32\include
3. Make Visual Studio to use the OpenSSL library directory e.g. C:\OpenSSL-Win32\lib
4. Add the following libraries as additional-linker-dependencies in Visual Studio:

    - libeay32.lib
    - libeay32MTd.lib
    - libeay32MT.lib
    - libeay32MDd.lib
    - libeay32MD.lib
    - ssleay32.lib
    - ssleay32MTd.lib
    - ssleay32MT.lib
    - ssleay32MDd.lib
    - ssleay32MD.lib

回答2:

I also had almost all of your problems and so I created this guide below which worked for me

My build: windows 10 x64 v1909

1. Install python (if not already installed). I had v3.7 64 bit
2. Install Win64OpenSSL-1_0_2u. Make sure you get exactly this version since I first downloaded some other versions and they didn't work.
3. Add environment variable for both user and system SET OPENSSL_CONF=C:\OpenSSL-Win64\bin\openssl.cfg (change path as per yours)
4. Install ActiveTcl-8.6.9.8609.2-MSWin32-x64. This was latest at this time and worked
5. Install visual studio (if not already installed). I had VS2019 v16.5.5
6. Get sqlcipher and extract to directory
7. Open visual studio x64 native tools and navigate to above extracted directory of sqlcipher
8. Run command nmake /f Makefile.msc. The final compilation will report an error, but it does not matter, we do not need a real compilation here, only need sqlite3.c
9. Get pysqlcipher3-1.0.3, extract to directory and navigate to it
10. Create folder named amalgamation in it and create another folder named sqlcipher in amalgamation folder
11. Copy sqlite3.h and sqlite3.c created in Visual Studio step (in sqlcipher-master folder) to amalgamation folder and sqlcipher folder both
12. Open administrative command prompt and navigate to above extracted directory of pysqlcipher3-1.0.3
13. Run following command python setup.py build_amalgamation
14. This will give linkage errors as OP has stated. Now install Win32OpenSSL-1_0_2u. Earlier you installed x64 and now installing x32. I learnt this hack after multiple failures -:)
15. Adjust the environment variables for SSL to point to x32 installed location instead of x64 as done in point-3
16. Close and reopen admin CMD and retype the command python setup.py build_amalgamation. This time it should work well without any errors.
17. Now run following command python setup.py install
18. If all goes well then this command should work in cmd>>python>>from pysqlcipher3 import dbapi2

Some Useful Links for me 1, 2, 3, 4, 5

If you don't get exact versions as mine due to links broken in future, you can comment and if I have them at that time, I'll upload to a cloud drive.

回答3:

On iMac running Catalina macOS 10.15.4, when I tried to install pysqlcipher3, it spat out error messages

But then I ran

$ brew install SQLCipher

which worked successfully:

$ brew install SQLCipher
==> Downloading https://homebrew.bintray.com/bottles/sqlcipher-4.4.0.catalina.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/9e/9e860b50ec668ef30f61377f39954241c4eeda4c4a664fbe1340e289229336bf?__gda__=exp=1590256907~hmac=9bb692e78355e890ab
######################################################################## 100.0%
==> Pouring sqlcipher-4.4.0.catalina.bottle.tar.gz