I'm trying (and failing) to compile a fortran module (specifically igrf12.f from the BGS) using f2py and Python 3.6 on Windows 10. Python was installed using Anaconda 4.4.10.
My setup:
- Python 3.6.3 |Anaconda custom (64-bit)| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)] on win32
- Windows 10 Enterprise (version 1703)
- NumPy 1.14.0
I followed f2py directions from the SciPy documentation and a very helpful guide from Dr.Michael Hirsch. Dr.Hirsch has created the pyigrf12 module, but installation through pip failed for me, which is what initially sparked my interest in f2py.
I will outline a few of the methods I am using. Regardless of the method, I always begin by creating a *.pyf signature file using f2py igrf12.f -m pyigrf12 -h igrf12.pyf
and adding intent(in/out) attributes appropriately.
- Method 1: Use C:\MinGW and
--compiler=mingw32
- Method 2: Use C:\MinGW and
--compiler=msvc
- Method 3: use anaconda version of mingw and
--compiler=mingw32
- Method 4: use anaconda version of mingw and
--compiler=msvc
Method 1:
For background, I have MinGW at C:\MinGW and I have added C:\MinGW\bin to my user Path. Unfortunately, I did not install this version of MinGW (I inherited this computer from a colleague), so I do not know where it was sourced. The gcc --version and gfortran --version is 5.3.0.
I run f2py -c igrf12.pyf igrf12.f --compiler=mingw32
. This fails with this error message:
Building import library (arch=AMD64):
"C:\Users\Sholes\AppData\Local\Continuum\anaconda3\libs\libpython36.a"
(from C:\Users\Sholes\AppData\Local\Continuum\anaconda3\python36.dll)
objdump.exe: C:\Users\Sholes\AppData\Local\Continuum\anaconda3\python36.dll: File format not recognized
Traceback (most recent call last):
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\Scripts\\f2py.py", line 28, in <module>
main()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\f2py\f2py2e.py", line 648, in main
run_compile()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\f2py\f2py2e.py", line 633, in run_compile
setup(ext_modules=[ext])
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\core.py", line 169, in setup
return old_setup(**new_attr)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\command\build.py", line 47, in run
old_build.run(self)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\command\build.py", line 135, in run
self.run_command(cmd_name)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\command\build_ext.py", line 117, in run
force=self.force)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\ccompiler.py", line 733, in new_compiler
compiler = klass(None, dry_run, force)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 104, in __init__
build_import_library()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 416, in build_import_library
return _build_import_library_amd64()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 472, in _build_import_library_amd64
generate_def(dll_file, def_file)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 302, in generate_def
raise ValueError("Symbol table not found")
ValueError: Symbol table not found
The issue seems to involve building libpython36.a from python36.dll.
After a quick google search, a suggestion on a github forum for pywafo was to use the msvc compiler instead of mingw32, leading to Method 2.
Method 2:
For background, the files related to my msvc are being pulled from C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.25017\bin\HostX86\x64\
. Not sure if that's helpful.
I run f2py -c igrf12.pyf igrf12.f --compiler=msvc
. This produces two files:
pyigrf12.cp36-win_amd64.pyd
in my current working directory and,libigrf12.BMWM6WD5Y3O3UTOEQITBXCIICXVMBEZS.gfortran-win_amd64.dll
in.\UNKNOWN\.libs\
When I try to import pyigrf12
, first I receive ImportError: DLL load failed: The specified module could not be found.
Using Dependency Walker, I receive a lot of errors:
but the most obvious one to address is related to the libigrf12.BMWM6WD5Y3O3UTOEQITBXCIICXVMBEZS.gfortran-win_amd64.dll
. I move this libigrf12 dll file to my current working directory, alongside pyigrf12.cp36-win_amd64.pyd
.
Now when I try import pyigrf12
, I receive ImportError: DLL load failed: %1 is not a valid Win32 application.
. Some stackoverflow posts seem to indicate that this is a problem with a conflict between a 32-bit dll and 64-bit Python. Can anyone offer insight into this? After more searching, I decided to try to use the anaconda version of mingw and libpython.
Method 3:
I run conda install mingw libpython
, and it installs mingw 4.7-1 and libpython 2.1-py36_0. I run f2py -c igrf12.pyf igrf12.f --compiler=mingw32
, and it fails with the following error message:
Building msvcr library:
"C:\Users\Sholes\AppData\Local\Continuum\anaconda3\libs\libvcruntime140.a"
(from C:\Users\Sholes\AppData\Local\Continuum\anaconda3\vcruntime140.dll)
objdump.exe: C:\Users\Sholes\AppData\Local\Continuum\anaconda3\vcruntime140.dll: File format not recognized
Traceback (most recent call last):
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\Scripts\\f2py.py", line 28, in <module>
main()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\f2py\f2py2e.py", line 648, in main
run_compile()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\f2py\f2py2e.py", line 633, in run_compile
setup(ext_modules=[ext])
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\core.py", line 169, in setup
return old_setup(**new_attr)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\command\build.py", line 47, in run
old_build.run(self)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\command\build.py", line 135, in run
self.run_command(cmd_name)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\command\build_ext.py", line 117, in run
force=self.force)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\ccompiler.py", line 733, in new_compiler
compiler = klass(None, dry_run, force)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 107, in __init__
msvcr_success = build_msvcr_library()
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 399, in build_msvcr_library
generate_def(dll_file, def_file)
File "C:\Users\Sholes\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 302, in generate_def
raise ValueError("Symbol table not found")
ValueError: Symbol table not found
Now the issue seems to relate to building libvcruntime140.a from vcruntime140.dll. Once again the dll file format is not recognized by objdump.exe.
Method 4:
My final attempt was to run f2py -c igrf12.pyf igrf12.f --compiler=msvc
with the anaconda version of mingw installed. For this case, gfortran failed with this error:
C:\Users\Sholes\AppData\Local\Continuum\anaconda3\Scripts\gfortran.bat -Wall -g -Wall -g -shared
..\..\..\AppData\Local\Temp\tmpugo__0q9\Release\igrf12.o -Lc:\users\sholes\appdata\local\continuum\anaconda3\mingw\lib\gcc\x86_64-w64-mingw32\4.7.0 -LC:\Users\Sholes\AppData\Local\Continuum\anaconda3\libs -LC:\Users\Sholes\AppData\Local\Continuum\anaconda3\PCbuild\amd64 -o C:\Users\Sholes\AppData\Local\Temp\tmpugo__0q9\Release\extra-dll\libigrf12.75XJA5DX6DTO7YIZ7X6ZHJYTRDCCYQYR.gfortran-win_amd64.dll -Wl,--allow-multiple-definition -Wl,--output-def,C:\Users\Sholes\AppData\Local\Temp\tmpugo__0q9\Release\libigrf12.75XJA5DX6DTO7YIZ7X6ZHJYTRDCCYQYR.gfortran-win_amd64.def -Wl,--export-all-symbols -Wl,--enable-auto-import -static -mlong-double-64
gfortran.exe: error: unrecognized command line option '-mlong-double-64'
At this point, I just want to know if it is possible to create fortran extensions with my set up and f2py. I don't have any background compiling C or fortran extensions on Windows, and based on all of the questions related to issues with Python 3.6 scipy and numpy installation on Windows, it seems like this is a common problem without a simple solution.
Any feedback or insight would be greatly appreciated.