Using f2py on a Fortran code linked to PETSc

Question

My question is related to this post:

Including a compiled module in module that is wrapped with f2py (Minimum working example)?

in which the poster was trying to compile a Fortran code (Test.f90) with f2py and link that to a pre-compiled library (or in my case, object, myex44f.o). The answer enabled me to compile the Fortran code and generated the python module.

My problem is different from the above posters problem in that my object is linked to PETSc. When I try to import my f2py-generated library into python, I get the error that it cannot locate 'VecDestroy', a PETSc subroutine. My most recent attempt was:

f2py -c --fcompiler=gfortran -I. myex44f.o ../../../Codes/third_party/petsc/include/petsc/finclude/petscdef.h -m test Test.f90

Here is the code Test.f90:

subroutine test  
USE petsctest  
call mainsub  
end subroutine test

which calls mainsub from the module petsctest:

      module petsctest   !   Solves the linear system  J x = f
#include <petsc/finclude/petscdef.h>
      contains
      subroutine mainsub
      use petscksp; use petscdm
      Vec x,f
      Mat J
      DM da
      KSP ksp
      PetscErrorCode ierr
      call PetscInitialize(PETSC_NULL_CHARACTER,ierr)

      call DMDACreate1d(MPI_COMM_WORLD,DM_BOUNDARY_NONE,8,1,1,        &
     &  PETSC_NULL_INTEGER,da,ierr)
      call DMCreateGlobalVector(da,x,ierr)
      call VecDuplicate(x,f,ierr)
      call DMSetMatType(da,MATAIJ,ierr)
      call DMCreateMatrix(da,J,ierr)

      call ComputeRHS(da,f,ierr)
      call ComputeMatrix(da,J,ierr)

      call KSPCreate(MPI_COMM_WORLD,ksp,ierr)
      call KSPSetOperators(ksp,J,J,ierr)
      call KSPSetFromOptions(ksp,ierr)
      call KSPSolve(ksp,f,x,ierr)

      call MatDestroy(J,ierr)
      call VecDestroy(x,ierr)
      call VecDestroy(f,ierr)
      call KSPDestroy(ksp,ierr)
      call DMDestroy(da,ierr)
      call PetscFinalize(ierr)
      end

The error that I get is:

import test Traceback (most recent call last): File "", line 1, in ImportError: ./test.so: undefined symbol: vecdestroy_

Does anyone have any suggestions? Thank you very much for any help you can provide me.

UPDATE: I generated the original myex44f.o object using the makefile provided with the PETSc examples. Looking at the link line, I reasoned that I might need to link the petsc library when compiling with f2py. My current attempt is:

f2py -c --fcompiler=gfortran -m test Test.f90 -I. myex44f.o -I/home/costoich/Documents/AFPWork/Codes/third_party/petsc/include -I/home/costoich/Documents/AFPWork/Codes/third_party/petsc/arch-linux2-c-debug/include -L/home/costoich/Documents/AFPWork/Codes/third_party/petsc/arch-linux2-c-debug/lib -lpetsc

This seems to be linking correctly during the compile steps (if I just write -lpetsc without the path the compiler fails). However, when I type ldd test.so, I get:

linux-vdso.so.1 => (0x00007ffe09886000) libpetsc.so.3.7 => not found libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007fc315be5000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc31581b000) libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fc3155dc000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc3152d3000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc3150bc000) /lib64/ld-linux-x86-64.so.2 (0x000055a3fad27000)

Do I need two use the link flags Wl,rpath? f2py seems to not understand these. Thank you for any comments.

RESOLVED I found my issue. I can't get f2py to accept the -Wl,rpath options, but if I define the environment variable LD_LIBRARY_PATH=/home/costoich/Documents/AFPWork/Codes/third_party/petsc/arch-linux2-c-debug/lib everything works out. Thank you for your help.

Answer 1

@VladimirF, has a point.

It looks like if the VecDestrou is not in the PETSC module you are using.

Seems to me following parts of PETSc are required in your module.

#include <petsc/finclude/petscsysdef.h>
#include <petsc/finclude/petscvecdef.h>
! Optional
#include <petsc/finclude/petscdef.h>
#include <petsc/finclude/petscdm.h>
#include <petsc/finclude/petscvec.h> 
#include <petsc/finclude/petscvec.h90>  
#include <petsc/finclude/petscmat.h>
#include <petsc/finclude/petscmat.h90>
! might be not completed
! Or
use petscksp
use petscdm
use petscvec
use petscmat 
!might be not completed

How to use PETSc with Fortran is diccussed here, personally I go for option 2 in that page. Most of the existing PETSc examples are following option 2 as well.

Please let me clarify that I am not encouraging you to use include over use, that's the way I am used to do only. PETSc documentation has an example that using Fortran modules, for example here. So you can choose any of these methods/or have both optionally (please realize that preprocessor option in that example, PETSC_USE_FORTRAN_MODULES), but still need to add required modules depends on what you are using.