pyCUDA with Flask gives pycuda._driver.LogicError:

Question

I want to run a pyCUDA code on a flask server. The file runs correctly directly using python3 but fails when the corresponding function is called using flask.

Here is the relevant code:

cudaFlask.py:

import pycuda.autoinit
import pycuda.driver as drv
import numpy

from pycuda.compiler import SourceModule

def cudaTest():
    mod = SourceModule("""
        int x = 4;
    """)

    print ("done")
    return

if __name__ == "__main__":
    cudaTest()

server.py (only the part which calls the function):

@app.route('/bundle', methods=['POST'])
def bundle_edges():
    cudaTest()
    return "success"

On running python cudaFlask.py I get the output done as expected but on starting the server and doing POST request at website/bundle I get the following error on the flask console:

pycuda._driver.LogicError: cuModuleLoadDataEx failed: invalid device context - 

on the line mod = SourceModule...

Where am I going wrong? There is a similar question out there but it has not been answered yet.

Answer 1

Solved the issue with lazy loading in flask and making the context manually (i.e. without pycuda.autoinit in PyCUDA.

Refer this for lazy loading in flask.

My views.py file:

import numpy as np
import pycuda.driver as cuda
from pycuda.compiler import SourceModule

def index():
    cuda.init()
    device = cuda.Device(0) # enter your gpu id here
    ctx = device.make_context()

    mod = SourceModule("""
        int x = 4;
    """)

    ctx.pop() # very important

    print ("done")
    return "success"

Answer 2

PyCUDA may not be compatible with WSGI web server contexts. You could possibly make it work if you use some kind of message queue like Celery, where the HTTP request places a job on the queue and the worker on the other side of the queue runs the CUDA program.

Edit: A quick and easy way would be to use Python Subprocess check_output function

In the web request:

subprocess.check_output(['python', 'cudaFlask.py'])