Put function in the background

Question

I have the following function:

def update_contacts(data):
    '''
    Update a user's contacts from Google: to be run as a background task.
    '''
    from users.google_oauth import GoogleOauthClient
    email = data['email']
    access_token = data['access_token']
    g = GoogleOauthClient()
    contacts = g.get_all_contacts(email=email, access_token=access_token, insert=True)
    log.info('Fetched and updated %s contacts' % (len(contacts)))

I am looking to create a generic function that will run other functions in the background, such as the above. Here is what I have so far:

def run_in_background(function):
    '''
    I want this to be able to receive a normal function call,
    such as `update_contacts(data)` or get_first_name('tom')
    '''

    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    loop.run_in_executor(None, function, data)

And I would then call this passing :

data={'email': email,'access_token': g.tokens['access_token']}
run_in_background (
    update_contacts(data)
)

The problem is, I think that it will first run the update_contacts function and not really do anything async. How would I properly write and call the run_in_background function?

Perhaps I would need to use something like partial to pass the function without actually calling it until it's executed?

Answer 1

Not sure about asyncio. However, I believe you can use threading for the same, Your function can then be modified as follows:

def background_function(func, params):
    t1 = threading.Thread(target=func, args = params)
    t1.start()

Sample call:

 def do_something(num):
   print('printing' + num + 'times')

def call_do_domething():
   background_function(do_something, args = [1000])

Answer 2

As other pointed out, the expression run_in_background(update_contacts(data)) will evaluate update_contacts(data) before even calling run_in_background. Regardless of how run_in_background is implemented, you have to pass it a function.

The run_in_background function you're looking for already exists, it's the submit method on the ThreadPoolExecutor:

with ThreadPoolExecutor() as executor:
    data = { ... }
    future = executor.submit(update_contacts, data)
    # ... the function now runs in a background thread,
    # and you can do other things here ...

You can use methods such as done() or result() on the returned Future object to test whether the submitted task is done or to pick up the result of the function. This is preferable to starting a thread manually because the executor can support a large number of background tasks by maintaining a pool of threads. There is also an executor that uses multiprocessing to provide true parallelism.

All this is completely independent of the asyncio library and its run_in_executor method, which serves to connect blocking code with code written specifically for asyncio, which you don't appear to have.

Answer 3

If you call run_in_background (update_contacts(data)), it means you already called function update_contacts(data). But you should only pass function and its arguments like this:

run_in_background(update_contacts, args=(data,))

And change your run_in_background function accordingly

def run_in_background(function, args=()):
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    loop.run_in_executor(None, function, *args)