Google Cloud Function Finishes Execution But Never

Question

I am trying to make ~25 requests to fetch some data and save the response to separate files. I am using the npm request module to make the requests with a basic google cloud function. Each request can take anywhere from 0.5 - 5 seconds to return a response. When the script is setup to make less than 10 requests, everything works as expected and saves everything to the proper files. However any more than that and I don't get a response from any request.

for (i = 0; i < popIDS.length; i++) {
    for (j = 0; j < genreIDs.length; j++) {
        fetchAndSaveToFile(popIDS[i], genreIDs[j], date);            
    }
}  

fetchAndSaveToFile makes the npm request and uses pako to save the compressed response to a file in Google Cloud Storage. This was originally an issue making ~5 requests but I increased the timeout for the function and it started working, however I maxed out the timeout for the function to 9 minutes. All functions are currently in one file.

Also it's worth noting that the value of popIDS[i], genreIDs[j] are not used inside the callback and 'i' and 'j' do not need to be captured.

Do I need to make async requests differently for this to work? I see there is async request module that handles this sort of thing. Are there function settings that I need to configure? Or do I need to split this out into separate files so that I can set a 9 minute timeout for each request?

Answer 1

I was able to resolve by updating to use Promise.all() and mapping the request calls to promise objects using new Promise. Returning Promise.all() to the main Google Cloud Function body let's the function know to wait for all promises to finish before exiting the function. Implementing callbacks instead of promises may cause the Google cloud function to exit prematurely before all callbacks are finished.