Poor throughput on Azure Functions

Question

I'm evaluating Azure Functions to create thumbnail images. The source URLs of larger images are placed in a storage queue, and a C# function with a queue trigger is used to process the URLs (download from source, resize, and upload to another location).

Each function call takes under 500ms for processing, which is fine. However, after running a bunch of tests, I've found that the overall parallel processing throughput is not that great. With workloads of 1500-2000 items in the queue, the platform only executes around 10 function instances per second.

Is there any way to scale out and make the platform execute more function instances concurrently?

Answer 1

When running under the Consumption (Dynamic) plan, the system will scale out to more instances automatically when we see that your function is not keeping up. This scale out is not instantaneous, so it might be that your test concluded before or shortly after more instances were added before the effects of those new instances could be seen. When running in an App Service (Classic) plan, you control the number of instances up front and can scale out to the number you require.

There are a few configuration knobs for queues that you can set in your host.json file that affect the amount of parallelism per Function App instance. Under the queues configuration section, you can set batchSize and newBatchThreshold, e.g.:

{
   "queues": {
      "batchSize": 32,
      "newBatchThreshold": 50
    }
}

batchSize is the number of messages that are pulled from the queue on each fetch. All the messages in a batch are then processed in parallel. newBatchThreshold governs when the next batch of messages will be fetched. A new batch of messages is only fetched from the queue when the number of messages currently being processed drops below this threshold. So increasing newBatchThreshold will allow more messages to be processed in parallel. See the wiki here for more details on these settings.

Note that you have to take your workload into account when adjusting these settings. For example, if your function is very memory/CPU intensive, you can't run too many of them in parallel on a single instance. So you may have to experiment a bit.

In addition to all of the above, you should also make sure your function is a proper async function, to ensure threads aren't being blocked unnecessarily on IO, and that no other potential bottle-necks exist in your function code itself.