How to parallelize many (100+) tasks without hitti

Question

The problem:

When lazy-loading a list of 100+ icons in the background, I hit the GCD thread limit (64 threads), which causes my app to freeze up with a semaphore_wait_trap on the main thread. I want to restructure my threading code to prevent this from happening, while still loading the icons asynchronous to prevent UI blocking.

Context:

My app loads a screen with SVG icons on it. The amount differs on average from 10-200. The icons get drawn by using a local SVG image or a remote SVG image (if it has a custom icon), then they get post-processed to get the final image result.

Because this takes some time, and they aren't vital for the user, I want to load and post-process them in the background, so they would pop in over time. For every icon I use the following:

dispatch_queue_t concurrentQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(concurrentQueue, ^{
    //code to be executed in the background
    SVGKImage *iconImage = [Settings getIconImage:location];
    dispatch_async(dispatch_get_main_queue(), ^{
        //code to be executed on the main thread when background task is finished
        if (iconImage) {
            [iconImgView setImage:iconImage.UIImage];
        }
    });
});

The getIconImage method handles the initial loading of the base SVG, which reads it synchronized with [NSInputStream inputStreamWithFileAtPath:path] if local, and [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&errorWithNSData] if it should load remotely. This all happens synchronous.

Then there is some post-processing of recoloring the SVG, before it gets returned and put in the UIImageView on the main thread.

Question:

Is there a way to structure my code to allow for parallelized background loading but prevent deadlock because of too many threads?

Solution EDIT:

_iconOperationQueue = [[NSOperationQueue alloc]init];
_iconOperationQueue.maxConcurrentOperationCount = 8;    

// Code will be executed on the background
[_iconOperationQueue addOperationWithBlock:^{
    // I/O code
    SVGKImage *baseIcon = [Settings getIconBaseSVG:location];

    // CPU-only code
    SVGKImage *iconImage = [Settings getIconImage:location withBaseSVG:baseIcon];
    UIImage *svgImage = iconImage.UIImage; // Converting SVGKImage to UIImage is expensive, so don't do this on the main thread
    [[NSOperationQueue mainQueue] addOperationWithBlock:^{
        // Code to be executed on the main thread when background task is finished
        if (svgImage) {
            [iconImgView setImage:svgImage];
        }
    }];
}];

Answer 1

You can stay with GCD by using a semaphore something like this running the whole operation in the background otherwise waiting for the semaphore will stall the UI:

dispatch_semaphore_t throttleSemaphore = dispatch_semaphore_create(8);
dispatch_queue_t concurrentQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
for /* Loop through your images */ {
    dispatch_semaphore_wait(throttleSemaphore, DISPATCH_TIME_FOREVER);
    dispatch_async(concurrentQueue, ^{
        //code to be executed in the background
        SVGKImage *iconImage = [Settings getIconImage:location];
        dispatch_async(dispatch_get_main_queue(), ^{
            //code to be executed on the main thread when background task is finished
            if (iconImage) {
                [iconImgView setImage:iconImage.UIImage];
            }
            dispatch_semaphore_signal(throttleSemaphore);
        });
    });
}

Answer 2

Instead of directly using GCD with a concurrent queue, use an NSOperationQueue. Set its maxConcurrentOperationCount to something reasonable, like 4 or 8.

If you can, you should also separate I/O from pure computation. Use the width-restricted operation queue for the I/O. The pure computation you can use an unrestricted operation queue or pure GCD for.

The reason is that I/O blocks. GCD detects that the system is idle and spins up another worker thread and starts another task from the queue. That blocks in I/O, too, so it does that some more until it hits its limit. Then, the I/O starts completing and the tasks unblock. Now you have oversubscribed the system resources (i.e. CPU) because there are more tasks in flight than cores and suddenly they are actually using CPU instead of being blocked by I/O.

Pure computation tasks don't provoke this problem because GCD sees that the system is actually busy and doesn't dequeue more tasks until earlier ones have completed.