Limitations of work-item load in GPU? CUDA/OpenCL

Question

I have a compute-intensive image algorithm that, for each pixel, needs to read many distant pixels. The distance is dependent on a constant defined at compile-time. My OpenCL algorithm performs well, but at a certain maximum distance - resulting in more heavy for loops - the driver seems to bail out. The screen goes black for a couple of seconds and then the command queue never finishes. A balloon message reveals that the driver is unhappy:

"Display driver AMD driver stopped responding and has successfully recovered."

(Running this on OpenCL 1.1 with an AMD FirePro V4900 (FireGL V) Graphics Adapter.)

1. Why does this occur?
2. Is it possible to, beforehand, tell the driver that everything is ok?

Answer 1

This is a known "feature" under Windows (not sure about Linux) - if the video driver stops responding, the OS will reset it. Except that, since OpenCL (and CUDA) is implemented by the driver, a kernel that takes too long will look like a frozen driver. There is a watchdog timer that keeps track of this (5 seconds, I believe).

Your options are:

1. You need to make sure that your kernels are not too time-consuming (best).
2. You can turn-off the watchdog timer: Timeout Detection and Recovery of GPUs.
3. You can run the kernel on a GPU that is not hooked up to a display.

I suggest you go with 1.