CUDA apps time out & fail after several seconds -

Question

I've noticed that CUDA applications tend to have a rough maximum run-time of 5-15 seconds before they will fail and exit out. I realize it's ideal to not have CUDA application run that long but assuming that it is the correct choice to use CUDA and due to the amount of sequential work per thread it must run that long, is there any way to extend this amount of time or to get around it?

Answer 1

The solution I use is:

1. Pass all information to device.
2. Run iterative versions of algorithms, where each iteration invokes the kernel on the memory already stored within the device.
3. Finally transfer memory to host only after all iterations have ended.

This enables control over iterations from CPU (including option to abort), without the costly device<-->host memory transfers between iterations.

Answer 2

This isn't possible. The time-out is there to prevent bugs in calculations from taking up the GPU for long periods of time.

If you use a dedicated card for CUDA work, the time limit is lifted. I'm not sure if this requires a Tesla card, or if a GeForce with no monitor connected can be used.