I'm running a process that periodically Curls to a list of URLs to test out performance to those websites. My main program is basically a do...sleep...while loop that calls a function, runTest() every N seconds, with a keyboard interrupt to kill it off. Pared down to the basics, my code is as follows:
public void runTest()
{
if(isRunning)
{
return; //Plus some logging that one or more of the tests hasn't completed
}
try {
isRunning = true;
Curl.GlobalInit((int)CURLinitFlag.CURL_GLOBAL_ALL);
CancellationTokenSource cts = new CancellationTokenSource(httpTimeoutValueMs * 2);
foreach (var url in urls)
taskList.Add(doAsyncCurls(url, cts))
List<CurlResults> listOfResults = Task.WhenAll(taskList.Select(x => x)).Result.ToList();
taskList.Clear();
}
catch {/*Some exception handling code*/}
finally {
isRunning = false;
Curl.GlobalCleanup();
}
}
private static async Task<CurlResults> doAsyncCurls(string url, CancellationTokenSource cts)
{
try
{
/*Initiate a Curl using libCurl*/
Easy easy = new Easy();
easy.SetOpt(CURLoption.CURLOPT_URL, url);
easy.SetOpt(CURLoption.CURLOPT_DNS_CACHE_TIMEOUT, 0); //Disables DNS cache
easy.SetOpt(CURLoption.CURLOPT_CONNECTTIMEOUT, httpTimeoutValueMs / 1000); //20sec timeout
Task t = new Task(() => easy.Perform(), cts.Token);
t.Start();
await t;
return new CurlResults(/*valid results parameters*/);
}
catch (TaskCanceledException)
{ /*Cancellation token invoked*/
return new CurlResults(/*Invalid results parameters*/);
}
catch (Exception e)
{ /*Other exception handling*/
return new CurlResults(/*Invalid results parameters*/);
}
The "doAsyncCurl" function does what it says on the tin, setting an http timeout value to half that of the cancellation token, so that the HTTP request should evaporate before the cancellation token is invoked, and generate a negative result. I added the cancellation token to try and deal with my problem as below.
I leave this running for ages - to start with, everything works well, but eventually (hundreds, if not more iterations through the periodic runTest() invocation) it seems that one of the tasks gets stuck and the cancellation token isn't invoked, and the curl testing process is effectively stuck.
Apart from picking up which URL(s) are problematic (and they're all valid ones), what can be done to diagnose what's going wrong? Or have I constructed my parallel Curls completely wrongly to start with? (I appreciate that I could instantiate new Curls to URLs that have finished from the previous round and leave the hung one to dangle, rather than waiting for them all to finish before kicking off a new batch, but I'm not bothered about that efficiency gain).