I know that I know nothing... I am investigating performance of different approaches of parraller programming. And have some controversual results duting test. I want to clarify this things. I have 3 approaches for simultaneous reading and processing several files.
- classic sequential (assume simplest but worse performance)
- classic threads (is not best for this task, because use I/O but not processor)
- new asyncronius (best for this task, beacause use I/O)
I run this test for 3 cases 100 times 5 concurent thread prolonged timeout 100s
ab -n 100 -c 5 -s 100
here results: min, avg and max
- 3730 8653 14565 (best performance)
- 5090 7627 13888
- 12985 47973 71764 (worst performance)
How is it could be, the best method for this case shows worst results?
see below code it reads files from folder in the memory and just calculates it's total length this way.
1 sequential
int directorySize = 0;string fileNames = "";
foreach (var fileName in Directory.GetFiles(@"f:\Old\My Images\"))
{
StreamReader sw = new StreamReader(fileName);
string res = sw.ReadToEnd();
directorySize += res.Length;
fileNames += fileName + "<br/>";
}
ViewData["Message"] = directorySize +" "+ fileNames;
2 threaded
int directorySize = 0;string fileNames = "";
List<Task<string>> lst = new List<Task<string>>();
foreach (var fileName in Directory.GetFiles(@"f:\Old\My Images\"))
{
lst.Add(Task.Factory.StartNew(() =>{
StreamReader sw = new StreamReader(fileName);
return sw.ReadToEnd();
}));
fileNames += fileName + "<br/>";
}
while (lst.Count > 0)
{
var res = await Task.WhenAny(lst); lst.Remove(res);
directorySize += res.Result.Length;
}
ViewData["Message"] = directorySize + " " + fileNames;
3 async
int directorySize = 0;string fileNames = "";
List<Task<string>> lst = new List<Task<string>>();
foreach (var fileName in Directory.GetFiles(@"f:\Old\My Images\"))
{
StreamReader sw = new StreamReader(fileName);
lst.Add(sw.ReadToEndAsync());
fileNames += fileName + "<br/>";
}
while(lst.Count>0)
{
var res = await Task.WhenAny(lst);lst.Remove(res);
directorySize += (await res).Length;
}
ViewData["Message"] = directorySize + " " + fileNames;