I would like to use system.time in R to get the total CPU time on a multicore function. The problem is that system.time does obviously not capture CPU time spend by the child processes spawned by the parallel package.
library(doParallel)
cl <- makeCluster(2)
registerDoParalllel(2)
timings <- system.time(foreach(i = 1:2) %do% rnorm(1e8))
Timings then looks like this
> timings
user system elapsed
16.883 5.731 22.899
The timings add up. Now if I use parallel processing:
timings <- system.time(foreach(i = 1:2) %dopar% rnorm(1e8))
> timings
user system elapsed
2.445 3.410 20.347
The user and system time are only capturing the master process. Specifically looking at the timings[4] and [5] shows me that the user.child and sys.child times are 0.
What do I have to do to measure total CPU time in R on parallel processing?
Note: Moving the cluster startup code into the system.time call did not make a difference.
> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 8 (jessie)
other attached packages:
[1] doParallel_1.0.10 iterators_1.0.8 foreach_1.4.3
@chinsoon12 pointed me in the right direction. user.child and sys.child are only populated when the cluster is created by registerDoParallel, e.g.
This is why it worked out of the box with doMC where I did not manually start and stop the cluster via the cl variable.