R package parallel: how to print the output from o

Question

I am using the function parSapply from the R package parallel. My call to this function is something like:

cl <- makeCluster(3, type="PSOCK",outfile="output.txt")
m<-10
parSapply(cl,as.list(1:m), FUN=function(mtmp){
 comp<-0
 for (ii in 1:10){
  print(ii)
  comp<-comp+rnorm(1)
 }
 return(comp)
})

The parallelized function prints a message to follow the process. This is very useful to estimate the time required by the function. With this code, the printed message for each node are stocken in a txt file (output.txt), but there are melted since all nodes generate messages in a same time. Therefore, to have printed messages that are readable in my txt file, I would like to follow the process for only one node. I was thinking that for mtmp in {1,4,7,10} the iterations was performed on a same node. So, I tried to add the condition:

if(mtmp%%3==1){print(ii)}

but, the messages are again melted, which indicates that calls for mtmp in {1,4,7,10} are not performed on a same node.

Therefore, my question is: how can I save in my txt file all the printed messages of one node and only of this one ? In addition, I would like this node is the one which handles the larger number of calls to my parallelized function.

Thank you very much for your help,

Vincent

Answer 1

Divert the output of each process to a separate file:

library(parallel)
cl <- makeCluster(3, type="PSOCK")
#divert to different files:
clusterEvalQ(cl, sink(paste0("E:/temp/output", Sys.getpid(), ".txt")))

m<-10

parSapply(cl,as.list(1:m), FUN=function(mtmp){
  comp<-0
  for (ii in 1:10){
    print(ii)
    comp<-comp+rnorm(1)
  }
  return(comp)
})

stopCluster(cl)