Create a Message Passing Interface (MPI) Cluster of R Workers for Parallel Processing

The makeClusterMPI() function creates an MPI cluster of R workers for parallel processing. This function utilizes makeCluster(..., type = "MPI") of the parallel package and tweaks the cluster in an attempt to avoid stopCluster() from hanging (1). WARNING: This function is very much in a beta version and should only be used if parallel::makeCluster(..., type = "MPI") fails.

makeClusterMPI(
  workers,
  ...,
  autoStop = FALSE,
  verbose = getOption2("parallelly.debug", FALSE)
)

Arguments

workers: The number workers (as a positive integer).
...: Optional arguments passed to makeCluster(workers, type = "MPI", ...).
autoStop: If TRUE, the cluster will be automatically stopped using stopCluster() when it is garbage collected, unless already stopped. See also autoStopCluster().
verbose: If TRUE, informative messages are outputted.

Value

An object of class c("RichMPIcluster", "MPIcluster", "cluster") consisting of a list of "MPInode" workers.

Details

Creating MPI clusters requires that the Rmpi and snow packages are installed.

Alternative usage

In R (>= 4.4.0), an alternatively to using cl <- parallelly::makeClusterMPI(workers) is:

cl <- parallel::makeCluster(workers, type = parallelly::MPI)

References

R-sig-hpc thread Rmpi: mpi.close.Rslaves() 'hangs' on 2017-09-28.

Examples

if (FALSE) {
if (requireNamespace("Rmpi") && requireNamespace("snow")) {
  cl <- makeClusterMPI(2, autoStop = TRUE)
  print(cl)
  y <- parLapply(cl, X = 1:3, fun = sqrt)
  print(y)
  rm(list = "cl")
}
}