pbapply | Adding progress bar to ' * apply ' functions in R | Development Tools library

The simplest way I've found so far to use a parallel lapply in R was through the following example code:

I use pblapply() to run code in parallel. It gives a progression bar, which is useful. I use function like clusterExport() or clusterEvalQ() to export objects or librairies into clusters. But I didn't manage to share a home-made function, available in my environment. Can you help me with this ?

Here is an very basic example:

I am trying to write a function which has a parallel computation option. To make it work in both windows, and mac or Linux environments, I am using a PSOCK system, which I believe is a default configuration in makeCluster(). My question is whether I should, or it is more desirable, to pass all arguments to the clusters using a clusterExport function. If I do this, I think I need to evaluate all input arguments-- instead of the default lazy evaluation. If some variables are used only in some special cases, this does not seem desirable.

For example, in the following code, I am wondering whether I should add clusterExport(varlist = c("a","b","c"),cl = cl,envir = environment()) in the function. The following code works fine in my computer, but a similar code failed in other's computer.

I would be very interested to hear about the best practice as well. Thank you!

I have N tasks which I wish to parallelize across M cores in R. These tasks vary significantly in how long they will take to perform. Fortunately, I have a rough indication of which tasks are going to be slow and which are going to be fast. I also have a progress bar which increments by the number of tasks completed.

I want to optimise the order of the tasks such that:

1. Total computation time is minimised
2. The progress bar is representative of the time remaining, not the tasks remaining.

To give a more practical example:

The makeCluster function sometimes hangs in my code and just needs to be rerun to fix the issue.

In order to do that in my code I'm trying to use a while loop plus the withTimeout function to have the makeCluster function timeout if it is hanging and rerun itself.

The issue is that when I create my cluster with withTimeout I can't access it later for my parallel lapply.

I am trying to use OMBP API in R but not able to use it properly

I would like to monitor the progress of my mapply function. The data consists of 2 lists and there is a function with 2 arguments.

If I do something similar with a function that takes 1 arguments I can use ldply instead of lapply. (I'd like to rbind.fill the output to a data.frame)

If I want to do the same with mdply it doesn't work as the function in mdply wants values taken from columns of a data frame or array. Mapply takes lists as input.

These plyr apply functions are handy, not just because I can get the output as a data.frame but also because I can use the progress bar.

I know there is the pbapply package but that there is no mapply version and there is the txtProgressBar function but I could not figure out how to use this with mapply.

I tried to create a reproducible example (takes around 30 s to run)

I guess bad example. My l1 is a list of scraped websites (rvest::read_html) which I cannot send as a data frame to mdply. The lists really need to be lists.

I have written the ggstatsplot package to carry out some statistical analysis. The package functions (in the development version) can either return a plot or a call that contains statistical details displayed in the subtitle of the plot.

Here is an example of a plot as a return:

I have this code where I want to sum the columns of a matrix.

I also want to have a couple of features (my matrix is much bigger than the example below):

• use multiple cores
• have a progress bar

I tried pbapply from the pbapply package but without luck. My code is: