microbenchmark | accurately measure and compare the execution time | Document Editor library

Marginally faster

You pasted the dependencies into the wrong section. You need to crate dependencies tag and paste the dependencies there:

It might be helpful if you provide at least some explanation of what it is you are doing.

First, I recommend breaking up the code into very small chunks, microbenchmarking each chunk, and finding bottleneck operations. This is better than throwing the whole function at the wall all at once.

join_rows(X,Xt)

This is a very slow operation, especially because you are adding a rowvec to a column-oriented mat. Armadillo matrices are stored as a vector in column-major order, so behind the scenes Armadillo is calling push_back in n non-contiguous locations, where n is the number of columns.

It seems you may be able to avoid this altogether, as finalX.row(i) is the only call that depends on finalX in this loop. Just figure out what .row(i) is.

You do a lot of transposing, maybe you should be working with a transposed matrix from the get-go?

Xi.t() is called twice in the same line: finalfirst -= (Xi.t() - rat.col(j)) * exp(arma::as_scalar(betas*Xi.t()));

Transposing a row vector is pretty pointless, just initialize it as a column vector and iterate through it with a good old-fashioned loop. A little more code isn't always a bad thing, it often makes your intent more explicit too.

This is a copy: final.col(i) = finalfirst;

Why not operate in-memory and update final(_, i) in-place rather than using a temporary finalfirst followed by a deep copy into your target matrix? This is a column, so memory is contiguous and the compiler will be able to optimize the access pattern for you as wonderfully as if you were working on any simple vector.

All said, I haven't fully wrapped my head around what it is exactly that you are doing, but @largest_prime_is_463035818 may be right about switching the for(i...) and for(j...) loops around. It appears you will be able to pull these two rows out of the nested loop:

One way could be:

You can try apply(A, 2, '*', B) and to come the the same like colmat_prod use array(apply(A, 2, '*', B), c(dim(B), ncol(A))):

[EDIT 2021-03-06: thanks to a series of commits over the past ~day (thanks, Liz!), this slowdown is now largely fixed on HEAD; these performance gains should show up in the next monthly release. I'm leaving the answer below as an example of how to dig into this sort of issue, but the specific problems it diagnosed have largely been resolved.]

Building on @Elizabeth Mattijsen's comment: The slow performance here is mostly due to the Rakudo compiler not properly optimizing the generated code (as of 2021-03-05). As the compiler continues to improve, the (idiomatic) code you wrote above should perform much better.

However, for today we can use a few workarounds to speed up this calculation. While it's still true that Raku's performance won't be particularly competitive with R here, some profiling-driven refactoring can make this code nearly an order of magnitude faster.

Here's how we do it:

First, we start by profiling the code. If you run your script with raku --profile=, then you'll get a profile written to . By default, this will be an HTML file that allows you to view the profile in your browser. My preference, however, is to specify an .sql extension, which generates an SQL profile. I then view this profile with MoarProf, the revised profiler that Timo Paulssen is building.

Looking at this profile shows exactly the issue that Liz mentioned: Calls that should be getting inlined are not. To fix this, let's create our own sorting function, which the JIT compiler will happily optimize:

Since I am not aware of a ready-made way of computing your output in any standard library, I came up with the implementation roll_mean_k_efficient below, which seems to speed up your computations considerably. Note that this implementation makes use of the rollapply and the na.locf methods from the zoo package.

It's actually quite simple to create an Rcpp SparseMatrix class! I was overthinking it.

Slots are stored as attributes. We have a couple of options for converting a slot to NULL.

Option 1: You can use the check=FALSE argument in slot<- to assign a slot as NULL without triggering an error.