Factorization | It's factorization. - | Video Game library

Your training data contains only a single sampling point with multiple values. Replace with more sound data and your code works:

There is at least two things wrong with your parallelization

Based on the code that you have provided it seems to me that you do not need any lock whatsoever, try the following parallel version:

You want a discriminated union type. C# does not support them natively as a first-class language feature but you can hack-it using OneOf: https://github.com/mcintyre321/OneOf

So you would end-up with:

The problem is that you don't change n or m inside the while loop. So, for example, for input n=4, m=2, k comes out to be n/m=2 which satisfies k%m==0 and since neither n nor m changes so it runs forever.

You can simplify the code by modifying n in the while loop to keep decreasing if it is divisible by the current divisor m. You can't do the same for k since k is reset to n again with the line k = n and it will start with the original number giving incorrect output.

Here is a bit modified version of the code with the outer while loop:

I think you need to look more closely at your data. If I run a modification of your code on Google Colab (Tesla T4) I get this:

Which looks largely like your figure. But look more closely (log scales help):

You can clearly see that up to a certain point, the runtime is largely independent of the number of matrices (around 2^8 = 64), but then scaling is linear as sizes increase. That is the transition from being able to parallelize the workload to reaching parallel capacity and having to schedule many parallel groups of operations to execute the workload. You might infer that for this particular GPU, the GPU run out of parallel capacity at between 64 and 128 concurrent operations (The T4 has 40 SM, so it might well be 80 if an SM could accommodate 2 operations per SM concurrently), after which runtime scales with multiples of that limiting size.

This is completely normal behaviour for any parallel computation architecture I am familiar with.

A lot of people use 1/2 MSE for the loss because it makes the derivative "easier". Given that they use the word "loss" rather than "MSE" or something like that, I'd bet this is what's going on.

For clarity, if your loss is

1/2n * [(y_1 - p_1)^2 + ... + (y_n - p_n)^2]

then the derivative (wrt p) would be

-1/n * [(y_1 - p_1) + ... + (y_n - p_n)]

The 2 goes away because you end up multiplying by 2 for the power rule.

pardon the formatting... I don't know how to do math stuff here.

There are several issues here. To start with, your factor function regenerates a new pq value, ignoring the one you produced earlier. This counts the time to get pq twice. Also, measuring time on a single execution for a process that uses a random number generator doesn't give you a good idea of the performance. You need to make several runs to get an average.

I cleaned up your code a bit and changed the way time is measured to get an average over 100 runs:

your code:

k = (exponent - 2) / 4 makes k a float, which means you potentially introduce numerical error in computations down the line. Use integer division to stay in int world from the start: