parallel | Parallel examples and test harness | Architecture library

The issue you are having has nothing to do with templates. You cannot invoke std::async() on a member function without binding it to an instance. Wrapping the call in a lambda does the trick.

Here's an example:

I'm trying to parallelize a merge-sort algorithm. What I'm doing is dividing the input array for each thread, then merging the threads results.

Ok, but yours is an unnecessarily difficult approach. At each step of the merge process, you want half of your threads to wait for the other half to finish, and the most natural way for one thread to wait for another to finish is to use pthread_join(). If you wanted all of your threads to continue with more work after synchronizing then that would be different, but in this case, those that are not responsible for any more merges have nothing at all left to do.

This is what I've tried:

Send your data to separate processes. I used concurrent.futures because I like the simple interface.

This runs in about 11 seconds on my computer.

There are multiple potential places to measure execution time, first might be defining very specifically what is the intended measurement.

1. Measuring the training time of each client as proposed is a great way to get a sense of the variability among clients. This could help identify whether rounds frequently have stragglers. Using tf.timestamp() at the beginning and end of the client_update function seems reasonable. The question correctly notes that this happens in parallel, summing all of these times would be akin to CPU time.

2. Measuring the time it takes to complete all client training in a round would generally be the maximum of the values above. This might not be true when simulating FL in TFF, as TFF maybe decided to run some number of clients sequentially due to system resources constraints. In practice all of these clients would run in parallel.

3. Measuring the time it takes to complete a full round (the maximum time it takes to run a client, plus the time it takes for the server to update) could be done by moving the tf.timestamp calls to the outer training loop. This would be wrapping the call to trainer.next() in the snippet on https://www.tensorflow.org/federated. This would be most similar to elapsed real time (wall clock time).

Pythons multiprocessing package is limited to shared memory parallelization. It spawns new processes that all have access to the main memory of a single machine.

You cannot simply scale out such a software onto multiple nodes. As the different machines do not have a shared memory that they can access.

To run your program on multiple nodes at once, you should have a look into MPI (Message Passing Interface). There is also a python package for that.

Depending on your task, it may also be suitable to run the program 4 times (so one job per node) and have it work on a subset of the data. It is often the simpler approach, but not always possible.

"Map" is also a synonym for "function" in the mathematical sense: something that sends an input to an output. You should be able to find it in any English dictionary. It can also be used as a verb for the process of transformation: "map each element to its square".

The word "map" for a geographic drawing is related, in that it also "maps" each point of the real terrain to a point on the paper map, or vice versa.

It is not an acronym.

https://docs.python.org/3/library/math.html

The math module consists mostly of thin wrappers around the platform C math library functions.

While python itself can only execute a single instruction at a time, a low level c function that is called by python does not have this limitation.
So it's not python that is using multiple cores but your system's well optimized math library that is wrapped by python's math module.

That basically answers both your questions.

Regarding the usefulness of multiprocessing: It is still useful for those cases, where you're trying to parallelize pure python code or code that does not call libraries that already use multiple cores. However, it comes with inter process communication (IPC) overhead that may or may not be larger than the performance gain that you get from using multiple cores. Tuning IPC is therefore often crucial for multiprocessing in python.

there is already an answer on how to run playbooks on multiple hosts answered here Ansible: deploy on multiple hosts in the same time

Maybe you could start form there. However if running only first servers in parallel interests you than it will be more difficult, as it would require writing a custom script or something similar

As a first solution I'd recommend maximizing the distance between the image of the line at infinity and the center of your picture.

Identify at least two pairs of lines that are parallel in the original image. Intersect the lines of each pair and connect the resulting points. Best do all of this in homogeneous coordinates so you won't have to worry about lines being still parallel in the transformed version. Compute the distance between the center of the image and that line, possibly taking the resolution of the image into account somehow to make the result invariant to resampling. The result will be infinity for an image obtained from a pure affine transformation. So the larger that value the closer you are to the affine scenario.