kandi X-RAY | IRIS-sample Summary
kandi X-RAY | IRIS-sample Summary
Test
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Run SVC .
IRIS-sample Key Features
IRIS-sample Examples and Code Snippets
Community Discussions
Trending Discussions on IRIS-sample
QUESTION
I would like to speed up the execution of a function my_func()
using parallel computation with dask.delayed
.
In a loop over 3 dimensions, my_func()
extracts a value from an iris.cube.Cube
(which is essentially a dask.array
loaded from a file outside the loop), and depending on the value, creates a random network using networkx
and finds the shortest path from node 0 to node 16. Calculation for each array point is independent.
- Why executing parallel code takes longer (5.43 s) than serial (2.94 s)?
- Is there a better way to speed it up with
dask
ormultiprocessing
or something else?
Here is a reproducible example:
...ANSWER
Answered 2021-Mar-04 at 15:50dask
will have some overhead, so on small samples it's not unusual for it to underperform. When I try increasing the number of computations by changing to for iy in lats[0:15]:
, I see that serial calculation takes 10 seconds, while dask completes it in 4 seconds.
(there is also serialization of the function, which might take some time, but it applies only to the first time the function is sent to workers)
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install IRIS-sample
You can use IRIS-sample like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page