Thread-Queue | The 'Thread-Queue ' module for Perl
kandi X-RAY | Thread-Queue Summary
kandi X-RAY | Thread-Queue Summary
The 'Thread-Queue' module for Perl.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of Thread-Queue
Thread-Queue Key Features
Thread-Queue Examples and Code Snippets
Community Discussions
Trending Discussions on Thread-Queue
QUESTION
In Python 3.8, concurrent.futures.ProcessPoolExecutor
has been updated to limit the max number of workers (processes) able to be used on Windows to 61. For the reasons why, see this and this, but to my understanding:
- On Windows,
multiprocessing
calls the Windows API functionWaitForMultipleObjects
, which is used to wait for processes to finish. It can wait on, at most, 63 objects, less the result queue reader and thread wakeup reader, hence the 61 limit. (i.e. Windows uses a thread per process to track processes).
(see also this SO issue)
multiprocessing
, however, still uses os.cpu_count()
. It throws a Value Error
at first, but then continues on and uses 100% of my CPU cores. For example,
ANSWER
Answered 2021-Apr-24 at 18:52Yours is an excellent question. Looking at the code it would appear that this would be an unrecoverable error. But it seems to me incomprehensible that there would be code in the ThreadPoolExecutor
to limit the pool size to 61 under Windows and not enforce that for the the multiprocessing.Pool
class. Anyway, it should be easy enough to check with the following program. If it does not print Done! and hangs, I would say there is definitely a problem and you should explicitly limit the pool size if you are using multiprocessing.Pool
:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install Thread-Queue
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page