arrayqueues | Multiprocessing queues for numpy arrays using shared memory | Machine Learning library

arrayqueues is a Python library typically used in Artificial Intelligence, Machine Learning, Deep Learning, Pytorch, Numpy applications. arrayqueues has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has low support. You can install using 'pip install arrayqueues' or download it from GitHub, PyPI.

This package provides a drop-in replacement for the Python multiprocessing Queue class which handles transport of large numpy arrays. It avoids pickling and uses the multiprocessing Array class in the background. The major difference between this implementation and the normal queue is that the maximal amount of memory that the queue can have must be specified beforehand. Attempting to send an array of a different shape or datatype of the previously inserted one resets the queue. Only passing of numpy arrays is supported, optionally annotated with timestamps if using the TimestampedArrayQueue class, but other object types can be supported by extending the class. The package has been tested on Python 3.6/3/7 on Windows and MacOS and Linux with Travis. Python 2.7 is not supported.