COSMA | Distributed Communication-Optimal Matrix | Machine Learning library

COSMA is a C++ library typically used in Artificial Intelligence, Machine Learning, Deep Learning applications. COSMA has no bugs, it has no vulnerabilities, it has a Permissive License and it has low support. You can download it from GitHub.

COSMA is a parallel, high-performance, GPU-accelerated, matrix-matrix multiplication algorithm that is communication-optimal for all combinations of matrix dimensions, number of processors and memory sizes, without the need for any parameter tuning. The key idea behind COSMA is to first derive a tight optimal sequential schedule and only then parallelize it, preserving I/O optimality between processes. This stands in contrast with the 2D and 3D algorithms, which fix process domain decomposition upfront and then map it to the matrix dimensions, which may result in asymptotically more communication. The final design of COSMA facilitates the overlap of computation and communication, ensuring speedups and applicability of modern mechanisms such as RDMA. COSMA allows to not utilize some processors in order to optimize the processor grid, which reduces the communication volume even further and increases the computation volume per processor. COSMA got the Best Student Paper Award at the prestigious Supercomputing 2019 conference in Denver, US.