Flotilla | Automated message queue orchestration for scaled-up

Flotilla is a Go library. Flotilla has no bugs, it has no vulnerabilities, it has a Permissive License and it has low support. You can download it from GitHub.

Flotilla is a work-in-progress tool for testing message queues in more realistic environments. Many benchmarks only measure performance characteristics on a single machine, sometimes with producers and consumers in the same process even. The reality is this information is marginally useful, if at all, and often deceiving. This blog post provides some more background on the motivation behind this project. Testing anything at scale can be difficult to achieve in practice. It generally takes a lot of resources and often requires ad hoc solutions. Flotilla attempts to provide automated orchestration for benchmarking message queues in scaled-up configurations. Simply put, we can benchmark a message broker with arbitrarily many producers and consumers distributed across arbitrarily many machines with a single command. In addition to simulating more realistic testing scenarios, Flotilla also tries to offer more statistically meaningful results in the benchmarking itself. It relies on HDR Histogram (or rather a Go variant of it) which supports recording and analyzing sampled data value counts at extremely low latencies. See the "Caveats" section below for potential benchmarking issues and areas for improvement.