benchmark-mirror | repo holds the benchmark tasks

benchmark-mirror is a Python library. benchmark-mirror has no bugs, it has no vulnerabilities and it has low support. However benchmark-mirror build file is not available. You can download it from GitHub.

This repo holds the benchmark tasks that are executed regularly and also the driver script. It is designed to be simple and flexible. Basically it is a thin wrapper on top of your benchmark script which provides basic utilities for managing tasks, profiling gpu and cpu memories, and extract metrics from your logging. To use the benchmark driver, you just need to look at the task_config_template.cfg to see what benchmark tasks are supported and use the command python benchmark_driver.py --task-name [TASK_NAME] --num-gpus [NUM_GPUS] then the driver will trigger a task and write the benchmark result to a JSON file called "dlbenchmark_result.json". e.g. There are also optional flags such as --framework, and --metrics-suffix, which are used for decorating the metrics names.