bashdatacatalog | A simple portable file cataloging tool for bash

bashdatacatalog is a Shell library. bashdatacatalog has no bugs, it has no vulnerabilities and it has low support. You can download it from GitHub.

The bashdatacatalog is a command-line tool that facilitates synchronizing local data collections with a remote data source. With the bashdatacatalog, you can run queries on your local data collections to answer questions like "What files am I missing?" or "What files aren't bitwise identical to remote data?". Queries can include a date range, in which case collections with temporal assets are filtered-out accordingly. The bashdatacatalog can format the results of queries as: a URL download list, a Globus transfer list, an rsync transfer list, or simply a file list. The bashdatacatalog was written to facilitate downloading input data for users of the GEOS-Chem atmospheric chemistry model. The canonical GEOS-Chem input data repository has >1M files and >100TB of data, and the input data required for a simulation depends on the model version and simulation parameters such as start and end date. Note: Please consider giving the bashdatacatalog a "Star" (:star:) on GitHub. It helps increase visibility.