cc-warc-examples | CommonCrawl WARC/WET/WAT examples

I am attempting to create a database of Digital Object Identifier (DOI) found on the internet.

By manually searching the CommonCrawl Index Server manually I have obtained some promising results.

However I wish to develop a programmatic solution.

This may result in my process only requiring to read the index files and not the underlying WARC data files.

The manual steps I wish to automate are these:-

1). for each CommonCrawl Currently available index collection(s):

2). I search ... "Search a url in this collection: (Wildcards -- Prefix: http://example.com/* Domain: *.example.com) " e.g. link.springer.com/*

3). this returns almost 6MB of json data that contains approx 22K unique DOIs.

How can I browse all available CommonCrawl indexes instead of searching for specific URLs?

From reading the API documentation for CommonCrawl I cannot see how I can browse all the indexes to extract all DOIs for all domains.

UPDATE

I found this example java code https://github.com/Smerity/cc-warc-examples/blob/master/src/org/commoncrawl/examples/S3ReaderTest.java

that shows how to access a common crawl dataset.

However when I run it I receive this exception