pigz | parallel implementation of gzip for modern multi

Hello Stackoverflowers!
I am trying to build the OpenJDK with OpenJ9 on Linux, but when I run the configure script, I get the error:

I'm trying to figure out why is the following tar command not working -

I've tried the following 2 versions & both don't work -

Version 1

I just can't understand what is the mistake with the following tar command that complains tar: Cowardly refusing to create an empty archive

I have the below script doing nightly backups of my databases.

If I execute directly via shell, everything works.

However, if via cron as same user execution, I get this error in the log file: nightly-backups.sh: 9: [[: not found

I installed Docker on my Unix machine and it was not working properly, so I tried to uninstall it to run through the steps again. I followed the uninstall steps here https://docs.docker.com/engine/install/ubuntu/#supported-storage-drivers

But I am receiving this error

here is my script:

tar cf - testdir | pv -s $(du -sb testdir | awk '{print $1}') | pigz -1 > pv.tar.gz

tar cf - testdir | pigz -1 > nopv.tar.gz

diff pv.tar.gz nopv.tar.gz

and then the output is "Binary files pv.tar.gz and nopv.tar.gz differ".

I execute hexdump

and I found that only the first line of these two files is slightly different

pv.tar.gz: 8b1f 0008 9e24 5fc8 0304 bdec 5f7b c71b

nopv.tar.gz: 8b1f 0008 9c18 5fc8 0304 bdec 5f7b c71b

But after I unzipped it and compared it again, the testdir is exactly the same.

What I want to ask is, how can I make the two tar.gz files consistent?

TL;DR: Is there a python library that allows parallel writes into a single gzip file from multiple processes?

Details:

I am trying to copy a large, compressed file (.gz) to another compressed file (.gz) using python. I will perform intermediate processing on the data that is not present in my code sample. I would like to be able to use multiprocessing with locks to write to the new gzip in parallel from multiple processes, but I get an invalid format error on the output gz file.

I assume that this is because a lock is not enough to support writing to a gzip in parallel. Since compressed data requires "knowledge" of the data that came before it in order to make correct entries into the archive I don't think that python can't handle this by default. I'd guess that each process maintains its own awareness of the gzip output and that this state diverges after the first write.

If I open the target file in the script without using gzip then this all works. I could also write to multiple gzips and merge them, but prefer to avoid that if possible.

Here is my source code:

I am trying to test a script I have developed locally on an interactive HPC node, and I keep running in this strange issue that mclapply works only on a single core. I see several R processes spawned in htop (as many as the number of the cores), but they all occupy only one core.

Here is how I obtain the interactive node:

My kernal version: