fq | jq for binary formats - tool, language and decoders for working with binary and text formats | JSON Processing library

I have received multiple fastq.gz files from Illumina Sequencing for 100 samples. But all the fastq.gz files for the respective samples are in separate folders according to the sample ID. Moreover, I have multiple (8-16) R1.fastq.gz and R2.fastq.gz files for one sample. So, I used the following code for concatenating all the R1.fastq.gz and R2.fastq.gz into a single R1.fastq.gz and R2.fastq.gz.

New here!

I'm trying to loop a python script in bash that gets stats from fastq files. I want it to loop through all the fastq files in a directory and save the outputs in a text file. Ideally don't want to edit the python script

This is the script that works when I'm not looping it:

I have a tuple channel containing entries like:

I am trying to create a robust glob pattern that will match most of the different naming conventions used for fastq files we receive. However, the version of nextflow I am using (20.10.0) on the HPC doesn't seem to accept what I've written.

Here are some examples of file names:

I've been trying to run a while-loop in parallel for a work that takes days.

I've seen other answers where parallel was implemented within a while-loop, but for that case it does work in blocks, where the next job only works after all previous jobs finished.

This is the code, which reproduced the two columns of the CSV file:

Hello I need to iterate over pairs of files and do something with them.

For example I have 4 files which are named AA2234_1.fastq.gz AA2234_2.fastq.gz AA3945_1.fastq.gz AA3945_2.fastq.gz

As you can propably tell the pairs are AA2234_1.fastq.gz <-> AA2234_2.fastq.gz and AA3945_1.fastq.gz <-> AA3945_2.fastq.gz (they share the name before _ sign)

I have a command with syntax looking like this:

initialize_of_command file1 file2 output_a output_b output_c output_d parameteres

I want this script to find the number of files with fastq.gz extension in a directory, divide them by 2 to find number of pairs then match the pairs together using probably regex (maybe to two variables) and execute this command for each pair once.

I have no idea how to pair up those files using regex and how to iterate over the pairs so the scripts knows through which pairs it have already iterated.

Here is my unfinished script:

I have a list of file names in a file called data.tsv. Each row corresponds to the same sample ID and there is a potential of up to 8 files per ID. I need to merge files that end in "_1.[variable extension]" and "_2.[variable extension].

Data below, however stackover flow coverts tabs to spaces - should be tab delimited:

This is a follow-up of a previous question about using a Python dictionary to generate a list of files to include as input for a single step. In this case, I'm interested in merging BAM files for a single sample that have been generated by mapping FASTQ files from multiple runs.

I am running into an error in my rule combine_bams only for a single sample:

I'm writing a Snakemake pipeline for scRNAseq sequence processing which uses STAR as the alignment tool.

Loading genome index into the memory for each alignment job is very time-consuming. Since I have a lot of memory at my disposal, I figure that it is feasible to use the "shared memory" module from STAR. With it I can load the genome into the memory and keep it there until all the jobs are done.

With shell this is very easy to achieve, for example here.

But chianing it in a Snakemake pipeline seems a non-trivial task, especially that the STAR "shared memory" module doesn't require any input or output anything. For example, I tried: