Cellecta uses the file transfer service Smash to transfer large NGS FASTQ files to and from our customers. In our experience, the transfer of huge files (those greater than 4GB) may result in incomplete or corrupt data. We recommend splitting huge files into more manageable smaller files of up to 4GB in order to avoid such issues.
Here are instructions for the recommended software 7-Zip running on Windows, Linux, or Linux command line for splitting huge archives into smaller parts.
Download here: 7-Zip
Below is a screenshot of the parameters we use to generate a multipart archive of FASTQ files in the folder specified.
To extract a file or files from the multipart archive, click on the .001 file of the archive you want to extract. Only this file needs to be selected–7-Zip will locate the rest. Then, click the Extract button and select the folder to which you would like to extract the files.
To create a multipart archive using CLI with multiple files of 4092 Mb each (less than 4GB[4096Mb]), use the command below. The filename for the archive you wish to create is “fastq.zip”, and the location of your FASTQ files is the directory called “fastq”.
7za -v4092m a fastq.zip fastq
To merge the files into the original archive, only the filename of the first archive (the one with the “.001” extension) needs to be entered–7-Zip will locate the rest.
7za x fastq.zip.001
Source: https://www.ericlin.me/2018/09/how-to-compress-and-extract-multipart-zip-files-on-linux/
