Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Create and run a script (fasterq-dump-rnaseq.sh) that retrieves RNA-Seq data files from the SRA and converts them to fastq format.

      See linked issue for a comment containing a first draft of the script code: IGBF-3040

      Put script under version-control in:

      • salty_rice/bseq_rice/src

        Attachments

          Issue Links

            Activity

            Hide
            ann.loraine Ann Loraine added a comment -

            This script will need to be run on the cluster to retrieve the data we want from the SRA and deploy onto the cluster where we do the next data processing steps.

            However, there may be a way to do a "dry run" to check that the script code is correct. If possible, try to do that.

            Show
            ann.loraine Ann Loraine added a comment - This script will need to be run on the cluster to retrieve the data we want from the SRA and deploy onto the cluster where we do the next data processing steps. However, there may be a way to do a "dry run" to check that the script code is correct. If possible, try to do that.
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Downloading and converting the files on the cluster now.

            Show
            ann.loraine Ann Loraine added a comment - - edited Downloading and converting the files on the cluster now.
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Sample output – not sure what "reads 0-length" means:

            concat :|-------------------------------------------------- 100%
            spots read      : 38,726,481
            reads read      : 77,452,962
            reads written   : 62,196,161
            reads 0-length  : 15,256,801
            join   :|-------------------------------------------------- 100%
            concat :|-------------------------------------------------- 100%
            spots read      : 47,593,574
            reads read      : 95,187,148
            reads written   : 76,499,566
            reads 0-length  : 18,687,582
            join   :|-------------------------------------------------- 100%
            concat :|-------------------------------------------------- 100%
            spots read      : 56,233,927
            reads read      : 112,467,854
            reads written   : 56,233,927
            reads 0-length  : 56,233,927
            join   :|-------------------------------------------------- 100%
            concat :|-------------------------------------------------- 100%
            spots read      : 52,074,128
            reads read      : 104,148,256
            reads written   : 52,074,128
            reads 0-length  : 52,074,128
            join   :|-------------------------------------------------- 100%
            concat :|-------------------------------------------------- 100%
            spots read      : 39,584,294
            reads read      : 79,168,588
            reads written   : 39,584,294
            reads 0-length  : 39,584,294
            join   :|-------------------------------------------------- 100%
            concat :|-------------------------------------------------- 100%
            spots read      : 57,668,415
            reads read      : 115,336,830
            reads written   : 89,783,065
            reads 0-length  : 25,553,765
            join   :|-------------------------------------------------- 100%
            concat :|-------------------------------------------------- 100%
            spots read      : 41,998,084
            reads read      : 83,996,168
            reads written   : 83,996,168
            join   :|-------------------------------------------------- 100%
            concat :|-------------------------------------------------- 100%
            spots read      : 45,085,096
            reads read      : 90,170,192
            reads written   : 45,085,096
            reads 0-length  : 45,085,096
            
            Show
            ann.loraine Ann Loraine added a comment - - edited Sample output – not sure what "reads 0-length" means: concat :|-------------------------------------------------- 100% spots read : 38,726,481 reads read : 77,452,962 reads written : 62,196,161 reads 0-length : 15,256,801 join :|-------------------------------------------------- 100% concat :|-------------------------------------------------- 100% spots read : 47,593,574 reads read : 95,187,148 reads written : 76,499,566 reads 0-length : 18,687,582 join :|-------------------------------------------------- 100% concat :|-------------------------------------------------- 100% spots read : 56,233,927 reads read : 112,467,854 reads written : 56,233,927 reads 0-length : 56,233,927 join :|-------------------------------------------------- 100% concat :|-------------------------------------------------- 100% spots read : 52,074,128 reads read : 104,148,256 reads written : 52,074,128 reads 0-length : 52,074,128 join :|-------------------------------------------------- 100% concat :|-------------------------------------------------- 100% spots read : 39,584,294 reads read : 79,168,588 reads written : 39,584,294 reads 0-length : 39,584,294 join :|-------------------------------------------------- 100% concat :|-------------------------------------------------- 100% spots read : 57,668,415 reads read : 115,336,830 reads written : 89,783,065 reads 0-length : 25,553,765 join :|-------------------------------------------------- 100% concat :|-------------------------------------------------- 100% spots read : 41,998,084 reads read : 83,996,168 reads written : 83,996,168 join :|-------------------------------------------------- 100% concat :|-------------------------------------------------- 100% spots read : 45,085,096 reads read : 90,170,192 reads written : 45,085,096 reads 0-length : 45,085,096
            Hide
            ann.loraine Ann Loraine added a comment -

            Script added to repository: rnaseq-fasterq-dump.sh.

            Show
            ann.loraine Ann Loraine added a comment - Script added to repository: rnaseq-fasterq-dump.sh.
            Hide
            ann.loraine Ann Loraine added a comment -

            Adding script to compress all the files using cluster nodes.

            Show
            ann.loraine Ann Loraine added a comment - Adding script to compress all the files using cluster nodes.
            Hide
            ann.loraine Ann Loraine added a comment -

            Compressed fastq files are here: /nobackup/lorainelab/salty_rice/rna-seq

            Show
            ann.loraine Ann Loraine added a comment - Compressed fastq files are here: /nobackup/lorainelab/salty_rice/rna-seq
            Hide
            ann.loraine Ann Loraine added a comment -
            Show
            ann.loraine Ann Loraine added a comment - Total size of all gzip'd files: 131 Gb Moving to Closed. Code added: https://bitbucket.org/lorainelab/bseq_rice/src/master/src/sbatch-doIt.sh https://bitbucket.org/lorainelab/bseq_rice/src/master/src/gzip.sh

              People

              • Assignee:
                ann.loraine Ann Loraine
                Reporter:
                ann.loraine Ann Loraine
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: