Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-3450

Run nf-core/rnaseq pipeline using SL4 target genome assembly

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Goal:

      Create BAM, coverage graph, and junction files for the Muday Lab time course data using SL4 genome and gene annotations as the target for alignment.

      When complete, please note the location of files created.

      Note: Don't use the renamed files. Use the original input files prior to their being renamed.

        Attachments

          Activity

          Hide
          Mdavis4290 Molly Davis added a comment - - edited

          Directory of original files that are NOT renamed: /projects/tomato_genome/rnaseq/muday144-timeSeries-checkReadMEFIRST/00_fastq

          SL4 Nf-core run location: /projects/tomato_genome/fnb/dataprocessing/muday-144/nfcore-SL4

          Show
          Mdavis4290 Molly Davis added a comment - - edited Directory of original files that are NOT renamed : /projects/tomato_genome/rnaseq/muday144-timeSeries-checkReadMEFIRST/00_fastq SL4 Nf-core run location : /projects/tomato_genome/fnb/dataprocessing/muday-144/nfcore-SL4
          Hide
          Mdavis4290 Molly Davis added a comment - - edited

          Nextflow Pipeline ran successfully with SL4 genome
          Directory: /projects/tomato_genome/fnb/dataprocessing/muday-144/nfcore-SL4
          MultiQC report notes: No errors or warnings were present in the report. The output file is named 'muday-144-SL4_multiqc_report.html'.

          Next Steps: Commit Multiqc report to documentation in flavonoid repo. Create Coverage graphs and junction files.

          Show
          Mdavis4290 Molly Davis added a comment - - edited Nextflow Pipeline ran successfully with SL4 genome Directory : /projects/tomato_genome/fnb/dataprocessing/muday-144/nfcore-SL4 MultiQC report notes : No errors or warnings were present in the report. The output file is named 'muday-144-SL4_multiqc_report.html'. Next Steps : Commit Multiqc report to documentation in flavonoid repo. Create Coverage graphs and junction files.
          Hide
          Mdavis4290 Molly Davis added a comment - - edited

          Launch renameBams.sh script:
          ./renameBams.sh
          Launch Scaled Coverage graphs script:
          ./sbatch-doIt.sh .bam bamCoverage.sh >jobs.out 2>jobs.err
          Launch Junction files script:
          ./sbatch-doIt.sh .bam find_junctions.sh >jobs.out 2>jobs.err

          Reviewer:
          Check that files have reasonable sizes (no "zero" size files, for example)
          Check that every "FJ.bed.gz" file has a corresponding "FJ.bed.gz.tbi" index file
          Check that every bam file has a corresponding "FJ.bed.gz" file
          Check that every bam file has a corresponding "scaled.bedgraph.gz" file
          Check that every "scaled.bedgraph.gz" has a corresponding "scaled.bedgraph.gz.tbi"

          Reviewer: [~RobertReid]

          Note: There were two warnings but no errors that I saw.

          Show
          Mdavis4290 Molly Davis added a comment - - edited Launch renameBams.sh script : ./renameBams.sh Launch Scaled Coverage graphs script : ./sbatch-doIt.sh .bam bamCoverage.sh >jobs.out 2>jobs.err Launch Junction files script : ./sbatch-doIt.sh .bam find_junctions.sh >jobs.out 2>jobs.err Reviewer : Check that files have reasonable sizes (no "zero" size files, for example) Check that every "FJ.bed.gz" file has a corresponding "FJ.bed.gz.tbi" index file Check that every bam file has a corresponding "FJ.bed.gz" file Check that every bam file has a corresponding "scaled.bedgraph.gz" file Check that every "scaled.bedgraph.gz" has a corresponding "scaled.bedgraph.gz.tbi" Reviewer: [~RobertReid] Note: There were two warnings but no errors that I saw.
          Hide
          Mdavis4290 Molly Davis added a comment - - edited

          Branch: https://bitbucket.org/mdavis4290/molly3-flavonoid-rnaseq/branch/IGBF-3450

          Note: Includes the SL4 Mutliqc report.

          [~aloraine]

          Show
          Mdavis4290 Molly Davis added a comment - - edited Branch : https://bitbucket.org/mdavis4290/molly3-flavonoid-rnaseq/branch/IGBF-3450 Note: Includes the SL4 Mutliqc report. [~aloraine]
          Hide
          ann.loraine Ann Loraine added a comment - - edited

          Change requests:

          1) "chmod" the entire contents of nfcore-SL4 to make them group editable

          This is so that I can move the bam, bedgraph, and junction files over to another directory for copying to the data host. I don't want to make copies due to the risk of error and also to save disk space and energy.

          2) Add this file to your branch:

          • /projects/tomato_genome/fnb/dataprocessing/muday-144/nfcoreSL4/results/star_salmon/salmon.merged.gene_counts.tsv

          as file:

          • 72_F3H_PollenTube/muday-144-SL4_salmon.merged.gene_counts.tsv

          Please add it to your branch and then submit a PR.

          Thank you!

          Ann

          Show
          ann.loraine Ann Loraine added a comment - - edited Change requests: 1) "chmod" the entire contents of nfcore-SL4 to make them group editable This is so that I can move the bam, bedgraph, and junction files over to another directory for copying to the data host. I don't want to make copies due to the risk of error and also to save disk space and energy. 2) Add this file to your branch: /projects/tomato_genome/fnb/dataprocessing/muday-144/nfcoreSL4/results/star_salmon/salmon.merged.gene_counts.tsv as file: 72_F3H_PollenTube/muday-144-SL4_salmon.merged.gene_counts.tsv Please add it to your branch and then submit a PR. Thank you! Ann
          Hide
          ann.loraine Ann Loraine added a comment - - edited

          Double-checking:

          The renamed fastq files were NOT used here. Is this right? That is, the original files with their original names were processed, the same exact files that we got from the sequencer?

          Show
          ann.loraine Ann Loraine added a comment - - edited Double-checking: The renamed fastq files were NOT used here. Is this right? That is, the original files with their original names were processed, the same exact files that we got from the sequencer?
          Hide
          Mdavis4290 Molly Davis added a comment - - edited

          These are the original files that are NOT renamed. Straight from the sequencer!
          This can be confirmed in the README in the original directory: /projects/tomato_genome/rnaseq/muday144-timeSeries-checkReadMEFIRST/00_fastq

          Results should also now be editable.
          [~aloraine]

          Show
          Mdavis4290 Molly Davis added a comment - - edited These are the original files that are NOT renamed. Straight from the sequencer! This can be confirmed in the README in the original directory: /projects/tomato_genome/rnaseq/muday144-timeSeries-checkReadMEFIRST/00_fastq Results should also now be editable. [~aloraine]
          Show
          Mdavis4290 Molly Davis added a comment - Pull Request : https://bitbucket.org/hotpollen/flavonoid-rnaseq/pull-requests/24
          Hide
          ann.loraine Ann Loraine added a comment -

          Testing suggestions:

          • Compare multiqc reports from SL4, SL5 assemblies. PCA plots should mostly match.
          Show
          ann.loraine Ann Loraine added a comment - Testing suggestions: Compare multiqc reports from SL4, SL5 assemblies. PCA plots should mostly match.
          Hide
          ann.loraine Ann Loraine added a comment -

          [~molly]: please try again with the "chmod-ing"

          I need to move these files into position for transferring to data.bioviz.org, like so:

          [aloraine@str-i1 muday-2022-timeseries]$ pwd
          /projects/tomato_genome/fnb/dataprocessing/muday-144/for_quickload/S_lycopersicum_Sep_2019/muday-2022-timeseries
          [aloraine@str-i1 muday-2022-timeseries]$ mv ../../../nfcore-SL4/results/star_salmon/*.bam .
          mv: cannot move '../../../nfcore-SL4/results/star_salmon/7-are-15-min-28C.bam' to './7-are-15-min-28C.bam': Permission denied
          

          I need to move all the bam, bam index, coverage graph, coverage graph index, junction, junction index files over to /projects/tomato_genome/fnb/dataprocessing/muday-144/for_quickload/S_lycopersicum_Sep_2019/muday-2022-timeseries but can't because of how the permissions are set.

          I would like to take some pictures of genes before tomorrow if at all possible! But I don't want to waste a lot of disk space and energy to meet a deadline

          Show
          ann.loraine Ann Loraine added a comment - [~molly] : please try again with the "chmod-ing" I need to move these files into position for transferring to data.bioviz.org, like so: [aloraine@str-i1 muday-2022-timeseries]$ pwd /projects/tomato_genome/fnb/dataprocessing/muday-144/for_quickload/S_lycopersicum_Sep_2019/muday-2022-timeseries [aloraine@str-i1 muday-2022-timeseries]$ mv ../../../nfcore-SL4/results/star_salmon/*.bam . mv: cannot move '../../../nfcore-SL4/results/star_salmon/7-are-15-min-28C.bam' to './7-are-15-min-28C.bam': Permission denied I need to move all the bam, bam index, coverage graph, coverage graph index, junction, junction index files over to /projects/tomato_genome/fnb/dataprocessing/muday-144/for_quickload/S_lycopersicum_Sep_2019/muday-2022-timeseries but can't because of how the permissions are set. I would like to take some pictures of genes before tomorrow if at all possible! But I don't want to waste a lot of disk space and energy to meet a deadline
          Hide
          Mdavis4290 Molly Davis added a comment - - edited

          chmod g+w /projects/tomato_genome/fnb/dataprocessing/muday-144/nfcore-SL4/results
          chmod -R g+w *

          Show
          Mdavis4290 Molly Davis added a comment - - edited chmod g+w /projects/tomato_genome/fnb/dataprocessing/muday-144/nfcore-SL4/results chmod -R g+w *
          Hide
          ann.loraine Ann Loraine added a comment -

          Moved or copied nf-core output files to HPC file system location: /projects/tomato_genome/fnb/dataprocessing/muday-144/for_quickload/S_lycopersicum_Sep_2019/muday-2022-timeseries

          On data.bioviz.org host, launched "screen" session and commenced copying nf-core output files to data.bioviz.org with:

          cd /mnt/igbdata/hotpollen/S_lycopersicum_Sep_2019
          rsync -rz aloraine@10.16.115.244:/projects/tomato_genome/fnb/dataprocessing/muday-144/for_quickload/S_lycopersicum_Sep_2019/muday-2022-timeseries .
          
          Show
          ann.loraine Ann Loraine added a comment - Moved or copied nf-core output files to HPC file system location: /projects/tomato_genome/fnb/dataprocessing/muday-144/for_quickload/S_lycopersicum_Sep_2019/muday-2022-timeseries On data.bioviz.org host, launched "screen" session and commenced copying nf-core output files to data.bioviz.org with: cd /mnt/igbdata/hotpollen/S_lycopersicum_Sep_2019 rsync -rz aloraine@10.16.115.244:/projects/tomato_genome/fnb/dataprocessing/muday-144/for_quickload/S_lycopersicum_Sep_2019/muday-2022-timeseries .
          Hide
          ann.loraine Ann Loraine added a comment -

          Added new SL4 data to the annots.xml configuration files for visualization in IGB. Updated "main" branch of https://bitbucket.org/hotpollen/genome-browser-visualization/src/main/

          Show
          ann.loraine Ann Loraine added a comment - Added new SL4 data to the annots.xml configuration files for visualization in IGB. Updated "main" branch of https://bitbucket.org/hotpollen/genome-browser-visualization/src/main/
          Hide
          ann.loraine Ann Loraine added a comment -

          Review passes:

          • counts file is added to the repo
          • data files are added to data.bioviz.org for visualization and permissions set to allow delivery via the Web server
          • also added multiqc report and its input data to the data.bioviz.org deployment site for easier visualization

          Note that these data are NOT using renamed labels. The files that were processed match the original file names from the sequencer.

          Moving to ready for testing.

          Show
          ann.loraine Ann Loraine added a comment - Review passes: counts file is added to the repo data files are added to data.bioviz.org for visualization and permissions set to allow delivery via the Web server also added multiqc report and its input data to the data.bioviz.org deployment site for easier visualization Note that these data are NOT using renamed labels. The files that were processed match the original file names from the sequencer. Moving to ready for testing.
          Hide
          Mdavis4290 Molly Davis added a comment - - edited

          Testing:

          Moving to done!

          Show
          Mdavis4290 Molly Davis added a comment - - edited Testing : Count file is present in repo: 72_F3H_PollenTube/muday-144-SL4_salmon.merged.gene_counts.tsv SL5 (PT time GM lab) and SL4 (PT time GM lab) datasets for Muday-144 are present and accessible in IGB quickload Found report and input data on deployment site here: https://data.bioviz.org/hotpollen/S_lycopersicum_Sep_2019/muday-2022-timeseries/ Moving to done!

            People

            • Assignee:
              Mdavis4290 Molly Davis
              Reporter:
              ann.loraine Ann Loraine
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: