[IGBF-3242] Run nextflow for seedlingPollen dataset - JIRA UNCC

Ann Loraine created issue - 17/Jan/23 1:15 PM

Ann Loraine made changes - 17/Jan/23 1:15 PM

Field	Original Value	New Value
Epic Link		IGBF-2993 [ 21429 ]

Ann Loraine made changes - 17/Jan/23 1:17 PM

Description

* Get data from google drive onto the cluster
* Migrate data from cluster to an igbquickload host

* Get data from google drive onto the cluster
* Migrate data from cluster to an igbquickload host

See: https://drive.google.com/drive/folders/1eHn-uK6N5PEr3fr8kfWHrySrarx5XOYU?usp=share_link

Ann Loraine made changes - 17/Jan/23 1:19 PM

Description

* Get data from google drive onto the cluster
* Migrate data from cluster to an igbquickload host

See: https://drive.google.com/drive/folders/1eHn-uK6N5PEr3fr8kfWHrySrarx5XOYU?usp=share_link

* Get data from google drive onto the cluster
* Migrate data from cluster to an igbquickload host

See: https://drive.google.com/drive/folders/1GJnZefP-7TE-ch-c0lblGZMOqpwSgZRK?usp=share_link

Ann Loraine made changes - 18/Jan/23 10:18 AM

Summary

Deploy new Dec 2022 data onto an igbquickload site

Run nextflow for mark-2022-timeseries

Ann Loraine made changes - 18/Jan/23 10:18 AM

Description

* Get data from google drive onto the cluster
* Migrate data from cluster to an igbquickload host

See: https://drive.google.com/drive/folders/1GJnZefP-7TE-ch-c0lblGZMOqpwSgZRK?usp=share_link

Run nextflow nfcore rnaseq pipeline on fastq files in:

/projects/tomato_genome/rnaseq/mark-2022-timeseries/30-771363348/00_fastq

Ann Loraine made changes - 18/Jan/23 10:39 AM

Description

Run nextflow nfcore rnaseq pipeline on fastq files in:

/projects/tomato_genome/rnaseq/mark-2022-timeseries/30-771363348/00_fastq

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness

Ann Loraine made changes - 18/Jan/23 10:39 AM

Summary

Run nextflow for mark-2022-timeseries

Run nextflow for seedlingPollen dataset

Ann Loraine made changes - 18/Jan/23 10:42 AM

Description

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness field.

Please just edit "samples.csv" in /nobackup/tomato_genome/seedlingPollen/nfcore.

Note about sample names:

MP - mature pollen
Se - seedling
N - nagcarlang
H - heinz
T - tamalipas

S - heat-treated, 34 degrees
C - non-heated treated, raised at 28 degrees

Ann Loraine made changes - 18/Jan/23 10:45 AM

Description

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness field.

Please just edit "samples.csv" in /nobackup/tomato_genome/seedlingPollen/nfcore.

Note about sample names:

MP - mature pollen
Se - seedling
N - nagcarlang
H - heinz
T - tamalipas

S - heat-treated, 34 degrees
C - non-heated treated, raised at 28 degrees

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness field.

Please just edit "samples.csv" in /nobackup/tomato_genome/seedlingPollen/nfcore.

Note about sample names:

MP - mature pollen
Se - seedling
N - nagcarlang
H - heinz
T - tamalipas
M - malintka
S - heat-treated, 37 degrees
C - non-heated treated, raised at 28 degrees

See "experiment" folder in the google drive

Ann Loraine made changes - 18/Jan/23 10:47 AM

Description

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness field.

Please just edit "samples.csv" in /nobackup/tomato_genome/seedlingPollen/nfcore.

Note about sample names:

MP - mature pollen
Se - seedling
N - nagcarlang
H - heinz
T - tamalipas
M - malintka
S - heat-treated, 37 degrees
C - non-heated treated, raised at 28 degrees

See "experiment" folder in the google drive

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness field.

Please just edit "samples.csv" in /nobackup/tomato_genome/seedlingPollen/nfcore.

Note about sample names:

MP - mature pollen
Se - seedling
N - nagcarlang
H - heinz
T - tamalipas
M - malintka
S - heat-treated, 37 degrees
C - non-heated treated, raised at 28 degrees

See "experiment" folder in the google drive with info about the experiment:

Experiments > Pollen Grain and Seedling RNA-seq > SRA Description - pollen grain and seedling experiments

Ann Loraine made changes - 18/Jan/23 10:47 AM

Description

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness field.

Please just edit "samples.csv" in /nobackup/tomato_genome/seedlingPollen/nfcore.

Note about sample names:

MP - mature pollen
Se - seedling
N - nagcarlang
H - heinz
T - tamalipas
M - malintka
S - heat-treated, 37 degrees
C - non-heated treated, raised at 28 degrees

See "experiment" folder in the google drive with info about the experiment:

Experiments > Pollen Grain and Seedling RNA-seq > SRA Description - pollen grain and seedling experiments

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness field.

Please just edit "samples.csv" in /nobackup/tomato_genome/seedlingPollen/nfcore.

Note about sample names:

MP - mature pollen
Se - seedling
N - nagcarlang
H - heinz
T - tamaulipas
M - malintka
S - heat-treated, 37 degrees
C - non-heated treated, raised at 28 degrees

See "experiment" folder in the google drive with info about the experiment:

Experiments > Pollen Grain and Seedling RNA-seq > SRA Description - pollen grain and seedling experiments

Ann Loraine made changes - 18/Jan/23 10:48 AM

Story Points

1

0.5

Molly Davis made changes - 18/Jan/23 11:03 AM

Status

To-Do [ 10305 ]

In Progress [ 3 ]

Molly Davis made changes - 18/Jan/23 11:03 AM

Assignee

Molly Davis [ molly ]

Hide

Permalink

Molly Davis added a comment - 18/Jan/23 11:41 AM - edited

Update: I fixed the sample sheet so that strandedness is reverse. When running nextflow I come across this error for trimgalore:

Command error:
  INFO:    Converting SIF file to temporary sandbox...
  Path to Cutadapt set as: 'cutadapt' (default)
  Cutadapt seems to be working fine (tested command 'cutadapt --version')
  Cutadapt version: 3.4
  Could not detect version of Python used by Cutadapt from the first line of Cutadapt (but found this: >>>#!/bin/sh<<<)
  Letting the (modified) Cutadapt deal with the Python version instead
  Parallel gzip (pigz) detected. Proceeding with multicore (de)compression using 4 cores
  
  No quality encoding type selected. Assuming that the data provided uses Sanger encoded Phred scores (default)
  
  Input file 'MP-MC-1_1.fastq.gz' seems to be completely empty. Consider respecifying!
  
  INFO:    Cleaning up image...

Work dir:
  /nobackup/tomato_genome/seedlingPollen/nfcore/work/15/4c200c1d0331cd19fc66b69eaad96b

Question: are all fastq files double stranded or single stranded? Or is there a naming issue in the sample csv file?

Next step: Instead of using symbolic links I will copy the fastq files to the following directory to see if it works with nextflow. I have had issues with symbolic links before with fastq files and copying the files instead seemed to fix the issue.
Copy From: /projects/tomato_genome/rnaseq/mark-2022-pollengrainSeedling/00_fastq/
Copy To: /nobackup/tomato_genome/seedlingPollen/nfcore

Issue: Do not have permission to copy fastq files.

Fixed Issue: Dr. Robert Reid gave permission so I could copy files over.

Show

Molly Davis added a comment - 18/Jan/23 11:41 AM - edited Update: I fixed the sample sheet so that strandedness is reverse. When running nextflow I come across this error for trimgalore: Command error: INFO: Converting SIF file to temporary sandbox... Path to Cutadapt set as: 'cutadapt' ( default ) Cutadapt seems to be working fine (tested command 'cutadapt --version') Cutadapt version: 3.4 Could not detect version of Python used by Cutadapt from the first line of Cutadapt (but found this : >>>#!/bin/sh<<<) Letting the (modified) Cutadapt deal with the Python version instead Parallel gzip (pigz) detected. Proceeding with multicore (de)compression using 4 cores No quality encoding type selected. Assuming that the data provided uses Sanger encoded Phred scores ( default ) Input file 'MP-MC-1_1.fastq.gz' seems to be completely empty. Consider respecifying! INFO: Cleaning up image... Work dir: /nobackup/tomato_genome/seedlingPollen/nfcore/work/15/4c200c1d0331cd19fc66b69eaad96b Question: are all fastq files double stranded or single stranded? Or is there a naming issue in the sample csv file? Next step : Instead of using symbolic links I will copy the fastq files to the following directory to see if it works with nextflow. I have had issues with symbolic links before with fastq files and copying the files instead seemed to fix the issue. Copy From : /projects/tomato_genome/rnaseq/mark-2022-pollengrainSeedling/00_fastq/ Copy To : /nobackup/tomato_genome/seedlingPollen/nfcore Issue: Do not have permission to copy fastq files. Fixed Issue: Dr. Robert Reid gave permission so I could copy files over.

Ann Loraine made changes - 18/Jan/23 11:58 AM

Description

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness field.

Please just edit "samples.csv" in /nobackup/tomato_genome/seedlingPollen/nfcore.

Note about sample names:

MP - mature pollen
Se - seedling
N - nagcarlang
H - heinz
T - tamaulipas
M - malintka
S - heat-treated, 37 degrees
C - non-heated treated, raised at 28 degrees

See "experiment" folder in the google drive with info about the experiment:

Experiments > Pollen Grain and Seedling RNA-seq > SRA Description - pollen grain and seedling experiments

Re-run nextflow rnaseq pipeline in:

/nobackup/tomato_genome/seedlingPollen/nfcore

Using new samples.csv file with "reverse" in the strandedness field.

Please just edit "samples.csv" in /nobackup/tomato_genome/seedlingPollen/nfcore.

Note about sample names:

MP - mature pollen
Se - seedling
N - nagcarlang
H - heinz
T - tamaulipas
M - malintka
S - stress
C - control

See folder in the google drive with info about the experiment:

* Experiments > Pollen Grain and Seedling RNA-seq

Molly Davis made changes - 18/Jan/23 3:36 PM

Attachment

Screen Shot 2023-01-18 at 3.35.37 PM.png [ 17653 ]

Hide

Permalink

Molly Davis added a comment - 18/Jan/23 3:36 PM - edited

Nextflow Pipeline Ran Successfully!
Directory: /nobackup/tomato_genome/seedlingPollen/nfcore
Next steps: Rename sorted bam files and make scaled coverage graphs.

Show

Molly Davis added a comment - 18/Jan/23 3:36 PM - edited Nextflow Pipeline Ran Successfully! Directory: /nobackup/tomato_genome/seedlingPollen/nfcore Next steps: Rename sorted bam files and make scaled coverage graphs.

Hide

Permalink

Molly Davis added a comment - 19/Jan/23 12:57 PM - edited

Scaled coverage graphs have been made and are located:

/nobackup/tomato_genome/seedlingPollen/nfcore/results/star_salmon

Notes: I can move the coverage graphs to their own directory if you would like. Let me know!

Multiqc Report:

scp mdavi258@hpc.uncc.edu:/nobackup/tomato_genome/seedlingPollen/nfcore/results/multiqc/star_salmon/multiqc_report.html nfcore_multiqc_report.html

nfcore_multiqc_report.html

Notes: Seems there are no issues with strandedness.

Next step: Pipeline, coverage graphs, and Multiqc report need to be reviewed.
[~aloraine]

Show

Molly Davis added a comment - 19/Jan/23 12:57 PM - edited Scaled coverage graphs have been made and are located: /nobackup/tomato_genome/seedlingPollen/nfcore/results/star_salmon Notes: I can move the coverage graphs to their own directory if you would like. Let me know! Multiqc Report: scp mdavi258@hpc.uncc.edu:/nobackup/tomato_genome/seedlingPollen/nfcore/results/multiqc/star_salmon/multiqc_report.html nfcore_multiqc_report.html nfcore_multiqc_report.html Notes: Seems there are no issues with strandedness. Next step : Pipeline, coverage graphs, and Multiqc report need to be reviewed. [~aloraine]

Molly Davis made changes - 19/Jan/23 12:58 PM

Assignee

Molly Davis [ molly ]

Molly Davis made changes - 19/Jan/23 12:59 PM

Status

In Progress [ 3 ]

Needs 1st Level Review [ 10005 ]

Molly Davis made changes - 19/Jan/23 1:33 PM

Attachment

nfcore_multiqc_report.html [ 17657 ]

Ann Loraine made changes - 28/Jan/23 1:01 PM

Assignee

Ann Loraine [ aloraine ]

Ann Loraine made changes - 28/Jan/23 1:01 PM

Status

Needs 1st Level Review [ 10005 ]

First Level Review in Progress [ 10301 ]

Hide

Permalink

Ann Loraine added a comment - 28/Jan/23 1:06 PM

Ran find junctions pipeline. Copied data to host with:

scp -J aloraine@hop.renci.org -r junctions aloraine@lorainelab-quickload.scidas.org:/projects/igbquickload/lorainelab/www/main/htdocs/hotpollen/S_lycopersicum_Jun_2022/mark-2022-timeseries/.

Show

Ann Loraine added a comment - 28/Jan/23 1:06 PM Ran find junctions pipeline. Copied data to host with: scp -J aloraine@hop.renci.org -r junctions aloraine@lorainelab-quickload.scidas.org:/projects/igbquickload/lorainelab/www/main/htdocs/hotpollen/S_lycopersicum_Jun_2022/mark-2022-timeseries/.

Hide

Permalink

Ann Loraine added a comment - 28/Jan/23 1:07 PM

Added samples.csv file to repository. See: https://bitbucket.org/hotpollen/splicing-analysis/src/main/ExternalData/seedlingPollen-samples.csv

Show

Ann Loraine added a comment - 28/Jan/23 1:07 PM Added samples.csv file to repository. See: https://bitbucket.org/hotpollen/splicing-analysis/src/main/ExternalData/seedlingPollen-samples.csv

Hide

Permalink

Ann Loraine added a comment - 28/Jan/23 1:09 PM

Got description of data set. See attached.

Show

Ann Loraine added a comment - 28/Jan/23 1:09 PM Got description of data set. See attached.

Ann Loraine made changes - 28/Jan/23 1:11 PM

Attachment

MichelleMedina-RNA-seq-Sample-Collection-pollen_grain-seedling.pdf [ 17668 ]

Hide

Permalink

Ann Loraine added a comment - 29/Jan/23 1:22 PM

Deployed data to quickload. Did not examine pattern of coverage graphs relative to other data sets.
Added sample sheet to repositor: https://bitbucket.org/hotpollen/splicing-analysis/src/main/ExternalData/seedlingPollen_sample_sheet.xlsx

Show

Ann Loraine added a comment - 29/Jan/23 1:22 PM Deployed data to quickload. Did not examine pattern of coverage graphs relative to other data sets. Added sample sheet to repositor: https://bitbucket.org/hotpollen/splicing-analysis/src/main/ExternalData/seedlingPollen_sample_sheet.xlsx