Next steps:

• Make spreadsheet of data for SL4 and SL5 reruns and prepare for quick load submission to IGB. Can do this by going on NCBI and looking at all SRA associated with NSF tomato grant.
• Allow permission to move files from cluster to IGB quick load.

Spreadsheet info needed:

SRP Experiments:
mark-2022-timeseries = SRP441343

• Directory Location: /projects/tomato_genome/fnb/dataprocessing/SRP441343
• Quickload files location SL4: /projects/tomato_genome/fnb/dataprocessing/SRP441343/nfcore-SL4/results/star_salmon
• Quickload files location SL5: /projects/tomato_genome/fnb/dataprocessing/SRP441343/nfcore-SL5/results/star_salmon
• Cluster Access Permission: chmod -R g+w *
• Sample Sheet Name: SRP441343_sample_sheet.xlsx
• SRA rerun review: Check SRA successful IGBF-3639

mark-2020-pollentube = SRP252265

• Directory Location: /projects/tomato_genome/fnb/dataprocessing/SRP252265
• Quickload files location SL4: /projects/tomato_genome/fnb/dataprocessing/SRP252265/nfcore-SL4/results/star_salmon
• Quickload files location SL5: /projects/tomato_genome/fnb/dataprocessing/SRP252265/nfcore-SL5/results/star_salmon
• Cluster Access Permission: chmod -R g+w *
• Sample Sheet Name: Not finished
• SRA rerun review:Not Finished

muday-2022-timeseries = SRP460750

• Directory Location: /projects/tomato_genome/fnb/dataprocessing/SRP460750
• Quickload files location SL4: /projects/tomato_genome/fnb/dataprocessing/SRP460750/nfcore-SL4-2024-05-21/results/star_salmon
• Quickload files location SL5: /projects/tomato_genome/fnb/dataprocessing/SRP460750/nfcore-SL5-2024-05-07/results/star_salmon
• Cluster Access Permission: chmod -R g+w *
• Sample Sheet Name: SRP460750_sample_sheet.xlsx
• SRA rerun review: Check SRA successful, possible edits required to metadata

muday-ARE-120min = SRP499796

• Directory Location: /projects/tomato_genome/fnb/dataprocessing/SRP499796
• Quickload files location SL4: /projects/tomato_genome/fnb/dataprocessing/SRP499796/nfcore-SL4/results/star_salmon
• Quickload files location SL5: /projects/tomato_genome/fnb/dataprocessing/SRP499796/nfcore-SL5/results/star_salmon
• Cluster Access Permission: chmod -R g+w *
• Sample Sheet Name: SRP499796_sample_sheet.xlsx
• SRA rerun review: Check SRA successful, possible edits required to metadata

seedlingPollen = SRP438952

• Directory Location: /projects/tomato_genome/fnb/dataprocessing/SRP438952
• Quickload files location SL4: /projects/tomato_genome/fnb/dataprocessing/SRP438952/nfcore-SL4/results/star_salmon
• Quickload files location SL5: /projects/tomato_genome/fnb/dataprocessing/SRP438952/nfcore-SL5/results/star_salmon
• Cluster Access Permission: chmod -R g+w *
• Sample Sheet Name: SRP438952_sample_sheet.xlsx
• SRA rerun review: Check SRA successful, possible edits required to metadata

Ravi 30-681594536 = SRP486761

• Directory Location: /projects/tomato_genome/fnb/dataprocessing/SRP486761
• Quickload files location SL4: /projects/tomato_genome/fnb/dataprocessing/SRP486761/nfcore-SL4/results/star_salmon
• Quickload files location SL5: /projects/tomato_genome/fnb/dataprocessing/SRP486761/nfcore-SL5/results/star_salmon
• Cluster Access Permission: chmod -R g+w *
• SRA rerun review: Check SRA successful, possible edits required to metadata

Ravi 30-804059537 = SRP487154

• Directory Location: /projects/tomato_genome/fnb/dataprocessing/SRP487154
• Quickload files location SL4: /projects/tomato_genome/fnb/dataprocessing/SRP487154/nfcore-SL4/results/star_salmon
• Quickload files location SL5: /projects/tomato_genome/fnb/dataprocessing/SRP487154/nfcore-SL5/results/star_salmon
• Cluster Access Permission: chmod -R g+w *
• SRA rerun review: Check SRA successful, possible edits required to metadata

Ravi 30-605730043 = SRP482647

• Directory Location: /projects/tomato_genome/fnb/dataprocessing/SRP482647
• Quickload files location SL4: /projects/tomato_genome/fnb/dataprocessing/SRP482647/nfcore-SL4/results/star_salmon
• Quickload files location SL5: /projects/tomato_genome/fnb/dataprocessing/SRP482647/nfcore-SL5/results/star_salmon
• Cluster Access Permission: chmod -R g+w *
• Sample Sheet Name:Not Finished
• SRA rerun review: Check SRA successful, possible edits required to metadata

Main task that is left:

• Compare new and old data processing to confirm the new data are OK
• For this, see "Not Finished" labels in the preceding comment.
• Schedule ~ 45 minutes with Ann Loraine to learn how to compare the pre- and post SRA-submission data using multiqc reports

Moving data to:

/projects/tomato_genome/fnb/dataprocessing/for_quickload/S_lycopersicum_Sep_2019
and
/projects/tomato_genome/fnb/dataprocessing/for_quickload/S_lycopersicum_Jun_2022

and copying data by logging into data.bioviz.org and running an rsync command from there, starting with data set SRP441343

rsync -rtpvz aloraine@hpc.charlotte.edu:/projects/tomato_genome/fnb/dataprocessing/for_quickload/S_lycopersicum_Jun_2022/SRP441343/* SRP441343/.

and

rsync -rtpvz 
aloraine@hpc.charlotte.edu:/projects/tomato_genome/fnb/dataprocessing/for_quickload/S_lycopersicum_Sep_2019/SRP252265-SL4/* SRP252265-SL4/.

All SRA-reruns are now moved to hosting at data.bioviz.org.

example transfer command:

aloraine@cci-vm12:/mnt/igbdata/hotpollen/S_lycopersicum_Jun_2022$ rsync -rtpvz aloraine@hpc.charlotte.edu:/projects/tomato_genome/fnb/dataprocessing/for_quickload/S_lycopersicum_Jun_2022/SRP487154-SL5/ SRP487154-SL5

Update:

• seedlingPollen = SRP438952; the stress temps reported in the SRA do not match those from our original sample spreadsheet. SRA says stress temperature was 37 degrees C, but the original sample sheet says the stress temperature was 34 degrees C

• mark-2022-timeseries = SRP441343; samples match and so do the sample codes. Can move ahead with releasing the data under the SRA code

• 30-605730043=SRP482647; Palanivelu lab 10-samples pilot; the sample codes in the SRA do not match with the sample codes. The SRA Run selector reports .1 and .2 library name suffixes. Also, duration (0 hr, etc) is not a variable in its own column. I would recommend updating the sample name / library name columns if possible. It might be good to change the replicate suffix to .3 instead of what is there now since replicates 1 and 2 are reported for Tamaulipas samples in the larger experiment testing the samples from this experiment.

• 30-804059537=SRP487154; Palanivelu lab 63-samples, ovary (3 samples) and self-pollinated pistil+style (60 samples); 4 varieties; 2 temperatures; 3 treatment durations; 3 replicates. The sample codes used by the SRA record are richer - they make it more clear with components represent temperature and which components represent treatment duration, e.g., M.37.8.S.1 is M.37C.8hr.S.1 in the Library Name field. No changes needed in my opinion. This is fine.

• 30-681594536=SRP486761; Palanivelu lab 55 samples, unpollinated pistils; 4 varieties; 3 durations; Tamaulipas samples have only two replicates due to redundancy with pilot study samples

• Muday lab time course experiment is good. I re-ran the "retest" markdown in flavonoid repository and got the same result - all 72 samples are fine.

Possibly interesting bioinformatics question: Will pilot samples cluster with similar samples types from the un-pollinated pistils study? I think the samples were made together but the libraries were made at a different time

Update

• Need to add 180 minute anthocyanin-reduced dataset
• Designing the following folder structure for data sets when displayed in IGB:

path to a file in the IGB model of how files (data sets) exist on a file system, as depicted in the interface:

• Research-PGR: [grant title reported on NSF Web site] / [PI] / [Study] / [Reads],[Graph - Scaled],[Junctions]

Update:

• All PGRP RNA-Seq data that have been uploaded to the SRA and checked against our local original copies have been deployed to quickloads and made available on-line
• The PGRP RNA-Seq are in a PGRP folder under the RNA-Seq quickload site
• All code required to create the QL directories are available in main branch of genome browser repository in hotpollen bitbucket workspace - see: https://bitbucket.org/hotpollen/genome-browser-visualization/src/main/
• Ready for testing (skipping first-level review)

To test:

• Launch latest build of IGB (get it from Early Access section of BioViz.org)
• Do this for the two latest versions of the tomato genome (nicknames: SL4 and SL5)
• Open the "PGRP" folder and navigate down to each data set
• Click on each dataset to add it as a track to IGB
• If a dataset cannot be added (typically because a file is missing from the remote host) make a note of which file was not able to be shown. Also mention the computer and network you are using to access the file, as this could also affect what happens when you try to load a file from the internet, which is what you are doing.
• Don't bother continuing to test if you find a problem file - e.g., a file with a checkbox in IGB that couldn't be loaded into a track. Just make a note of it, kick this back to "To-Do", and re-assign to the developer (me), as per usual
• If all datasets can be loaded, move this to DONE

Thank you for testing!

The one that generated an error for me is for SL4 (2019), Muday Lab/pollen tube, anthocyanin reduced (are) heat stress, 15 - 75 min (SRP460750)/anthocyanin reduced (are) 28 Celsius 15 min, rep A.28.15.7 (SRR25478302) alignments.

I am on a Windows laptop, and I am connected to the Eduroam network

Thank you for the info Brandon Bendickson!

I have looked into what the problem was. It turns out that I had forgotten to make all the files world-readable. I have made that change on the server and checked a few of the files from the Muday lab datasets. It looks good from my end.

Please re-commence testing Brandon Bendickson when ready!

Ran into the same problem again, with the same file from the previous comment. (Muday Lab/pollen tube, anthocyanin reduced (are) heat stress, 15 - 75 min (SRP460750)/anthocyanin reduced (are) 28 Celsius 15 min, rep A.28.15.7 (SRR25478302) alignments)

The error message was: The feature http://igbquickload.org/hotpollen/S_lycopersicum_Sep_2019/SRP460750-SL4/SRR25478302.bam is not reachable. moving ticket back to To do

Thank you Brandon Bendickson! Sorry, I thought I had fixed the problem. Now I have - I think!

I modified permissions using:

find . -type f | xargs chmod a+r

and noticed there were some other problems with the permissions, which I fixed using commands like:

find . -type d | xargs chmod a+x
find . -type f | xargs chmod a-x
find . | xargs chmod o-w

Viewing and checking permissions for the entire file tree with:

find . -type d | xargs ls -ld # check directory permissions
find . -type f | xargs ls -l # checking file permissions 

Brandon Bendickson please check again and thank you for noticing the problem!

All SL4 is good and all SL5 is good. I ran into no further errors, moving this to done!