TSA submission: SUB14767088

1st step is the Fasta asembly uploads to FTP. After tthat, they ask for the metadata. Backwards to SRA and GEO.

Near line 14383, the local id is too long. Its length is 51 but the maximum allowed local id length is 50. Please find and correct all local ids that are too long.Allowed characters in Sequence ID include letters, digits, hyphens , underscores (_), periods (.), colons (, asterisks , number signs (#), and forward slash .Each sequence must be uniquely identified by a valid Sequence ID.

Need to rename our fasta headers to comply with the message above.

The 4 files are located here on the HPC;

/projects/tomato_genome/fnb/dataprocessing/TSA-transcriptomeShotgunAssembly/kelsieData

We need to:

1. Shorten the fasta header to be less than 51NT.
2. make sure it complies with the symbols mentions above
3. Ensure than when we truncate the header, we don't end up creating 2 identical names. Must be unique.

Completed python script and results are located in /projects/tomato_genome/fnb/dataprocessing/TSA-transcriptomeShotgunAssembly/kelsieData

Truncated all headers to match this pattern: N=NODE, L=Length, C=Coverage, S=Soly

All headers are unique, I used this command to check: cat output_file.fna | grep '^>' | uniq -cd

Moving to first level review to make sure headers are acceptable.

Fasta files look god. I will try to re-upload these to the TSA again. And we will find out what they will yell about this time.

Brandon has made changes to the fasta.

However.........according the SRA Gods:

Each sequence must be uniquely identified by a Sequence ID. Identical Sequence IDs that differ only by upper and lower case letters (eg, SEQ1 and seq1) are treated the same.

So we need to tweak these some more as the files once again have failed!

Let's devise a new fasta header format that provides some useful info (closest matching Soly ID !) and fits the requirements of the TSA guidelines.

Our original assembly IDs look like this:

>NODE_135_length_12475_cov_126.378004_g59_i0 Solyc08T001192.3 351

Let's change these to:

>Heinz-contig1-Solyc08T001192.3-351

We add s plant variety as first part, a counter for the 2nd part, the SSolyID as 3 rd part and length as 4rth part.
Use a "-" as the separator.

Wrote new python scrip to modify our assembly IDs to match TSA requirements

Script and results files are located: /projects/tomato_genome/fnb/dataprocessing/TSA-transcriptomeShotgunAssembly/kelsieData

I'm moving this to first level review to ensure the output files match what we were looking for.

Fixed script to keep the ">" in the headers.

Resubmitting to TSA. I suspect we will get the "univec" issue having some sequences match these vectors.
Will have to remove them if so.

97 lines are similar to univec adapter. Out of 113,000 sequences.

I will just remove these 97.

Well holey moly, they accept these edits.

BUT found a whole host of new issues!!!

SUBID BioProject BioSample Organism
--------------------------------------------------------
SUB14767088 PRJNA1071054 SAMN39670530 Solanum lycopersicum

We ran your sequences through our Contamination Screen. The screen found
contigs that need to be trimmed and/or excluded. The results are in the
Contamination.txt file posted in your submission on the TSA submission portal
https://submit.ncbi.nlm.nih.gov/subs/tsa/. Please adjust the sequences
appropriately and then resubmit your sequences. After you remove the
contamination, trim any Ns at the ends of the sequence and remove any sequences
that are shorter than 200 nt. More details about the contamination
screening process are available at https://github.com/ncbi/fcs

Note that mismatches between the name of the adaptor/primer identified in the screen
and the sequencing technology used to generate the sequencing data should not be used
to discount the validity of the screen results as the adaptors/primers of many
different sequencing platforms share sequence similarity.

Adaptor:
[] Some of the sequences hit primers or adaptors used in Illumina,
454, or other sequencing strategies or platforms. Adaptor at the
end of a sequence should be removed. However, if adaptors are
present within sequences then you should strongly consider
splitting the sequences at the adaptor because the adaptor sequence
could have been the region of overlap, causing a misassembly.

Screened 110,811 sequences, 356,161,895 bp.
32 sequences to exclude, 5871 sequences to trim
Potentially duplicated: 19796 sequences (with 6839 distinct checksums).

Exclude:
Sequence name, length, apparent source
Heinz-contig-1235-Solyc08T000639.1-43 4419 anml:primates
Heinz-contig-18322-Solyc09T002314.1-43 332 anml:primates
Heinz-contig-2122-Solyc09T002529.1-92 386 anml:primates
Heinz-contig-21490-Solyc12T000674.1-47 329 anml:primates
Heinz-contig-23453-Solyc07T001278.1-129 266 adaptor:multiple
Heinz-contig-24448-Solyc04T000746.1-30 513 anml:primates
Heinz-contig-24812-Solyc01T001543.1-31 384 anml:primates
Heinz-contig-5234-Solyc12T001795.1-68 348 anml:primates
Heinz-contig-7572-Solyc12T000116.1-36 5094 anml:primates
Heinz-contig-8571-Solyc08T001151.1-36 4772 anml:primates
Malintka-contig-18823-Solyc12T000116.1-36 1946 anml:primates
Malintka-contig-24783-Solyc04T001004.1-34 385 anml:primates
Malintka-contig-320-Solyc11T001603.2-94 419 anml:primates
Malintka-contig-4208-Solyc05T002076.1-648 5777 anml:primates
Malintka-contig-7496-Solyc01T001018.1-81 415 anml:primates
Nagcarlang-contig-14759-PRAM_61217.1.p2-399 922 virs:eukaryotic viruses
Nagcarlang-contig-15281-Solyc06T002346.1-39 4984 anml:primates
Nagcarlang-contig-19731-Solyc11T001218.1-279 1311 fung:basidiomycetes
Nagcarlang-contig-19771-Solyc11T001184.1-288 1311 fung:basidiomycetes
Nagcarlang-contig-20258-Solyc11T001193.1-321 1804 fung:ascomycetes
Nagcarlang-contig-35507-Solyc11T002222.6-47 893 anml:primates
Nagcarlang-contig-35508-Solyc11T002222.12-47 893 anml:primates
Nagcarlang-contig-35509-Solyc11T002222.10-47 893 anml:primates
Nagcarlang-contig-38075-Solyc09T002527.1-41 319 anml:primates
Nagcarlang-contig-7273-Solyc08T001151.1-36 7445 anml:primates
Nagcarlang-contig-9790-Solyc11T001243.1-185 3397 anml:primates
Nagcarlang-contig-9797-Solyc11T001211.1-171 943 fung:basidiomycetes
Nagcarlang-contig-9798-Solyc11T001206.1-251 943 fung:basidiomycetes
Tamaulipas-contig-1268-Solyc00T000012.4-305 1159 fung:ascomycetes
Tamaulipas-contig-13649-Solyc11T001238.1-177 2112 anml:primates
Tamaulipas-contig-1407-Solyc05T001817.1-73 333 anml:primates
Tamaulipas-contig-19628-Solyc12T000116.1-36 1818 anml:primates

Removed leading and trailing Ns from submission file, should be ready to resubmit.

Accidentally put this comment on ticket 3944: Modified script, results found in Kelsie data. I wrote function that looked at the last 12 characters, and if there was an N in those 12 it removed the all 12. Should be ready for resubmission. May it please the TSA gods.

Bringing this back one more time!!

Will submit the sequences once they get filtered using NCBI filter tool (Ticket IGB-3966)

We get filtered reads via NCBI tool.
Rob reloads to TSA ONE MORE time.

The 2 filtering steps are:
FCS_adaptor
FCS_GX

Brandon successfully ran the filtering steps and the results are here:

/projects/tomato_genome/fnb/dataprocessing/TSA-transcriptomeShotgunAssembly/kelsieData/FCS_tools/FCS_GX/tom_runs/fin_results

I am pulling them down locally and then uploading to the TSA.

I submitted this again.
I am closing the ticket!

We will generate a new ticket if needed.

Noter from Brandon where the final contigs are:

Removed duplicates and wrote them to their own file (Variety_dupes.fa). The new files are located in /projects/tomato_genome/fnb/dataprocessing/TSA-transcriptomeShotgunAssembly/kelsieData/FCS_tools/FCS_GX/tom_runs/fin_results

They follow this pattern (Variety_contaminant3_removed.fa), We should be ready for another submission. Python script is fix_contam3_v2.py and is in same directory