Details
-
Type:
Improvement
-
Status: Closed (View Workflow)
-
Priority:
Minor
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: None
-
Labels:
-
Story Points:1.5
-
Epic Link:
-
Sprint:Fall 6, Fall 7
Description
Situation: The GFF3 file format may include a Sequence Section in FASTA format at the end of the file, but IGB is not currently able to parse a GFF3 file when this section is present. See IGBF-3924 for a report on the resulting error when a GFF3 file with this section is loaded into IGB.
Here's some more info from the GFF3 documentation - https://gmod.org/wiki/GFF3
GFF3 Sequence Section
GFF3 files can also include sequence in FASTA format at the end of the file. The FASTA sequences are preceded by a ##FASTA line. This sequence section is optional. If present, the sequence section can define sequence for any landmark used in column 1 (the frame of reference). For example:##gff-version 3 ctg123 . exon 1300 1500 . + . ID=exon00001 ctg123 . exon 1050 1500 . + . ID=exon00002 ctg123 . exon 3000 3902 . + . ID=exon00003 ctg123 . exon 5000 5500 . + . ID=exon00004 ctg123 . exon 7000 9000 . + . ID=exon00005 ##FASTA >ctg123 cttctgggcgtacccgattctcggagaacttgccgcaccattccgccttg tgttcattgctgcctgcatgttcattgtctacctcggctacgtgtggcta tctttcctcggtgccctcgtgcacggagtcgagaaaccaaagaacaaaaa aagaaattaaaatatttattttgctgtggtttttgatgtgtgttttttat aatgatttttgatgtgaccaattgtacttttcctttaaatgaaatgtaat cttaaatgtatttccgacgaattcgaggcctgaaaagtgtgacgccattc ...When the GFF3 file is processed the IDs on the header line of FASTA entries are matched with IDs used in column 1 in the annotation section of the file.
You don’t have to store the FASTA in the GFF file. You can also store your sequences in a separate file containing only FASTA entries.
Task: Upgrade the GFF parser logic to be able to handle GFF3 files with a Sequence Section.
Example files which are not being parsed correctly:
- prodigal_Lambda_phage_sequences.gff
- FragGeneScan_Lambda_phage_sequences.gff
Link to those files on Google Drive: https://drive.google.com/drive/folders/14noPsmKYMxX9jgHYQhkjqaTGzT8z8bSK
Link to files on Loraine Lab Google Drive: https://drive.google.com/drive/folders/1MLsVItXNcskfiCAg62GFmxWc1-NR40Tx?usp=drive_link
Attachments
Issue Links
Activity
We discussed this ticket during today's scrum and we've decided to shift the focus of this investigation slightly.
We think what’s supposed to happen when attempting to open a .gff file in IGB is that the ##FASTA section gets parsed as the reference sequence and annotations from the rest of the .gff file are added to IGB as annotations, but we’re not sure. I'm creating a follow-up ticket to investigate whether or not this is the case.
Closing this ticket!
Question for Paige Kulzer:
I notice the files with ##FASTA sections that can be loaded are lacking line breaks in the sequence part of the file.
Does the specification allow for line breaks in the ##FASTA section?
Since making this ticket, I've come across some cases where the presence of this ##FASTA section in a file does not cause IGB to throw errors when it's being loaded.
This suggests that IGB is already set up to correctly parse this ##FASTA section, but the logic for parsing the fasta headers themselves needs to be improved.
Example of ##FASTA section that does load in IGB (prodigal_soil_flongle_DNA_AGB452.gff):
##FASTA >c8597f60-210d-4449-b1e5-0eeba67c98c7 runid=03d89df21b3aa3865c4ff07c593cd7fb3c58dec9 read=8 ch=121 start_time=2021-07-15T23:10:12Z flow_cell_id=AGB452 protocol_group_id=Sss_infested_soil_2_total_DNA sample_id=Sss_infested_soil_14_galls_Aug_2020 TCGTCTGTATGCTTCGTTCAGTTTTACGTACCTTATCGAAATCCGGTTTACGTTTGCTGTTCTCTACATCGGCGTTCAACTGCTGTAACTGAATATTTCGGTCGATGCTCCTGATGTCGCTTCGGGCTGAGGCAAGTGCTGCTGAATCTGTTAACTGCTGCCTCATAATCATGAACAACTATATTGGTATCGACTGCAAAGGCAGTATTCTTGTCGCAGTTCATTAAGGTGTTGAGTATGATATTTTTTTGCCGAATCTCGTTAT >c52dc868-7a86-48de-a312-0a37a64447f9 runid=03d89df21b3aa3865c4ff07c593cd7fb3c58dec9 read=11 ch=74 start_time=2021-07-15T23:10:12Z flow_cell_id=AGB452 protocol_group_id=Sss_infested_soil_2_total_DNA sample_id=Sss_infested_soil_14_galls_Aug_2020 CCGACGGCCTGTCCTGCCAAGGGAACTGTAGATTAATTTTCTTATTGTGCAGGCAAAGACGCCTGGCTGGTGACTACGACCGTGCGGTCGTGAAGCAGGTAGTGAGGTTCGCCGGGTCAACGGGGTGAAGTGGAGTAAACCTCGGCGCAGACATTTCATGTCCAGAGCTTCATGGGGTCTTTCTGATTGAATTCCTCGAGGAAGGCGTCGGAGCAGCTTGCTGCTGCAGGATGTTGGCGCCGGCCGGGCGAGTGGCTTCTTTTTCAGCGTCAGATGCATGCGTTCATGGCGGCCGTTCTGCTGAGGATGGCCGGGGCGAATCCGCTCAATGGAGAT >71059d69-f81b-4347-a7b2-943c1399f9b3 runid=03d89df21b3aa3865c4ff07c593cd7fb3c58dec9 read=9 ch=54 start_time=2021-07-15T23:10:12Z flow_cell_id=AGB452 protocol_group_id=Sss_infested_soil_2_total_DNA sample_id=Sss_infested_soil_14_galls_Aug_2020 AAATCCACAAAACTTGCGCCCAGCATTGTTGCCGCGCTGCTGGAAAATCCTTCCGTACGTGATGGCGCACTGGTGGATTTGATGGCTGCTGGTTCGAAGAACGTAATCGAAGTGATGTTGAAGAGTCCGCGGATAAGCGTTCTGCCCGGCTAACGGAGGGCTTAGAAAAGCCCACCTCGTCGAACGCAAAGTCGACGAAGGCGCCAGAAGAACCGGTTCCAGTTACACCAGCGGTGGTTGAATCCAAAAACCAGCGACAGGGAATTCAGAGTTTCCCGAGTTTTCTGTGGAAGCGCCACTGGACGAAAACGCAGAAAGTACCGGATGAAATGGTGAATGCTTACTTTGATGAACACTCCGCCGAACTTACAGCTGAAGGAGAGAAGCCGTTTAAAGCGATCAGCGGTGTATACGATGAAATCGATTTAGGAGCACCGGAAGAAAATGCAACGCCGCGAGTTAAGAA
Example of ##FASTA section that does not load in IGB (prodigal_Lambda_phage_sequences.gff):
##FASTA >NC_001416.1 |Enterobacteria phage lambda, complete genome GGGCGGCGACCTCGCGGGTTTTCGCTATTTATGAAAATTTTCCGGTTTAAGGCGTTTCCG TTCTTCTTCGTCATAACTTAATGTTTTTATTTAAAATACCCTCTGAAAAGAAAGGAAACG ACAGGTGCTGAAAGCGAGGCTTTTTGGCCTCTGTCGTTTCCTTTCTCTGTTTTTGTCCGT GGAATGAACAATGGAAGTCAACAAAAAGCAGCTGGCTGACATTTTCGGTGCGAGTATCCG TACCATTCAGAACTGGCAGGAACAGGGAATGCCCGTTCTGCGAGGCGGTGGCAAGGGTAA TGAGGTGCTTTATGACTCTGCCGCCGTCATAAAATGGTATGCCGAAAGGGATGCTGAAAT TGAGAACGAAAAGCTGCGCCGGGAGGTTGAAGAACTGCGGCAGGCCAGCGAGGCAGATCT CCAGCCAGGAACTATTGAGTACGAACGCCATCGACTTACGCGTGCGCAGGCCGACGCACA GGAACTGAAGAATGCCAGAGACTCCGCTGAAGTGGTGGAAACCGCATTCTGTACTTTCGT GCTGTCGCGGATCGCAGGTGAAATTGCCAGTATTCTCGACGGGCTCCCCCTGTCGGTGCA GCGGCGTTTTCCGGAACTGGAAAACCGACATGTTGATTTCCTGAAACGGGATATCATCAA AGCCATGAACAAAGCAGCCGCGCTGGATGAACTGATACCGGGGTTGCTGAGTGAATATAT CGAACAGTCAGGTTAACAGGCTGCGGCATTTTGTCCGCGCCGGGCTTCGCTCACTGTTCA GGCCGGAGCCACAGACCGCCGTTGAATGGGCGGATGCTAATTACTATCTCCCGAAAGAAT
There is some logic in the IGB codebase for handling FASTA in GFF. See GFF3.java and GFF3Parser.java and look for ##FASTA.
Error in IGB when we try to load data with FASTA on bottom:
15:12:34.242 ERROR c.a.genometry.thread.CThreadWorker - class com.affymetrix.genometry.symloader.GFF3 cannot be cast to class com.affymetrix.genometry.quickload.QuickLoadSymLoader (com.affymetrix.genometry.symloader.GFF3 and com.affymetrix.genometry.quickload.QuickLoadSymLoader are in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @489f8b99) java.lang.ClassCastException: class com.affymetrix.genometry.symloader.GFF3 cannot be cast to class com.affymetrix.genometry.quickload.QuickLoadSymLoader (com.affymetrix.genometry.symloader.GFF3 and com.affymetrix.genometry.quickload.QuickLoadSymLoader are in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @489f8b99) at com.affymetrix.igb.view.load.GeneralLoadUtils$3.runInBackground(GeneralLoadUtils.java:1022) at com.affymetrix.igb.view.load.GeneralLoadUtils$3.runInBackground(GeneralLoadUtils.java:1001) at com.affymetrix.genometry.thread.CThreadWorker.doInBackground(CThreadWorker.java:73) at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:305) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:342) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) 15:12:34.253 ERROR c.a.igb.view.load.GeneralLoadUtils - null java.util.concurrent.ExecutionException: java.lang.ClassCastException: class com.affymetrix.genometry.symloader.GFF3 cannot be cast to class com.affymetrix.genometry.quickload.QuickLoadSymLoader (com.affymetrix.genometry.symloader.GFF3 and com.affymetrix.genometry.quickload.QuickLoadSymLoader are in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @489f8b99) at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191) at java.desktop/javax.swing.SwingWorker.get(SwingWorker.java:612) at com.affymetrix.igb.view.load.GeneralLoadUtils$3.finished(GeneralLoadUtils.java:1050) at com.affymetrix.genometry.thread.CThreadWorker.done(CThreadWorker.java:51) at java.desktop/javax.swing.SwingWorker$4.run(SwingWorker.java:749) at java.desktop/javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.run(SwingWorker.java:847) at java.desktop/sun.swing.AccumulativeRunnable.run(AccumulativeRunnable.java:112) at java.desktop/javax.swing.SwingWorker$DoSubmitAccumulativeRunnable.actionPerformed(SwingWorker.java:857) at java.desktop/javax.swing.Timer.fireActionPerformed(Timer.java:311) at java.desktop/javax.swing.Timer$DoPostEvent.run(Timer.java:243) at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:318) at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:773) at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:720) at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:714) at java.base/java.security.AccessController.doPrivileged(AccessController.java:400) at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:87) at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:742) at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90) Caused by: java.lang.ClassCastException: class com.affymetrix.genometry.symloader.GFF3 cannot be cast to class com.affymetrix.genometry.quickload.QuickLoadSymLoader (com.affymetrix.genometry.symloader.GFF3 and com.affymetrix.genometry.quickload.QuickLoadSymLoader are in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @489f8b99) at com.affymetrix.igb.view.load.GeneralLoadUtils$3.runInBackground(GeneralLoadUtils.java:1022) at com.affymetrix.igb.view.load.GeneralLoadUtils$3.runInBackground(GeneralLoadUtils.java:1001) at com.affymetrix.genometry.thread.CThreadWorker.doInBackground(CThreadWorker.java:73) at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:305) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:342) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583)
Note: The issue appears to be with a second new line character after the fasta section causing a null pointer exception. The issue is not with the fasta section itself.