[IGBF-3441] Fix DAS error - JIRA UNCC

Nowlan Freese created issue - 13/Sep/23 1:44 PM

Nowlan Freese made changes - 13/Sep/23 1:44 PM

Field	Original Value	New Value
Epic Link		IGBF-1765 [ 17855 ]

Nowlan Freese made changes - 13/Sep/23 1:45 PM

Status

To-Do [ 10305 ]

In Progress [ 3 ]

Hide

Permalink

Nowlan Freese added a comment - 13/Sep/23 1:45 PM

Error:

10:21:14.674 ERROR o.l.igb.das.utils.DasServerUtils - Could not initialize JAXBContext for org.lorainelab.igb.das.model.dsn.DasDsn 
javax.xml.bind.UnmarshalException: null
	at javax.xml.bind.helpers.AbstractUnmarshallerImpl.createUnmarshalException(AbstractUnmarshallerImpl.java:335) ~[na:1.8.0_332]
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.createUnmarshalException(UnmarshallerImpl.java:563) ~[na:1.8.0_332]
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:249) ~[na:1.8.0_332]
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:214) ~[na:1.8.0_332]
	at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:140) ~[na:1.8.0_332]
	at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:123) ~[na:1.8.0_332]
	at org.lorainelab.igb.das.utils.DasServerUtils.retrieveDsnResponse(DasServerUtils.java:68) ~[das-9.1.10.jar:na]
	at org.lorainelab.igb.das.DasDataProvider.initialize(DasDataProvider.java:103) [das-9.1.10.jar:na]
	at com.affymetrix.igb.general.DataProviderManager.initializeDataProvider(DataProviderManager.java:333) [igb-9.1.10.jar:na]
	at com.affymetrix.igb.general.DataProviderManager.lambda$addDataProvider$87(DataProviderManager.java:324) [igb-9.1.10.jar:na]
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311) ~[na:1.8.0_332]
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758) ~[na:1.8.0_332]
	at java.awt.EventQueue.access$500(EventQueue.java:97) ~[na:1.8.0_332]
	at java.awt.EventQueue$3.run(EventQueue.java:709) ~[na:1.8.0_332]
	at java.awt.EventQueue$3.run(EventQueue.java:703) ~[na:1.8.0_332]
	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_332]
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74) ~[na:1.8.0_332]
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728) ~[na:1.8.0_332]
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205) ~[na:1.8.0_332]
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) ~[na:1.8.0_332]
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) ~[na:1.8.0_332]
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) ~[na:1.8.0_332]
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) ~[na:1.8.0_332]
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) ~[na:1.8.0_332]
Caused by: org.xml.sax.SAXParseException: White spaces are required between publicId and systemId.
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:204) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:178) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:399) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:326) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1466) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.impl.XMLScanner.scanExternalID(XMLScanner.java:1066) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.scanDoctypeDecl(XMLDocumentScannerImpl.java:641) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:923) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:601) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:504) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) ~[na:1.8.0_332]
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:642) ~[na:1.8.0_332]
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:243) ~[na:1.8.0_332]
	... 21 common frames omitted

Show

Nowlan Freese added a comment - 13/Sep/23 1:45 PM Error: 10:21:14.674 ERROR o.l.igb.das.utils.DasServerUtils - Could not initialize JAXBContext for org.lorainelab.igb.das.model.dsn.DasDsn javax.xml.bind.UnmarshalException: null at javax.xml.bind.helpers.AbstractUnmarshallerImpl.createUnmarshalException(AbstractUnmarshallerImpl.java:335) ~[na:1.8.0_332] at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.createUnmarshalException(UnmarshallerImpl.java:563) ~[na:1.8.0_332] at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:249) ~[na:1.8.0_332] at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:214) ~[na:1.8.0_332] at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:140) ~[na:1.8.0_332] at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:123) ~[na:1.8.0_332] at org.lorainelab.igb.das.utils.DasServerUtils.retrieveDsnResponse(DasServerUtils.java:68) ~[das-9.1.10.jar:na] at org.lorainelab.igb.das.DasDataProvider.initialize(DasDataProvider.java:103) [das-9.1.10.jar:na] at com.affymetrix.igb.general.DataProviderManager.initializeDataProvider(DataProviderManager.java:333) [igb-9.1.10.jar:na] at com.affymetrix.igb.general.DataProviderManager.lambda$addDataProvider$87(DataProviderManager.java:324) [igb-9.1.10.jar:na] at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311) ~[na:1.8.0_332] at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758) ~[na:1.8.0_332] at java.awt.EventQueue.access$500(EventQueue.java:97) ~[na:1.8.0_332] at java.awt.EventQueue$3.run(EventQueue.java:709) ~[na:1.8.0_332] at java.awt.EventQueue$3.run(EventQueue.java:703) ~[na:1.8.0_332] at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_332] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74) ~[na:1.8.0_332] at java.awt.EventQueue.dispatchEvent(EventQueue.java:728) ~[na:1.8.0_332] at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205) ~[na:1.8.0_332] at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) ~[na:1.8.0_332] at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) ~[na:1.8.0_332] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) ~[na:1.8.0_332] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) ~[na:1.8.0_332] at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) ~[na:1.8.0_332] Caused by: org.xml.sax.SAXParseException: White spaces are required between publicId and systemId. at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:204) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:178) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:399) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:326) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1466) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.impl.XMLScanner.scanExternalID(XMLScanner.java:1066) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.scanDoctypeDecl(XMLDocumentScannerImpl.java:641) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:923) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:601) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:504) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) ~[na:1.8.0_332] at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:642) ~[na:1.8.0_332] at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:243) ~[na:1.8.0_332] ... 21 common frames omitted

Hide

Permalink

Nowlan Freese added a comment - 13/Sep/23 1:46 PM

Note that the DAS still appears normally as a data provider in the Data Sources tab.

Show

Nowlan Freese added a comment - 13/Sep/23 1:46 PM Note that the DAS still appears normally as a data provider in the Data Sources tab.

Nowlan Freese made changes - 13/Sep/23 1:47 PM

Description

Situation: When starting IGB the below error is occurring, causing the DAS data to fail to load.

Task: Investigate the DAS error.

Situation: When starting IGB 9.1.10 the below error is occurring, causing the DAS data to fail to load.

Task: Investigate the DAS error.

Hide

Permalink

Nowlan Freese added a comment - 13/Sep/23 4:17 PM

The primary issue seems to be that UCSC is now forcing a redirect from http.

When starting IGB the DAS server is initialized and IGB attempts to retrieve the following URL: http://genome.cse.ucsc.edu/cgi-bin/das/dsn
However, this URL now redirects to https://genome.cse.ucsc.edu/cgi-bin/das/dsn
Simply changing the URL in igbDefaultPrefs.json corrects the issue.

However, the above URL retrieves an XML file. Inside of that file are links to the lists of files for each genome supported. These links are old http links that now also force a redirect:
http://genome.ucsc.edu:80/cgi-bin/das/hg38 -> https://genome.ucsc.edu/cgi-bin/das/hg38

I am unsure of how to correct the issue as the DasServerUtils.java retrieveDasTypesResponse HttpRequest method is set to follow redirects, but that does not seem to be working correctly.

Show

Nowlan Freese added a comment - 13/Sep/23 4:17 PM The primary issue seems to be that UCSC is now forcing a redirect from http. When starting IGB the DAS server is initialized and IGB attempts to retrieve the following URL: http://genome.cse.ucsc.edu/cgi-bin/das/dsn However, this URL now redirects to https://genome.cse.ucsc.edu/cgi-bin/das/dsn Simply changing the URL in igbDefaultPrefs.json corrects the issue. However, the above URL retrieves an XML file. Inside of that file are links to the lists of files for each genome supported. These links are old http links that now also force a redirect: http://genome.ucsc.edu:80/cgi-bin/das/hg38 -> https://genome.ucsc.edu/cgi-bin/das/hg38 I am unsure of how to correct the issue as the DasServerUtils.java retrieveDasTypesResponse HttpRequest method is set to follow redirects, but that does not seem to be working correctly.

Hide

Permalink

Ann Loraine added a comment - 13/Sep/23 5:29 PM

Potentially yucky solution:

We could modify the IGB code to insert "https" instead of "http" in the DAS text retrieved from the resource.

Show

Ann Loraine added a comment - 13/Sep/23 5:29 PM Potentially yucky solution: We could modify the IGB code to insert "https" instead of "http" in the DAS text retrieved from the resource.

Nowlan Freese made changes - 15/Sep/23 11:45 AM

Link

This issue relates to ~~IGBF-3103~~ [ ~~IGBF-3103~~ ]

Hide

Permalink

Nowlan Freese added a comment - 15/Sep/23 1:15 PM - edited

Branch: https://bitbucket.org/nfreese/nowlanfork-igb/branch/IGBF-3441

I used code that Karthik had written for ~~IGBF-3103~~. Note that the commit for ~~IGBF-3103~~ was reverted due to an issue with secure quickloads, however, that shouldn't be an issue with loading the DAS data.

Changes are broken into two commits, one that includes the code changes for identifying the redirect url, and a separate commit where I updated the various UCSC urls used during testing or as part of igbDefaultPrefs files. These were causing some tests to fail.

To test:

Fetch my branch
Build IGB with tests (mvn clean install) - all tests should pass
Start IGB
Click on the mouse genome (M_musculus_Jun_2020).
See that the UCSC (DAS) folder is available in the Data Access tab under Available Data.
Select and load a data file from the UCSC (DAS) folder (make sure to click Load Data)
Next, open the chicken genome (G_gallus_Mar_2018) (note that this version of the genome is not available currently from IGB quickload, so all of the data comes from UCSC)
Select and load a data file from the UCSC (DAS) folder (make sure to click Load Data)
Zoom in to 1:72,715,677-72,715,720
Click Load Sequence
Check the IGB logs to make sure there are no errors

Show

Nowlan Freese added a comment - 15/Sep/23 1:15 PM - edited Branch: https://bitbucket.org/nfreese/nowlanfork-igb/branch/IGBF-3441 I used code that Karthik had written for IGBF-3103 . Note that the commit for IGBF-3103 was reverted due to an issue with secure quickloads, however, that shouldn't be an issue with loading the DAS data. Changes are broken into two commits, one that includes the code changes for identifying the redirect url, and a separate commit where I updated the various UCSC urls used during testing or as part of igbDefaultPrefs files. These were causing some tests to fail. To test: Fetch my branch Build IGB with tests (mvn clean install) - all tests should pass Start IGB Click on the mouse genome (M_musculus_Jun_2020). See that the UCSC (DAS) folder is available in the Data Access tab under Available Data. Select and load a data file from the UCSC (DAS) folder (make sure to click Load Data) Next, open the chicken genome (G_gallus_Mar_2018) (note that this version of the genome is not available currently from IGB quickload, so all of the data comes from UCSC) Select and load a data file from the UCSC (DAS) folder (make sure to click Load Data) Zoom in to 1:72,715,677-72,715,720 Click Load Sequence Check the IGB logs to make sure there are no errors

Nowlan Freese made changes - 15/Sep/23 1:16 PM

Status

In Progress [ 3 ]

Needs 1st Level Review [ 10005 ]

Nowlan Freese made changes - 15/Sep/23 1:16 PM

Assignee

Nowlan Freese [ nfreese ]

Hide

Permalink

Karthik Raveendran added a comment - 17/Sep/23 1:40 AM

No errors in logs. Requesting Pull Request.

Show

Karthik Raveendran added a comment - 17/Sep/23 1:40 AM No errors in logs. Requesting Pull Request.

Karthik Raveendran made changes - 17/Sep/23 1:40 AM

Status

Needs 1st Level Review [ 10005 ]

First Level Review in Progress [ 10301 ]

Karthik Raveendran made changes - 17/Sep/23 1:40 AM

Status

First Level Review in Progress [ 10301 ]

Ready for Pull Request [ 10304 ]

Hide

Permalink

Nowlan Freese added a comment - 18/Sep/23 10:01 AM - edited

[~aloraine] - Pull request: https://bitbucket.org/lorainelab/integrated-genome-browser/pull-requests/948

Show

Nowlan Freese added a comment - 18/Sep/23 10:01 AM - edited [~aloraine] - Pull request: https://bitbucket.org/lorainelab/integrated-genome-browser/pull-requests/948

Nowlan Freese made changes - 18/Sep/23 10:01 AM

Status

Ready for Pull Request [ 10304 ]

Pull Request Submitted [ 10101 ]

Ann Loraine made changes - 18/Sep/23 11:55 AM

Sprint

Fall 1 - Sep 5 [ 82 ]

Fall 1 - Sep 5, Fall 2 2023 Sep 17 [ 82, 178 ]

Ann Loraine made changes - 18/Sep/23 11:55 AM

Rank

Ranked higher

Nowlan Freese made changes - 18/Sep/23 1:16 PM

Assignee

Ann Loraine [ aloraine ]

Ann Loraine made changes - 18/Sep/23 1:25 PM

Status

Pull Request Submitted [ 10101 ]

Reviewing Pull Request [ 10303 ]

Ann Loraine made changes - 18/Sep/23 1:26 PM

Status

Reviewing Pull Request [ 10303 ]

Merged Needs Testing [ 10002 ]

Ann Loraine made changes - 18/Sep/23 1:26 PM

Assignee

Ann Loraine [ aloraine ]

Hide

Permalink

Ann Loraine added a comment - 18/Sep/23 1:27 PM

PR is merged into main-JDK8.

Show

Ann Loraine added a comment - 18/Sep/23 1:27 PM PR is merged into main-JDK8.

Nowlan Freese made changes - 20/Sep/23 10:29 AM

Summary

Investigate DAS error

Fix DAS error

Karthik Raveendran made changes - 20/Sep/23 10:37 AM

Status

Merged Needs Testing [ 10002 ]

Post-merge Testing In Progress [ 10003 ]

Hide

Permalink

Karthik Raveendran added a comment - 20/Sep/23 10:37 AM

After including this changes in main-JDK8-r6, I tested it with no errors.

Show

Karthik Raveendran added a comment - 20/Sep/23 10:37 AM After including this changes in main-JDK8-r6, I tested it with no errors.

Karthik Raveendran made changes - 20/Sep/23 10:38 AM

Status

Post-merge Testing In Progress [ 10003 ]

Merged Needs Testing [ 10002 ]

Karthik Raveendran made changes - 21/Sep/23 10:36 AM

Assignee

Karthik Raveendran [ karthik ]

Karthik Raveendran made changes - 21/Sep/23 10:36 AM

Status

Merged Needs Testing [ 10002 ]

Post-merge Testing In Progress [ 10003 ]

Hide

Permalink

Karthik Raveendran added a comment - 21/Sep/23 10:37 AM

Tested the installer in https://bitbucket.org/lorainelab/integrated-genome-browser/downloads/. No errors found.

Moving to close.

Show

Karthik Raveendran added a comment - 21/Sep/23 10:37 AM Tested the installer in https://bitbucket.org/lorainelab/integrated-genome-browser/downloads/ . No errors found. Moving to close.

Karthik Raveendran made changes - 21/Sep/23 10:37 AM

Resolution		Done [ 10000 ]
Status	Post-merge Testing In Progress [ 10003 ]	Closed [ 6 ]

Karthik Raveendran made changes - 21/Sep/23 10:37 AM

Assignee

Karthik Raveendran [ karthik ]

Nowlan Freese [ nfreese ]

Paige Kulzer made changes - 05/Feb/24 12:46 PM

Fix Version/s

10.0.0 Major Release [ 10900 ]

Nowlan Freese made changes - 07/Feb/24 1:52 PM

Issue Type

Task [ 3 ]

Bug [ 1 ]

Fix DAS error

Details

Description

Attachments

Issue Links

Activity

People

Dates