Details

      Description

      Situation: A user has asked for the UCSC DAS functionality to be returned to IGB.

      Task: Revert the changes to remove DASv1 support from IGB.

        Attachments

          Issue Links

            Activity

            Hide
            nfreese Nowlan Freese added a comment -

            Branch: https://bitbucket.org/nfreese/nowlanfork-igb/branch/IGBF-2793

            To test:
            Open the mouse genome (M_musculus_Dec_2011)
            Navigate to chr1:52,182,882-52,235,630
            Select the External View tab
            In the dropdown menu, select UCSC (should be UCSC by default)
            Click update (Gene annotations should appear in the External View tab)
            Next, select View > View Region in UCSC Browser (a web browser should open to genome.ucsc.edu)
            Next, select the Data Access tab
            Under Available Data, there should be a folder for UCSC (DAS)
            Click the arrow to see all of the options under UCSC (DAS)
            Select ncbiRefSeqCurated (note that the DAS menu and data may move slowly as they utilize an external resource)
            Once the track has appeared in IGB, click Load Data (gene models should appear)

            Show
            nfreese Nowlan Freese added a comment - Branch: https://bitbucket.org/nfreese/nowlanfork-igb/branch/IGBF-2793 To test: Open the mouse genome (M_musculus_Dec_2011) Navigate to chr1:52,182,882-52,235,630 Select the External View tab In the dropdown menu, select UCSC (should be UCSC by default) Click update (Gene annotations should appear in the External View tab) Next, select View > View Region in UCSC Browser (a web browser should open to genome.ucsc.edu ) Next, select the Data Access tab Under Available Data, there should be a folder for UCSC (DAS) Click the arrow to see all of the options under UCSC (DAS) Select ncbiRefSeqCurated (note that the DAS menu and data may move slowly as they utilize an external resource) Once the track has appeared in IGB, click Load Data (gene models should appear)
            Hide
            nfreese Nowlan Freese added a comment -

            Restoring DASv1 support in IGB fixes the issues with the External View tab (IGBF-2776) and the View Region in UCSC Browser (IGBF-2777), as they both used DAS to determine what genomes were available in UCSC (initUCSCSources() method in UCSCViewAction.java).

            The issue with the DAS files not loading in IGB was due to the XML being returned containing a link to a dead dtd file (http://www.biodas.org/dtd/dasgff.dtd). At some point in the last couple of years the biodas.org website appears to have gone offline. IGB versions prior to 8.4.0 did not validate the XML using dtd and thus had no issues. IGB versions 8.4.0 to 9.1.6 attempt to validate the XML using dtd and because the biodas.org link is no longer active failed to validate causing the DAS data to fail to load in IGB.

            Show
            nfreese Nowlan Freese added a comment - Restoring DASv1 support in IGB fixes the issues with the External View tab ( IGBF-2776 ) and the View Region in UCSC Browser ( IGBF-2777 ), as they both used DAS to determine what genomes were available in UCSC (initUCSCSources() method in UCSCViewAction.java). The issue with the DAS files not loading in IGB was due to the XML being returned containing a link to a dead dtd file ( http://www.biodas.org/dtd/dasgff.dtd ). At some point in the last couple of years the biodas.org website appears to have gone offline. IGB versions prior to 8.4.0 did not validate the XML using dtd and thus had no issues. IGB versions 8.4.0 to 9.1.6 attempt to validate the XML using dtd and because the biodas.org link is no longer active failed to validate causing the DAS data to fail to load in IGB.
            Hide
            nfreese Nowlan Freese added a comment - - edited

            Many files were removed/edited when DASv1 support was removed from IGB. I reverted three commits to restore the functionality, but this required resolving several merge conflicts. In addition, I tried to maintain the removal of DASv2 support as it does not appear to be necessary. However, many of the merge conflicts were due to commenting out both DASv1 and DASv2 at the same time.

            Show
            nfreese Nowlan Freese added a comment - - edited Many files were removed/edited when DASv1 support was removed from IGB. I reverted three commits to restore the functionality, but this required resolving several merge conflicts. In addition, I tried to maintain the removal of DASv2 support as it does not appear to be necessary. However, many of the merge conflicts were due to commenting out both DASv1 and DASv2 at the same time.
            Hide
            nfreese Nowlan Freese added a comment - - edited

            Many of the functions provided by DAS can be accomplished with the newer UCSC Rest API. At some point it would be good to replace the DAS functionality with the REST API, though this may require extensive work.

            Show
            nfreese Nowlan Freese added a comment - - edited Many of the functions provided by DAS can be accomplished with the newer UCSC Rest API . At some point it would be good to replace the DAS functionality with the REST API, though this may require extensive work.
            Hide
            rweidenh Logan Weidenhammer (Inactive) added a comment - - edited

            (reviewed on Windows 10 OS)
            Functional Review:

            Following the steps from the first comment works as instructed.
            I encountered no errors.

            Code Review:

            I don't feel like I am able to understand the changes well enough to review them.
            It looks like the DAS Parser was told not to support DTD, and some other files were added but then removed.

            But thank you for all of the information in the previous comments on this issue. They are very informative and helpful.

            Show
            rweidenh Logan Weidenhammer (Inactive) added a comment - - edited (reviewed on Windows 10 OS) Functional Review: Following the steps from the first comment works as instructed. I encountered no errors. Code Review: I don't feel like I am able to understand the changes well enough to review them. It looks like the DAS Parser was told not to support DTD, and some other files were added but then removed. But thank you for all of the information in the previous comments on this issue. They are very informative and helpful.
            Show
            nfreese Nowlan Freese added a comment - Pull request: https://bitbucket.org/lorainelab/integrated-genome-browser/pull-requests/865/igbf-2793
            Hide
            pbadzuh Philip Badzuh (Inactive) added a comment -

            Testing on macOS following the instructions in the first comment, everything seems to be working as expected. Closing issue.

            Show
            pbadzuh Philip Badzuh (Inactive) added a comment - Testing on macOS following the instructions in the first comment, everything seems to be working as expected. Closing issue.
            Hide
            ann.loraine Ann Loraine added a comment -

            From the Wayback Machine.

            Show
            ann.loraine Ann Loraine added a comment - From the Wayback Machine.
            Hide
            nfreese Nowlan Freese added a comment - - edited

            I reopened this ticket as there was an issue with loading the sequence from genomes available through DAS.

            The issue is that the dtd file - http://www.biodas.org/dtd/dasdna.dtd - is no longer available. I could not find the file online including on the Wayback Machine.

            I have changed the IGB logic to ignore the dtd file when loading residues following logic from here and here.

            Branch: https://bitbucket.org/nfreese/nowlanfork-igb/branch/IGBF-2793

            If we are able to acquire the dtd file, it may be worth implementing a different solution where we use the file internally along with an entity resolver as discussed here and here.

            However, the best solution may be to begin using the UCSC API instead of DAS.

            Show
            nfreese Nowlan Freese added a comment - - edited I reopened this ticket as there was an issue with loading the sequence from genomes available through DAS. The issue is that the dtd file - http://www.biodas.org/dtd/dasdna.dtd - is no longer available. I could not find the file online including on the Wayback Machine. I have changed the IGB logic to ignore the dtd file when loading residues following logic from here and here . Branch: https://bitbucket.org/nfreese/nowlanfork-igb/branch/IGBF-2793 If we are able to acquire the dtd file, it may be worth implementing a different solution where we use the file internally along with an entity resolver as discussed here and here . However, the best solution may be to begin using the UCSC API instead of DAS.
            Hide
            nfreese Nowlan Freese added a comment - - edited

            To test, download IGBF-2793 installer.

            In IGB with UCSC DAS enabled:
            Select the Caenorhabditis brenneri in the Species dropdown
            Select the C_brenneri_Feb_2008 Genome Version
            Navigate to Un:55,401,094-55,401,137
            Click Load Sequence

            The sequence should appear if working correctly.

            Show
            nfreese Nowlan Freese added a comment - - edited To test, download IGBF-2793 installer . In IGB with UCSC DAS enabled: Select the Caenorhabditis brenneri in the Species dropdown Select the C_brenneri_Feb_2008 Genome Version Navigate to Un:55,401,094-55,401,137 Click Load Sequence The sequence should appear if working correctly.
            Hide
            omarne Omkar Marne (Inactive) added a comment -

            The sequence appeared as expected.

            Show
            omarne Omkar Marne (Inactive) added a comment - The sequence appeared as expected.
            Show
            nfreese Nowlan Freese added a comment - Pull request: https://bitbucket.org/lorainelab/integrated-genome-browser/pull-requests/876/igbf-2793-fix-das-load-sequence
            Hide
            ann.loraine Ann Loraine added a comment -

            Merged to master branch and built master branch installers. Ready for testing.

            Show
            ann.loraine Ann Loraine added a comment - Merged to master branch and built master branch installers. Ready for testing.
            Hide
            rweidenh Logan Weidenhammer (Inactive) added a comment -

            Tested on Windows.
            sequence loads.
            UCSC (DAS) appears in the Data Access panel and those files load as well.

            moving to closed.

            Show
            rweidenh Logan Weidenhammer (Inactive) added a comment - Tested on Windows. sequence loads. UCSC (DAS) appears in the Data Access panel and those files load as well. moving to closed.

              People

              • Assignee:
                nfreese Nowlan Freese
                Reporter:
                nfreese Nowlan Freese
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: