Investigate: ENCODE RNA track hub does not appear after added to IGB

Some problem occurs with other track hubs, including GTEx Analysis Hub.

Please see the PR that fixes this issue here. I have tested both ENCODE RNA and GTEx Analysis trackhubs. All others that I have tested seem to be working as well, however, my testing has not been extensive, so more could be beneficial.

To clarify, the changes at IGBF-2944 are required for the ones made for this ticket to work properly.

Comment on branch: https://bitbucket.org/pbadzuh/igb_pbdev/branch/IGBF-2944#diff

Please look into more effective use of URL object methods, such as the "getProtocol" method.

Read: https://www.baeldung.com/java-url

It's a bit weird how the code in "QuickloadUtils.java" creates a URL, which has a "getProtocol" method, but then uses the String strAnnotsURL and its "startsWith" method to determine the protocol. This is a bit ugly and should be fixed. Now is a good time to fix this, I think!

Also, we need to increment the IGB version number before making new changes to the IGB code base itself (see IGBF-2936 "Update main branch to 9.1.10").

I agree. I have updated my branch with the requested change. Please see it here.

A couple of questions about this proposed line of code in SymLoader.java in IGB:

+        extension = extension.split("\\?")[0]; // strip off potential query params

• What happens if the file name contains a "?" character?
• More importantly, I'm trying to understand why the URL for a file would contain query parameters downstream of the file extension.

My understanding (and this could be wrong, of course!) is that the annots endpoint's "file" tags contain "name" parameters that terminate the file extension. Are some of the URLs provided by trackhub endpoints violating this assumption?

We can discuss above issue on Wednesday and then summarize the discussion in a new comment.
Thank you Philip Badzuh!

The ENCODE RNA data source includes data files with names such as:

https://www.encodeproject.org/files/ENCFF949MYP/@@download/ENCFF949MYP.bigBed?proxy=true

This interferes with IGB's ability to determine the file type.

I have updated IGB-related changes at IGBF-2944 and can submit a PR for the facade changes associated with this ticket.

Please submit PR for the track hub facade codebase, for branch: https://bitbucket.org/pbadzuh/hub-api_pbdev/branch/IGBF-2910#diff

Please see PR here.

PR is merged. Moving back to "To-Do" pending update of IGB code base to use 9.1.10 version string instead of 9.1.8.

IGB code base is updated to 9.1.10.
Philip Badzuh - please rebase and submit PR.

PR for IGBF-2944 submitted.

PRs are all merged and the new track hub facade back end code is deployed to the "back end" site used by BioViz.org. You can test the track hub functionality by visiting the main bioviz.org Web site and clicking the link in the middle part of the end of the page. From there, you can convert sites and try them out in the new IGB version.

Philip Badzuh - the server for https://api.wenglab.org/screen_v13/hubs/rna22/hub.txt seems to be down. Is there another example I can use for testing?

Seems to be up now.

In case it goes down again, to test the first commit any converted trackhub URL that has an unsecured/http protocol can be used e.g. this one.

The issue was that the annots.xml validation was not very robust - the presence of 'https' or 'http' was checked for in the data source URL, however, since multiple protocols are present in converted trackhub quickloads, this led to issues. Now the first and actual protocol of the data source is being checked.

Regarding the second commit, I am not sure if there are any other data files in any trackhub's annots.xml that have query parameters. You may have to catch the ENCODE RNA server being up.

I was able to test using the URL: https://translate.bioviz.org/rest_api/?hubUrl=https://api.wenglab.org/screen_v13/hubs/rna22/hub.txt&fileName=/

Note that there is an issue where the protocol is not added to the translated track hub webpage (for example the url is missing https://). This prevents IGB from correctly adding the Quickload site. There appears to be some code to handle this on bioviz.org/trackhubs, but it appears to only work for localhost.

I was able to see the quickload folder for hg38 and mm10 as well as load data for both. I was unable to see the quickload folder for hg19. When I tried to reload the quickload the translate.bioviz.org server appeared to go down.

I am going to close this issue as the primary issues have been solved.