Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-4105

Create hub.txt file and genomes.txt file for quickload to track hub conversion

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      As part of quickload to trackhub conversion, we need to generate hub.txt file.
      Sample hub.txt :
      hub IGBQuickLoad
      shortLabel IGB QUICK LOAD
      longLabel Generated from IGB QuickLoad
      genomesFile genomes.txt
      email myEmail@address

      api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt

      Sample genome.txt:
      genome tair9
      trackDb tair9/trackDb.txt
      organism araTha
      defaultPos Chr1:0-30427671
      twoBitPath http://igbquickload.org/quickload/A_thaliana_Jun_2009/A_thaliana_Jun_2009.2bit

      api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/genomes.txt

        Attachments

          Issue Links

            Activity

            Show
            uchinta Udaya Chinta (Inactive) added a comment - Made changes to generate hub.txt and genomes.txt files. Sample local url paths: hub.txt: http://127.0.0.1:8000/api/igbTrackHubs?quickLoadUrl=http://igbquickload.org/quickload&filePath=/hub.txt genomes.txt: http://127.0.0.1:8000/api/igbTrackHubs?quickLoadUrl=http://igbquickload.org/quickload&filePath=/genomes.txt
            Show
            uchinta Udaya Chinta (Inactive) added a comment - Branch : https://bitbucket.org/lorainelab_udaya/hub-facade/branch/IGBF-4105 Jaya Sravani Sirigineedi , Dr. Nowlan Freese , please verify the commit.
            Hide
            jsirigin Jaya Sravani Sirigineedi (Inactive) added a comment -

            Reviewed the code and everything looks good, I only have two suggestions: "\n" add this to a string variable delimiter and use that when joining strings to make it more readable and add res.text not null condition as well to the status code check just to avoid any null pointer exceptions. Build the branch locally and ran it, was able to get both files. The content in the files is as expected, optional rows are only present for genomes that are not in UCSC for the genomes that are in UCSC we don't need those fields as those are retrieved from UCSC itself.

            Show
            jsirigin Jaya Sravani Sirigineedi (Inactive) added a comment - Reviewed the code and everything looks good, I only have two suggestions: "\n" add this to a string variable delimiter and use that when joining strings to make it more readable and add res.text not null condition as well to the status code check just to avoid any null pointer exceptions. Build the branch locally and ran it, was able to get both files. The content in the files is as expected, optional rows are only present for genomes that are not in UCSC for the genomes that are in UCSC we don't need those fields as those are retrieved from UCSC itself.
            Show
            uchinta Udaya Chinta (Inactive) added a comment - - edited Commit 1: https://bitbucket.org/lorainelab_udaya/hub-facade/commits/e29c464e4cafe65da8717a06ddf170306d19d6dd Created an endpoint that generates synonyms.txt (IGB genome name and UCSC genome name) file by fetching the genomes name from https://api.genome.ucsc.edu/list/ucscGenomes and checks the genome in IGB synonyms.txt api : http://127.0.0.1:8000/api/generateUcscSynonyms Endpoint: http://127.0.0.1:8000/api/generateUcscSynonyms Commit 2: https://bitbucket.org/lorainelab_udaya/hub-facade/commits/919ec25600d592014f7e47d9e287744b79f240a9 changes to generate hub.txt and genomes.txt hub.txt: http://127.0.0.1:8000/api/igbTrackHubs?quickLoadUrl=http://igbquickload.org/quickload&filePath=/hub.txt genomes.txt: http://127.0.0.1:8000/api/igbTrackHubs?quickLoadUrl=http://igbquickload.org/quickload&filePath=/genomes.txt Commit 3: https://bitbucket.org/lorainelab_udaya/hub-facade/commits/2ab4cdb4cf4813959b12586cf2249974a94d1231 changes to generate trackDb.txt trackDb.txt : http://127.0.0.1:8000/api/igbTrackHubs?quickLoadUrl=http://igbquickload.org/quickload&filePath=/ce6/trackDb.txt
            Hide
            uchinta Udaya Chinta (Inactive) added a comment - - edited

            How to test changes :
            Commit 1:

            1. Hit the Ucsc genome endpoint - https://api.genome.ucsc.edu/list/ucscGenomes
            2. Check the ucsc genomes are present in IGB synonyms.txt in IGB repository (path : core/synonym-lookup/src/main/resources/synonyms.txt). If ucsc genomes are present in IGB synonyms.txt, then IGB genome name(which is the first column in IGB synonyms.txt) and Ucsc genome name should be added in new Synonyms.txt in hub_facade repo (path :igb_trackhub/synonyms.txt).

            Commit 2&3:

            Api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt
            Track hub Bitbucket : https://bitbucket.org/lorainelab_udaya/testquickload/src/main/trackHub2/
            Quickload bitbucket : https://bitbucket.org/lorainelab_udaya/testquickload/src/main/quickload1/
            Created the above folder in bitbuket after testing the changes in my local.

            1. Hub.txt file is a static data. Check if the data generated is same as below.
              hub IGBQuickLoad
              shortLabel IGB Quick Load
              longLabel Track hub generated from IGB
              genomesFile genomes.txt
              email igbhelper@gmail.com
            2. Genomes.txt
            • genome - It is either igb genome name or ucsc genome name. The preference is ucsc genome name.
            • trackDb - It is always {genome_name}/trackDb.txt
              The below three fields are present only when the genome name is IGB genome name
              * organism - we get this from IGB species.txt (the first column) by checking with igb genome name.
              * defaultPos - It is always the first row of {genome_name}

              /genome.txt

            • twoBitPath - We get the twoBitPath from name field in {genome_name}/annots.xml whose reference value is true.
              # trackDb.txt - We get this data from {genome_name}

              /annots.xml

            • track - the first word of title field whose reference is not true
            • bigDataUrl - name field whose reference is not true
            • shortLabel - the last word of title field whose reference is not true
            • longLabel - description field whose reference is not true
            • type - type is assigned depending on the extension of the bigDataUrl
            • visibility - It is a fixed value. visibility = "dense"
            1. Check if above fields are generated properly
            2. Copy the hub.txt api provided above and paste in https://genome.ucsc.edu/cgi-bin/hgHubConnect and click add hub
            3. Check if the genomes are visibile in ucsc
            Show
            uchinta Udaya Chinta (Inactive) added a comment - - edited How to test changes : Commit 1: Hit the Ucsc genome endpoint - https://api.genome.ucsc.edu/list/ucscGenomes Check the ucsc genomes are present in IGB synonyms.txt in IGB repository (path : core/synonym-lookup/src/main/resources/synonyms.txt). If ucsc genomes are present in IGB synonyms.txt, then IGB genome name(which is the first column in IGB synonyms.txt) and Ucsc genome name should be added in new Synonyms.txt in hub_facade repo (path :igb_trackhub/synonyms.txt). Commit 2&3: Api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt Track hub Bitbucket : https://bitbucket.org/lorainelab_udaya/testquickload/src/main/trackHub2/ Quickload bitbucket : https://bitbucket.org/lorainelab_udaya/testquickload/src/main/quickload1/ Created the above folder in bitbuket after testing the changes in my local. Hub.txt file is a static data. Check if the data generated is same as below. hub IGBQuickLoad shortLabel IGB Quick Load longLabel Track hub generated from IGB genomesFile genomes.txt email igbhelper@gmail.com Genomes.txt genome - It is either igb genome name or ucsc genome name. The preference is ucsc genome name. trackDb - It is always {genome_name}/trackDb.txt The below three fields are present only when the genome name is IGB genome name * organism - we get this from IGB species.txt (the first column) by checking with igb genome name. * defaultPos - It is always the first row of {genome_name} /genome.txt twoBitPath - We get the twoBitPath from name field in {genome_name}/annots.xml whose reference value is true. # trackDb.txt - We get this data from {genome_name} /annots.xml track - the first word of title field whose reference is not true bigDataUrl - name field whose reference is not true shortLabel - the last word of title field whose reference is not true longLabel - description field whose reference is not true type - type is assigned depending on the extension of the bigDataUrl visibility - It is a fixed value. visibility = "dense" Check if above fields are generated properly Copy the hub.txt api provided above and paste in https://genome.ucsc.edu/cgi-bin/hgHubConnect and click add hub Check if the genomes are visibile in ucsc
            Hide
            uchinta Udaya Chinta (Inactive) added a comment -

            Dr. Nowlan Freese and Jaya Sravani Sirigineedi Please verify the changes.

            Show
            uchinta Udaya Chinta (Inactive) added a comment - Dr. Nowlan Freese and Jaya Sravani Sirigineedi Please verify the changes.
            Show
            uchinta Udaya Chinta (Inactive) added a comment - update branch : https://bitbucket.org/lorainelab_udaya/hub-facade/branch/IGBF-4105
            Hide
            nfreese Nowlan Freese added a comment -

            Tested following instructions above. Everything looks to be working. I think we should go ahead and deploy the changes so that we can test the changes on the UCSC genome browser.

            Note that there are currently merge conflicts due to IGBF-4139. Once the conflicts are resolved, ready for PR and deployment.

            Show
            nfreese Nowlan Freese added a comment - Tested following instructions above. Everything looks to be working. I think we should go ahead and deploy the changes so that we can test the changes on the UCSC genome browser. Note that there are currently merge conflicts due to IGBF-4139 . Once the conflicts are resolved, ready for PR and deployment.
            Hide
            jsirigin Jaya Sravani Sirigineedi (Inactive) added a comment -

            Reviewed the latest code chagnes, everyhting looks good.

            Show
            jsirigin Jaya Sravani Sirigineedi (Inactive) added a comment - Reviewed the latest code chagnes, everyhting looks good.
            Hide
            uchinta Udaya Chinta (Inactive) added a comment -

            PR: https://bitbucket.org/lorainelab/hub-facade/pull-requests/40/overview
            Dr. Ann Loraine, Please review and merge the PR. This ticket can be tested only after deployment.

            Show
            uchinta Udaya Chinta (Inactive) added a comment - PR: https://bitbucket.org/lorainelab/hub-facade/pull-requests/40/overview Dr. Ann Loraine , Please review and merge the PR. This ticket can be tested only after deployment.
            Hide
            ann.loraine Ann Loraine added a comment -

            PR is merged and new code is deployed to translate.bioviz.org. Ready for testing.

            Show
            ann.loraine Ann Loraine added a comment - PR is merged and new code is deployed to translate.bioviz.org. Ready for testing.
            Hide
            nfreese Nowlan Freese added a comment -

            Testing on Chrome:
            The hub.txt appears correct: https://translate.bioviz.org/api/igbTrackHubs?quickLoadUrl=https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/quickload1&filePath=/hub.txt
            The genomes.txt appears correct: https://translate.bioviz.org/api/igbTrackHubs?quickLoadUrl=https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/quickload1&filePath=/genomes.txt

            However, this does not work when I add it as a UCSC Track Hub. I have created IGBF-4192 to further test this.

            As the hub.txt and genomes.txt are being returned, closing ticket.

            Show
            nfreese Nowlan Freese added a comment - Testing on Chrome: The hub.txt appears correct: https://translate.bioviz.org/api/igbTrackHubs?quickLoadUrl=https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/quickload1&filePath=/hub.txt The genomes.txt appears correct: https://translate.bioviz.org/api/igbTrackHubs?quickLoadUrl=https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/quickload1&filePath=/genomes.txt However, this does not work when I add it as a UCSC Track Hub. I have created IGBF-4192 to further test this. As the hub.txt and genomes.txt are being returned, closing ticket.
            Hide
            ann.loraine Ann Loraine added a comment -

            Latest PR is merged and new code is deployed to translate.bioviz.org as per usual.
            Ready for testing.

            Show
            ann.loraine Ann Loraine added a comment - Latest PR is merged and new code is deployed to translate.bioviz.org as per usual. Ready for testing.

              People

              • Assignee:
                uchinta Udaya Chinta (Inactive)
                Reporter:
                uchinta Udaya Chinta (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: