Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-4105

Create hub.txt file and genomes.txt file for quickload to track hub conversion

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      As part of quickload to trackhub conversion, we need to generate hub.txt file.
      Sample hub.txt :
      hub IGBQuickLoad
      shortLabel IGB QUICK LOAD
      longLabel Generated from IGB QuickLoad
      genomesFile genomes.txt
      email myEmail@address

      api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt

      Sample genome.txt:
      genome tair9
      trackDb tair9/trackDb.txt
      organism araTha
      defaultPos Chr1:0-30427671
      twoBitPath http://igbquickload.org/quickload/A_thaliana_Jun_2009/A_thaliana_Jun_2009.2bit

      api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/genomes.txt

        Attachments

          Issue Links

            Activity

            uchinta Udaya Chinta (Inactive) created issue -
            uchinta Udaya Chinta (Inactive) made changes -
            Field Original Value New Value
            Epic Link IGBF-1908 [ 17998 ]
            uchinta Udaya Chinta (Inactive) made changes -
            Link This issue relates to IGBF-4090 [ IGBF-4090 ]
            uchinta Udaya Chinta (Inactive) made changes -
            Link This issue relates to IGBF-4104 [ IGBF-4104 ]
            uchinta Udaya Chinta (Inactive) made changes -
            Epic Link IGBF-1908 [ 17998 ] IGBF-2831 [ 19524 ]
            uchinta Udaya Chinta (Inactive) made changes -
            Summary Develop code to convert quickload to trackhubs Create hub.txt file for quickload to track hub conversion
            uchinta Udaya Chinta (Inactive) made changes -
            Description As part of quickload to trackhub conversion, we need to generate hub.txt file.
            Sample hub.txt :
            hub IGBQuickLoad
            shortLabel IGB QUICK LOAD
            longLabel Generated from IGB QuickLoad
            genomesFile genomes.txt
            email myEmail@address

            api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt
            uchinta Udaya Chinta (Inactive) made changes -
            Status To-Do [ 10305 ] In Progress [ 3 ]
            ann.loraine Ann Loraine made changes -
            Sprint Spring 2 [ 211 ] Spring 2, Spring 3 [ 211, 212 ]
            ann.loraine Ann Loraine made changes -
            Rank Ranked higher
            uchinta Udaya Chinta (Inactive) made changes -
            Story Points 2 4
            uchinta Udaya Chinta (Inactive) made changes -
            Summary Create hub.txt file for quickload to track hub conversion Create hub.txt file and genomes.txt file for quickload to track hub conversion
            uchinta Udaya Chinta (Inactive) made changes -
            Description As part of quickload to trackhub conversion, we need to generate hub.txt file.
            Sample hub.txt :
            hub IGBQuickLoad
            shortLabel IGB QUICK LOAD
            longLabel Generated from IGB QuickLoad
            genomesFile genomes.txt
            email myEmail@address

            api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt
            As part of quickload to trackhub conversion, we need to generate hub.txt file.
            Sample hub.txt :
            hub IGBQuickLoad
            shortLabel IGB QUICK LOAD
            longLabel Generated from IGB QuickLoad
            genomesFile genomes.txt
            email myEmail@address

            sample genome.txt


            api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt
            uchinta Udaya Chinta (Inactive) made changes -
            Description As part of quickload to trackhub conversion, we need to generate hub.txt file.
            Sample hub.txt :
            hub IGBQuickLoad
            shortLabel IGB QUICK LOAD
            longLabel Generated from IGB QuickLoad
            genomesFile genomes.txt
            email myEmail@address

            sample genome.txt


            api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt
            As part of quickload to trackhub conversion, we need to generate hub.txt file.
            *Sample hub.txt :*
            hub IGBQuickLoad
            shortLabel IGB QUICK LOAD
            longLabel Generated from IGB QuickLoad
            genomesFile genomes.txt
            email myEmail@address

            *api* : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt

            *Sample genome.txt:*
            genome tair9
            trackDb tair9/trackDb.txt
            organism araTha
            defaultPos Chr1:0-30427671
            twoBitPath http://igbquickload.org/quickload/A_thaliana_Jun_2009/A_thaliana_Jun_2009.2bit

            *api* : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/genomes.txt


            Show
            uchinta Udaya Chinta (Inactive) added a comment - Made changes to generate hub.txt and genomes.txt files. Sample local url paths: hub.txt: http://127.0.0.1:8000/api/igbTrackHubs?quickLoadUrl=http://igbquickload.org/quickload&filePath=/hub.txt genomes.txt: http://127.0.0.1:8000/api/igbTrackHubs?quickLoadUrl=http://igbquickload.org/quickload&filePath=/genomes.txt
            ann.loraine Ann Loraine made changes -
            Sprint Spring 2, Spring 3 [ 211, 212 ] Spring 2, Spring 3, Spring 4 [ 211, 212, 213 ]
            ann.loraine Ann Loraine made changes -
            Rank Ranked higher
            Show
            uchinta Udaya Chinta (Inactive) added a comment - Branch : https://bitbucket.org/lorainelab_udaya/hub-facade/branch/IGBF-4105 Jaya Sravani Sirigineedi , Dr. Nowlan Freese , please verify the commit.
            uchinta Udaya Chinta (Inactive) made changes -
            Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
            uchinta Udaya Chinta (Inactive) made changes -
            Assignee Udaya Chinta [ uchinta ]
            jsirigin Jaya Sravani Sirigineedi (Inactive) made changes -
            Assignee Jaya Sravani Sirigineedi [ jsirigin ]
            jsirigin Jaya Sravani Sirigineedi (Inactive) made changes -
            Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
            Hide
            jsirigin Jaya Sravani Sirigineedi (Inactive) added a comment -

            Reviewed the code and everything looks good, I only have two suggestions: "\n" add this to a string variable delimiter and use that when joining strings to make it more readable and add res.text not null condition as well to the status code check just to avoid any null pointer exceptions. Build the branch locally and ran it, was able to get both files. The content in the files is as expected, optional rows are only present for genomes that are not in UCSC for the genomes that are in UCSC we don't need those fields as those are retrieved from UCSC itself.

            Show
            jsirigin Jaya Sravani Sirigineedi (Inactive) added a comment - Reviewed the code and everything looks good, I only have two suggestions: "\n" add this to a string variable delimiter and use that when joining strings to make it more readable and add res.text not null condition as well to the status code check just to avoid any null pointer exceptions. Build the branch locally and ran it, was able to get both files. The content in the files is as expected, optional rows are only present for genomes that are not in UCSC for the genomes that are in UCSC we don't need those fields as those are retrieved from UCSC itself.
            jsirigin Jaya Sravani Sirigineedi (Inactive) made changes -
            Status First Level Review in Progress [ 10301 ] To-Do [ 10305 ]
            jsirigin Jaya Sravani Sirigineedi (Inactive) made changes -
            Assignee Jaya Sravani Sirigineedi [ jsirigin ] Udaya Chinta [ uchinta ]
            uchinta Udaya Chinta (Inactive) made changes -
            Status To-Do [ 10305 ] In Progress [ 3 ]
            ann.loraine Ann Loraine made changes -
            Sprint Spring 2, Spring 3, Spring 4 [ 211, 212, 213 ] Spring 2, Spring 3, Spring 4, Spring 5 [ 211, 212, 213, 214 ]
            ann.loraine Ann Loraine made changes -
            Rank Ranked higher
            Show
            uchinta Udaya Chinta (Inactive) added a comment - - edited Commit 1: https://bitbucket.org/lorainelab_udaya/hub-facade/commits/e29c464e4cafe65da8717a06ddf170306d19d6dd Created an endpoint that generates synonyms.txt (IGB genome name and UCSC genome name) file by fetching the genomes name from https://api.genome.ucsc.edu/list/ucscGenomes and checks the genome in IGB synonyms.txt api : http://127.0.0.1:8000/api/generateUcscSynonyms Endpoint: http://127.0.0.1:8000/api/generateUcscSynonyms Commit 2: https://bitbucket.org/lorainelab_udaya/hub-facade/commits/919ec25600d592014f7e47d9e287744b79f240a9 changes to generate hub.txt and genomes.txt hub.txt: http://127.0.0.1:8000/api/igbTrackHubs?quickLoadUrl=http://igbquickload.org/quickload&filePath=/hub.txt genomes.txt: http://127.0.0.1:8000/api/igbTrackHubs?quickLoadUrl=http://igbquickload.org/quickload&filePath=/genomes.txt Commit 3: https://bitbucket.org/lorainelab_udaya/hub-facade/commits/2ab4cdb4cf4813959b12586cf2249974a94d1231 changes to generate trackDb.txt trackDb.txt : http://127.0.0.1:8000/api/igbTrackHubs?quickLoadUrl=http://igbquickload.org/quickload&filePath=/ce6/trackDb.txt
            Hide
            uchinta Udaya Chinta (Inactive) added a comment - - edited

            How to test changes :
            Commit 1:

            1. Hit the Ucsc genome endpoint - https://api.genome.ucsc.edu/list/ucscGenomes
            2. Check the ucsc genomes are present in IGB synonyms.txt in IGB repository (path : core/synonym-lookup/src/main/resources/synonyms.txt). If ucsc genomes are present in IGB synonyms.txt, then IGB genome name(which is the first column in IGB synonyms.txt) and Ucsc genome name should be added in new Synonyms.txt in hub_facade repo (path :igb_trackhub/synonyms.txt).

            Commit 2&3:

            Api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt
            Track hub Bitbucket : https://bitbucket.org/lorainelab_udaya/testquickload/src/main/trackHub2/
            Quickload bitbucket : https://bitbucket.org/lorainelab_udaya/testquickload/src/main/quickload1/
            Created the above folder in bitbuket after testing the changes in my local.

            1. Hub.txt file is a static data. Check if the data generated is same as below.
              hub IGBQuickLoad
              shortLabel IGB Quick Load
              longLabel Track hub generated from IGB
              genomesFile genomes.txt
              email igbhelper@gmail.com
            2. Genomes.txt
            • genome - It is either igb genome name or ucsc genome name. The preference is ucsc genome name.
            • trackDb - It is always {genome_name}/trackDb.txt
              The below three fields are present only when the genome name is IGB genome name
              * organism - we get this from IGB species.txt (the first column) by checking with igb genome name.
              * defaultPos - It is always the first row of {genome_name}

              /genome.txt

            • twoBitPath - We get the twoBitPath from name field in {genome_name}/annots.xml whose reference value is true.
              # trackDb.txt - We get this data from {genome_name}

              /annots.xml

            • track - the first word of title field whose reference is not true
            • bigDataUrl - name field whose reference is not true
            • shortLabel - the last word of title field whose reference is not true
            • longLabel - description field whose reference is not true
            • type - type is assigned depending on the extension of the bigDataUrl
            • visibility - It is a fixed value. visibility = "dense"
            1. Check if above fields are generated properly
            2. Copy the hub.txt api provided above and paste in https://genome.ucsc.edu/cgi-bin/hgHubConnect and click add hub
            3. Check if the genomes are visibile in ucsc
            Show
            uchinta Udaya Chinta (Inactive) added a comment - - edited How to test changes : Commit 1: Hit the Ucsc genome endpoint - https://api.genome.ucsc.edu/list/ucscGenomes Check the ucsc genomes are present in IGB synonyms.txt in IGB repository (path : core/synonym-lookup/src/main/resources/synonyms.txt). If ucsc genomes are present in IGB synonyms.txt, then IGB genome name(which is the first column in IGB synonyms.txt) and Ucsc genome name should be added in new Synonyms.txt in hub_facade repo (path :igb_trackhub/synonyms.txt). Commit 2&3: Api : https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/trackHub2/hub.txt Track hub Bitbucket : https://bitbucket.org/lorainelab_udaya/testquickload/src/main/trackHub2/ Quickload bitbucket : https://bitbucket.org/lorainelab_udaya/testquickload/src/main/quickload1/ Created the above folder in bitbuket after testing the changes in my local. Hub.txt file is a static data. Check if the data generated is same as below. hub IGBQuickLoad shortLabel IGB Quick Load longLabel Track hub generated from IGB genomesFile genomes.txt email igbhelper@gmail.com Genomes.txt genome - It is either igb genome name or ucsc genome name. The preference is ucsc genome name. trackDb - It is always {genome_name}/trackDb.txt The below three fields are present only when the genome name is IGB genome name * organism - we get this from IGB species.txt (the first column) by checking with igb genome name. * defaultPos - It is always the first row of {genome_name} /genome.txt twoBitPath - We get the twoBitPath from name field in {genome_name}/annots.xml whose reference value is true. # trackDb.txt - We get this data from {genome_name} /annots.xml track - the first word of title field whose reference is not true bigDataUrl - name field whose reference is not true shortLabel - the last word of title field whose reference is not true longLabel - description field whose reference is not true type - type is assigned depending on the extension of the bigDataUrl visibility - It is a fixed value. visibility = "dense" Check if above fields are generated properly Copy the hub.txt api provided above and paste in https://genome.ucsc.edu/cgi-bin/hgHubConnect and click add hub Check if the genomes are visibile in ucsc
            uchinta Udaya Chinta (Inactive) made changes -
            Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
            Hide
            uchinta Udaya Chinta (Inactive) added a comment -

            Dr. Nowlan Freese and Jaya Sravani Sirigineedi Please verify the changes.

            Show
            uchinta Udaya Chinta (Inactive) added a comment - Dr. Nowlan Freese and Jaya Sravani Sirigineedi Please verify the changes.
            uchinta Udaya Chinta (Inactive) made changes -
            Assignee Udaya Chinta [ uchinta ]
            uchinta Udaya Chinta (Inactive) made changes -
            Link This issue relates to IGBF-4162 [ IGBF-4162 ]
            nfreese Nowlan Freese made changes -
            Assignee Nowlan Freese [ nfreese ]
            ann.loraine Ann Loraine made changes -
            Sprint Spring 2, Spring 3, Spring 4, Spring 5 [ 211, 212, 213, 214 ] Spring 2, Spring 3, Spring 4, Spring 5, Spring 6 [ 211, 212, 213, 214, 215 ]
            ann.loraine Ann Loraine made changes -
            Rank Ranked higher
            nfreese Nowlan Freese made changes -
            Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
            Show
            uchinta Udaya Chinta (Inactive) added a comment - update branch : https://bitbucket.org/lorainelab_udaya/hub-facade/branch/IGBF-4105
            Hide
            nfreese Nowlan Freese added a comment -

            Tested following instructions above. Everything looks to be working. I think we should go ahead and deploy the changes so that we can test the changes on the UCSC genome browser.

            Note that there are currently merge conflicts due to IGBF-4139. Once the conflicts are resolved, ready for PR and deployment.

            Show
            nfreese Nowlan Freese added a comment - Tested following instructions above. Everything looks to be working. I think we should go ahead and deploy the changes so that we can test the changes on the UCSC genome browser. Note that there are currently merge conflicts due to IGBF-4139 . Once the conflicts are resolved, ready for PR and deployment.
            nfreese Nowlan Freese made changes -
            Assignee Nowlan Freese [ nfreese ] Udaya Chinta [ uchinta ]
            nfreese Nowlan Freese made changes -
            Status First Level Review in Progress [ 10301 ] Ready for Pull Request [ 10304 ]
            Hide
            jsirigin Jaya Sravani Sirigineedi (Inactive) added a comment -

            Reviewed the latest code chagnes, everyhting looks good.

            Show
            jsirigin Jaya Sravani Sirigineedi (Inactive) added a comment - Reviewed the latest code chagnes, everyhting looks good.
            Hide
            uchinta Udaya Chinta (Inactive) added a comment -

            PR: https://bitbucket.org/lorainelab/hub-facade/pull-requests/40/overview
            Dr. Ann Loraine, Please review and merge the PR. This ticket can be tested only after deployment.

            Show
            uchinta Udaya Chinta (Inactive) added a comment - PR: https://bitbucket.org/lorainelab/hub-facade/pull-requests/40/overview Dr. Ann Loraine , Please review and merge the PR. This ticket can be tested only after deployment.
            uchinta Udaya Chinta (Inactive) made changes -
            Status Ready for Pull Request [ 10304 ] Pull Request Submitted [ 10101 ]
            uchinta Udaya Chinta (Inactive) made changes -
            Assignee Udaya Chinta [ uchinta ] Ann Loraine [ aloraine ]
            ann.loraine Ann Loraine made changes -
            Status Pull Request Submitted [ 10101 ] Reviewing Pull Request [ 10303 ]
            Hide
            ann.loraine Ann Loraine added a comment -

            PR is merged and new code is deployed to translate.bioviz.org. Ready for testing.

            Show
            ann.loraine Ann Loraine added a comment - PR is merged and new code is deployed to translate.bioviz.org. Ready for testing.
            ann.loraine Ann Loraine made changes -
            Status Reviewing Pull Request [ 10303 ] Merged Needs Testing [ 10002 ]
            ann.loraine Ann Loraine made changes -
            Assignee Ann Loraine [ aloraine ]
            uchinta Udaya Chinta (Inactive) made changes -
            Assignee Nowlan Freese [ nfreese ]
            nfreese Nowlan Freese made changes -
            Status Merged Needs Testing [ 10002 ] Post-merge Testing In Progress [ 10003 ]
            nfreese Nowlan Freese made changes -
            Link This issue relates to IGBF-4192 [ IGBF-4192 ]
            Hide
            nfreese Nowlan Freese added a comment -

            Testing on Chrome:
            The hub.txt appears correct: https://translate.bioviz.org/api/igbTrackHubs?quickLoadUrl=https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/quickload1&filePath=/hub.txt
            The genomes.txt appears correct: https://translate.bioviz.org/api/igbTrackHubs?quickLoadUrl=https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/quickload1&filePath=/genomes.txt

            However, this does not work when I add it as a UCSC Track Hub. I have created IGBF-4192 to further test this.

            As the hub.txt and genomes.txt are being returned, closing ticket.

            Show
            nfreese Nowlan Freese added a comment - Testing on Chrome: The hub.txt appears correct: https://translate.bioviz.org/api/igbTrackHubs?quickLoadUrl=https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/quickload1&filePath=/hub.txt The genomes.txt appears correct: https://translate.bioviz.org/api/igbTrackHubs?quickLoadUrl=https://api.bitbucket.org/2.0/repositories/lorainelab_udaya/testquickload/src/main/quickload1&filePath=/genomes.txt However, this does not work when I add it as a UCSC Track Hub. I have created IGBF-4192 to further test this. As the hub.txt and genomes.txt are being returned, closing ticket.
            nfreese Nowlan Freese made changes -
            Assignee Nowlan Freese [ nfreese ] Udaya Chinta [ uchinta ]
            nfreese Nowlan Freese made changes -
            Resolution Done [ 10000 ]
            Status Post-merge Testing In Progress [ 10003 ] Closed [ 6 ]
            Hide
            ann.loraine Ann Loraine added a comment -

            Latest PR is merged and new code is deployed to translate.bioviz.org as per usual.
            Ready for testing.

            Show
            ann.loraine Ann Loraine added a comment - Latest PR is merged and new code is deployed to translate.bioviz.org as per usual. Ready for testing.

              People

              • Assignee:
                uchinta Udaya Chinta (Inactive)
                Reporter:
                uchinta Udaya Chinta (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: