Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-3503 Implement UCSC REST API logic in IGB
  3. IGBF-3506

Implement an API to get the available datasets for a selected genome

    Details

    • Type: Sub-task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 10.1.0
    • Labels:
      None
    • Story Points:
      2
    • Sprint:
      Fall 7, Fall 8, Spring 1, Spring 2, Spring 3

      Description

      Develop code to connect to the https://api.genome.ucsc.edu/list/tracks?genome=hg38 API and get the available datasets for a selected genome

        Attachments

          Activity

          jsirigin Jaya Sravani Sirigineedi created issue -
          jsirigin Jaya Sravani Sirigineedi made changes -
          Field Original Value New Value
          Status To-Do [ 10305 ] In Progress [ 3 ]
          jsirigin Jaya Sravani Sirigineedi made changes -
          Assignee Jaya Sravani Sirigineedi [ jsirigin ]
          jsirigin Jaya Sravani Sirigineedi made changes -
          Status In Progress [ 3 ] To-Do [ 10305 ]
          jsirigin Jaya Sravani Sirigineedi made changes -
          Status To-Do [ 10305 ] In Progress [ 3 ]
          Hide
          jsirigin Jaya Sravani Sirigineedi added a comment -

          To get the available datasets for a genome use https://api.genome.ucsc.edu/list/tracks?genome=hg38 this API and implement a method similar to getAvailableDataSets(). Creating module classes from the JSON Response of the API.

          Show
          jsirigin Jaya Sravani Sirigineedi added a comment - To get the available datasets for a genome use https://api.genome.ucsc.edu/list/tracks?genome=hg38 this API and implement a method similar to getAvailableDataSets(). Creating module classes from the JSON Response of the API.
          Hide
          jsirigin Jaya Sravani Sirigineedi added a comment -

          Checking the get track data endpoint to investigate how to create the parser as it returns different JSON formats for different file types. Also, while checking this endpoint https://api.genome.ucsc.edu/list/tracks?genome=ailMel1 which returns available tracks/datasets for selected genome, found that there are some child nodes for a few datasets, which can be removed from the response by using this parameter trackLeavesOnly=1, also few dataset names have different genome name specified in it, need to connect with Nowlan Freese on this and clarify my doubts on how to parse this API.

          Show
          jsirigin Jaya Sravani Sirigineedi added a comment - Checking the get track data endpoint to investigate how to create the parser as it returns different JSON formats for different file types. Also, while checking this endpoint https://api.genome.ucsc.edu/list/tracks?genome=ailMel1 which returns available tracks/datasets for selected genome, found that there are some child nodes for a few datasets, which can be removed from the response by using this parameter trackLeavesOnly=1, also few dataset names have different genome name specified in it, need to connect with Nowlan Freese on this and clarify my doubts on how to parse this API.
          Hide
          jsirigin Jaya Sravani Sirigineedi added a comment - - edited

          As discussed with Nowlan Freese, will be working on only the genePred (refGene) file type first, after testing it will be moving to the other file types. Currently, working on fixing an error that I got while testing the getAvailableDatasets API implementation. Below is the error log:

          10:12:15.661 [main] ERROR com.affymetrix.main.OSGiHandler - Unable to resolve ucsc-rest-api-service [82](R 82.0): missing requirement [ucsc-rest-api-service [82](R 82.0)] osgi.wiring.package; (osgi.wiring.package=org.apache.http.client.utils) Unresolved requirements: [[ucsc-rest-api-service [82](R 82.0)] osgi.wiring.package; (osgi.wiring.package=org.apache.http.client.utils)]
          org.osgi.framework.BundleException: Unable to resolve ucsc-rest-api-service [82](R 82.0): missing requirement [ucsc-rest-api-service [82](R 82.0)] osgi.wiring.package; (osgi.wiring.package=org.apache.http.client.utils) Unresolved requirements: [[ucsc-rest-api-service [82](R 82.0)] osgi.wiring.package; (osgi.wiring.package=org.apache.http.client.utils)]

          Show
          jsirigin Jaya Sravani Sirigineedi added a comment - - edited As discussed with Nowlan Freese , will be working on only the genePred (refGene) file type first, after testing it will be moving to the other file types. Currently, working on fixing an error that I got while testing the getAvailableDatasets API implementation. Below is the error log: 10:12:15.661 [main] ERROR com.affymetrix.main.OSGiHandler - Unable to resolve ucsc-rest-api-service [82] (R 82.0): missing requirement [ucsc-rest-api-service [82] (R 82.0)] osgi.wiring.package; (osgi.wiring.package=org.apache.http.client.utils) Unresolved requirements: [[ucsc-rest-api-service [82] (R 82.0)] osgi.wiring.package; (osgi.wiring.package=org.apache.http.client.utils)] org.osgi.framework.BundleException: Unable to resolve ucsc-rest-api-service [82] (R 82.0): missing requirement [ucsc-rest-api-service [82] (R 82.0)] osgi.wiring.package; (osgi.wiring.package=org.apache.http.client.utils) Unresolved requirements: [[ucsc-rest-api-service [82] (R 82.0)] osgi.wiring.package; (osgi.wiring.package=org.apache.http.client.utils)]
          Hide
          jsirigin Jaya Sravani Sirigineedi added a comment -

          Added the dependencies in the shared-lib-wrapper which resolved the issue and resolved another issue by connecting with Dr.Freese. Working on the JSON mapping issue right now.

          Show
          jsirigin Jaya Sravani Sirigineedi added a comment - Added the dependencies in the shared-lib-wrapper which resolved the issue and resolved another issue by connecting with Dr.Freese. Working on the JSON mapping issue right now.
          Hide
          jsirigin Jaya Sravani Sirigineedi added a comment -

          The issue I faced while doing the JSON mapping is that the JSON returned from the API https://api.genome.ucsc.edu/list/tracks?genome=hg38;trackLeavesOnly=1 has a dynamic key for the genome name which has the tracks (database info) as a sub-element in it. As the key name changes dynamically it's not possible to directly map the JSON object to a Java class, so I have used a setter method in the class to parse the JSON for the tracks alone separately by getting the JSON object using the genomeVersionName and initialize the tracks. Also, added a filter here in the tracks to only include genePred datasets/tracks as currently we will be implementing that alone and the API returns a lot of tracks with various types.

          Show
          jsirigin Jaya Sravani Sirigineedi added a comment - The issue I faced while doing the JSON mapping is that the JSON returned from the API https://api.genome.ucsc.edu/list/tracks?genome=hg38;trackLeavesOnly=1 has a dynamic key for the genome name which has the tracks (database info) as a sub-element in it. As the key name changes dynamically it's not possible to directly map the JSON object to a Java class, so I have used a setter method in the class to parse the JSON for the tracks alone separately by getting the JSON object using the genomeVersionName and initialize the tracks. Also, added a filter here in the tracks to only include genePred datasets/tracks as currently we will be implementing that alone and the API returns a lot of tracks with various types.
          jsirigin Jaya Sravani Sirigineedi made changes -
          Hide
          jsirigin Jaya Sravani Sirigineedi added a comment - - edited

          Tested the API implementation, working as expected. Available tracks (or datasets) are being loaded on application start. Code is available on the branch https://bitbucket.org/jaya-sravani/integrated-genome-browser/branch/IGBF-3501.

          To test:

          1. Download https://bitbucket.org/jaya-sravani/integrated-genome-browser/downloads/ installer or clone this branch to local https://bitbucket.org/jaya-sravani/integrated-genome-browser/branch/IGBF-3501 and start the application.
          2. Select a particular genome.
          3. See whether UCSC (Rest) datasets are visible in the Available data panel.
          Show
          jsirigin Jaya Sravani Sirigineedi added a comment - - edited Tested the API implementation, working as expected. Available tracks (or datasets) are being loaded on application start. Code is available on the branch https://bitbucket.org/jaya-sravani/integrated-genome-browser/branch/IGBF-3501 . To test: Download https://bitbucket.org/jaya-sravani/integrated-genome-browser/downloads/ installer or clone this branch to local https://bitbucket.org/jaya-sravani/integrated-genome-browser/branch/IGBF-3501 and start the application. Select a particular genome. See whether UCSC (Rest) datasets are visible in the Available data panel.
          jsirigin Jaya Sravani Sirigineedi made changes -
          Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
          jsirigin Jaya Sravani Sirigineedi made changes -
          Assignee Jaya Sravani Sirigineedi [ jsirigin ]
          nfreese Nowlan Freese made changes -
          Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
          nfreese Nowlan Freese made changes -
          Assignee Nowlan Freese [ nfreese ]
          Hide
          nfreese Nowlan Freese added a comment -

          Testing on Mac:

          1. Cloned Sravani's IGBF-3501 branch and started IGB.
          2. Selected the galGal6 genome.
          3. The UCSC (UCSC REST) datasets are visible in the Available Data window (augustusGene, ncbiRefSeqPredicted, ncbiRefSeqCurated, ncbiRefSeq)
          4. There are no errors in the IGB logs.
            Note -> when I click Load Data no data are loaded. I assume this is because the data loading part has not been wired/implemented yet.
          Show
          nfreese Nowlan Freese added a comment - Testing on Mac: Cloned Sravani's IGBF-3501 branch and started IGB. Selected the galGal6 genome. The UCSC (UCSC REST) datasets are visible in the Available Data window (augustusGene, ncbiRefSeqPredicted, ncbiRefSeqCurated, ncbiRefSeq) There are no errors in the IGB logs. Note -> when I click Load Data no data are loaded. I assume this is because the data loading part has not been wired/implemented yet.
          nfreese Nowlan Freese made changes -
          Status First Level Review in Progress [ 10301 ] Ready for Pull Request [ 10304 ]
          nfreese Nowlan Freese made changes -
          Assignee Nowlan Freese [ nfreese ]
          jsirigin Jaya Sravani Sirigineedi made changes -
          Assignee Jaya Sravani Sirigineedi [ jsirigin ]
          jsirigin Jaya Sravani Sirigineedi made changes -
          Status Ready for Pull Request [ 10304 ] Pull Request Submitted [ 10101 ]
          jsirigin Jaya Sravani Sirigineedi made changes -
          Status Pull Request Submitted [ 10101 ] Reviewing Pull Request [ 10303 ]
          jsirigin Jaya Sravani Sirigineedi made changes -
          Status Reviewing Pull Request [ 10303 ] Merged Needs Testing [ 10002 ]
          jsirigin Jaya Sravani Sirigineedi made changes -
          Status Merged Needs Testing [ 10002 ] Post-merge Testing In Progress [ 10003 ]
          jsirigin Jaya Sravani Sirigineedi made changes -
          Resolution Done [ 10000 ]
          Status Post-merge Testing In Progress [ 10003 ] Closed [ 6 ]
          Hide
          jsirigin Jaya Sravani Sirigineedi added a comment -

          As discussed, closing this ticket because the data loading part is not implemented yet, will be done as part of this https://jira.bioviz.org/browse/IGBF-3528 subtask.

          Show
          jsirigin Jaya Sravani Sirigineedi added a comment - As discussed, closing this ticket because the data loading part is not implemented yet, will be done as part of this https://jira.bioviz.org/browse/IGBF-3528 subtask.
          nfreese Nowlan Freese made changes -
          Story Points 2
          nfreese Nowlan Freese made changes -
          Fix Version/s 10.1.0 [ 11000 ]

            People

            • Assignee:
              jsirigin Jaya Sravani Sirigineedi
              Reporter:
              jsirigin Jaya Sravani Sirigineedi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: