Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-3640

Add expand color by and Filter by to include the props

    Details

    • Type: Improvement
    • Status: Reviewing Pull Request (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Story Points:
      5
    • Sprint:
      Spring 5, Spring 6, Spring 7, Spring 8, Spring 9, Spring 10

      Description

      Situation: Many of the files provided by the UCSC REST API (see IGBF-3129) include additional coumns (see comments in IGBF-3598). These additional columns are added as "props" and can contain many different types of information.

      In IGB, if you right-click on a track there are options to "Color by..." or "Filter...". Some of the current options include values for Length, RGB, Score, Property. However, the values stored in the "props" are not included.

      Task: Investigate the possibility of enabling a user to use the "Color by..." or "Filter..." on the "props" values.

        Attachments

          Issue Links

            Activity

            nfreese Nowlan Freese created issue -
            nfreese Nowlan Freese made changes -
            Field Original Value New Value
            Epic Link IGBF-1765 [ 17855 ]
            nfreese Nowlan Freese made changes -
            Link This issue relates to IGBF-3598 [ IGBF-3598 ]
            Hide
            nfreese Nowlan Freese added a comment -

            Example:
            Open the most recent chicken genome (G_gallus_Mar_2018).
            Navigate to 1:79,200,818-79,317,139
            Under UCSC REST in the Data Access tab under Available Data, select cpgIslandExt
            Load Data
            Select one of the data points and look at the Selection Info tab, you should see props such as "perCpg"

            For the example above you will either need to be on Sravani's branch, or wait until we merge her changes into the Loraine Lab repo.

            Show
            nfreese Nowlan Freese added a comment - Example: Open the most recent chicken genome (G_gallus_Mar_2018). Navigate to 1:79,200,818-79,317,139 Under UCSC REST in the Data Access tab under Available Data, select cpgIslandExt Load Data Select one of the data points and look at the Selection Info tab, you should see props such as "perCpg" For the example above you will either need to be on Sravani's branch , or wait until we merge her changes into the Loraine Lab repo.
            nfreese Nowlan Freese made changes -
            Status To-Do [ 10305 ] In Progress [ 3 ]
            nfreese Nowlan Freese made changes -
            Status In Progress [ 3 ] To-Do [ 10305 ]
            nfreese Nowlan Freese made changes -
            Status To-Do [ 10305 ] In Progress [ 3 ]
            Hide
            kgopu Kaushik Gopu added a comment - - edited
            • The property filters such as id, name, score, and title available in the Color By panel are assigned during compile time (static variables). For example, see the below code snippet.
              static {
                      PROPERTY_VALUES.add(ID);
                      PROPERTY_VALUES.add(NAME);
                      PROPERTY_VALUES.add(SCORE);
                      PROPERTY_VALUES.add(TITLE);
                  }
              
            • We cannot assign the props at compile time (like the above) if they change from data source to data source. Per the code, the Color By options support three types of file categories: Annotation, Alignment, and ProbSet. Do they all have the same type of props?
            • I am still figuring out if I can extract the props of the selected data source so we can display the props w.r.t data source.
            Show
            kgopu Kaushik Gopu added a comment - - edited The property filters such as id, name, score, and title available in the Color By panel are assigned during compile time (static variables). For example, see the below code snippet. static { PROPERTY_VALUES.add(ID); PROPERTY_VALUES.add(NAME); PROPERTY_VALUES.add(SCORE); PROPERTY_VALUES.add(TITLE); } We cannot assign the props at compile time (like the above) if they change from data source to data source. Per the code, the Color By options support three types of file categories: Annotation, Alignment, and ProbSet. Do they all have the same type of props? I am still figuring out if I can extract the props of the selected data source so we can display the props w.r.t data source.
            Hide
            kgopu Kaushik Gopu added a comment - - edited

            This task can be divided into multiple subtasks:

            1. How to get / where to store the schema of the loaded data in IGB?
              How to get the schema: UCSC provides a rich set of APIs for getting genome data and its metadata(or schema). We can get the schema of a specific genome using /list/schema API. For example, hit this URL to view the schema of cpgIslandExt.
              Where to store the metadata (or schema): The IGB dataset holds several properties, such as Name, Properties, Data Container, and so on (check in DataSet.java). The variable Properties would be the right place to store the metadata (or schema).
            2. will add more details about further steps here.
            Show
            kgopu Kaushik Gopu added a comment - - edited This task can be divided into multiple subtasks: How to get / where to store the schema of the loaded data in IGB? How to get the schema : UCSC provides a rich set of APIs for getting genome data and its metadata(or schema). We can get the schema of a specific genome using /list/schema API. For example, hit this URL to view the schema of cpgIslandExt. Where to store the metadata (or schema): The IGB dataset holds several properties, such as Name, Properties, Data Container, and so on (check in DataSet.java). The variable Properties would be the right place to store the metadata (or schema). will add more details about further steps here.
            ann.loraine Ann Loraine made changes -
            Sprint Spring 5 [ 189 ] Spring 5, Spring 6 [ 189, 190 ]
            ann.loraine Ann Loraine made changes -
            Rank Ranked higher
            kgopu Kaushik Gopu made changes -
            Link This issue blocks IGBF-3660 [ IGBF-3660 ]
            ann.loraine Ann Loraine made changes -
            Sprint Spring 5, Spring 6 [ 189, 190 ] Spring 5, Spring 6, Spring 7 [ 189, 190, 191 ]
            ann.loraine Ann Loraine made changes -
            Rank Ranked higher
            ann.loraine Ann Loraine made changes -
            Sprint Spring 5, Spring 6, Spring 7 [ 189, 190, 191 ] Spring 5, Spring 6, Spring 7, Spring 8 [ 189, 190, 191, 192 ]
            ann.loraine Ann Loraine made changes -
            Rank Ranked higher
            nfreese Nowlan Freese made changes -
            Summary Investigate expand color by and Filter by to include the props Add expand color by and Filter by to include the props
            nfreese Nowlan Freese made changes -
            Link This issue relates to IGBF-3698 [ IGBF-3698 ]
            Hide
            nfreese Nowlan Freese added a comment - - edited

            Kaushik branch: https://bitbucket.org/kaushik-gopu/kgopu_integrated-genome-browser/branch/IGBF-3640

            Filter and Color by are working mostly as I would expect. I found some existing issues, also present in IGB 10.0.0, where a filter can be applied to the negative strand and a warning is thrown (see below). As this is an existing issue, I have created a separate ticket, IGBF-3698, to fix it.

            WARNING: Found a glyph with null info at location java.awt.geom.Rectangle2D$Double[x=0.0,y=53.764827058823535,w=0.0,h=20.0]

            Note that Kaushik's changes also fix an issue with certain comparators (for example >) causing an error to appear in IGB.

            Kaushik Gopu - When using Color By > Property, can we utilize the heatmap option for properties that are numeric? For example, using the galGal6 genome with cpgIslandExt selected, if I select Color By > Property > perCpG it would be nice if the heatmap Edit option would appear.
            UPDATE - Sravani and I discussed this idea. While this idea may be possible, it would require significant refactoring of this code.

            Show
            nfreese Nowlan Freese added a comment - - edited Kaushik branch: https://bitbucket.org/kaushik-gopu/kgopu_integrated-genome-browser/branch/IGBF-3640 Filter and Color by are working mostly as I would expect. I found some existing issues, also present in IGB 10.0.0, where a filter can be applied to the negative strand and a warning is thrown (see below). As this is an existing issue, I have created a separate ticket, IGBF-3698 , to fix it. WARNING: Found a glyph with null info at location java.awt.geom.Rectangle2D$ Double [x=0.0,y=53.764827058823535,w=0.0,h=20.0] Note that Kaushik's changes also fix an issue with certain comparators (for example >) causing an error to appear in IGB. Kaushik Gopu - When using Color By > Property, can we utilize the heatmap option for properties that are numeric? For example, using the galGal6 genome with cpgIslandExt selected, if I select Color By > Property > perCpG it would be nice if the heatmap Edit option would appear. UPDATE - Sravani and I discussed this idea. While this idea may be possible, it would require significant refactoring of this code.
            nfreese Nowlan Freese made changes -
            Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
            nfreese Nowlan Freese made changes -
            Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
            nfreese Nowlan Freese made changes -
            Status First Level Review in Progress [ 10301 ] To-Do [ 10305 ]
            ann.loraine Ann Loraine made changes -
            Sprint Spring 5, Spring 6, Spring 7, Spring 8 [ 189, 190, 191, 192 ] Spring 5, Spring 6, Spring 7, Spring 8, Spring 9 [ 189, 190, 191, 192, 193 ]
            ann.loraine Ann Loraine made changes -
            Rank Ranked higher
            jsirigin Jaya Sravani Sirigineedi made changes -
            Assignee Kaushik Gopu [ kgopu ] Jaya Sravani Sirigineedi [ jsirigin ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Status To-Do [ 10305 ] In Progress [ 3 ]
            Hide
            jsirigin Jaya Sravani Sirigineedi added a comment -

            While going through the code, found out that the tierLabelManager is available in ColorByAction itself so we don't have to send it from the SeqMapView class, which would let us change only few files to add this functionality. Also, while investigating I found an issue with the existing code when the user selects Color by and chooses the property option there are 4 default options available in the dropdown, if the user selects any option other than the pre-selected one, it shows an error, even though user gives correct data. Nowlan Freese or Kaushik Gopu It would be great if anyone of you can test this and let me know if this issue exists for you as well, I already found the solution too.

            Show
            jsirigin Jaya Sravani Sirigineedi added a comment - While going through the code, found out that the tierLabelManager is available in ColorByAction itself so we don't have to send it from the SeqMapView class, which would let us change only few files to add this functionality. Also, while investigating I found an issue with the existing code when the user selects Color by and chooses the property option there are 4 default options available in the dropdown, if the user selects any option other than the pre-selected one, it shows an error, even though user gives correct data. Nowlan Freese or Kaushik Gopu It would be great if anyone of you can test this and let me know if this issue exists for you as well, I already found the solution too.
            Hide
            jsirigin Jaya Sravani Sirigineedi added a comment -

            Development is completed and the props are populated in the properties dropdown for both color by and filter by, need to test one more scenario. once that is done will move the ticket to review. Updated code is at branch: https://bitbucket.org/jaya-sravani/integrated-genome-browser/branch/IGBF-3640

            Show
            jsirigin Jaya Sravani Sirigineedi added a comment - Development is completed and the props are populated in the properties dropdown for both color by and filter by, need to test one more scenario. once that is done will move the ticket to review. Updated code is at branch: https://bitbucket.org/jaya-sravani/integrated-genome-browser/branch/IGBF-3640
            jsirigin Jaya Sravani Sirigineedi made changes -
            jsirigin Jaya Sravani Sirigineedi made changes -
            jsirigin Jaya Sravani Sirigineedi made changes -
            jsirigin Jaya Sravani Sirigineedi made changes -
            Hide
            jsirigin Jaya Sravani Sirigineedi added a comment -

            Merged all the other code changes and did a few fixes. Updated code is available at: https://bitbucket.org/jaya-sravani/integrated-genome-browser/branch/IGBF-3640. Tested the code and it is working as expected. Below are screenshots:
            Selected single track with extra properties:

            Selected single track without extra properties:

            Selected two tracks, both have similar extra properties:

            Selected two tracks, one has similar extra properties and the other one doesn't have:

            To Test:

            1. Build the jar file for the branch.
            2. Start the application in debug mode, select any species, and select any one of the ucsc rest tracks that has extra properties (bed, bigBed, bedDetail, and bigGenePRed track types have extra properties).
            3. Now while loading the empty track, an API call is made and the featureProps are loaded.
            4. Load data for the track and right-click on the track name to see the options menu
            5. Click on Color by option and select property from the dropdown items
            6. Now, in the property dropdown list, you should be able to see the extra properties that are present in that track
            Show
            jsirigin Jaya Sravani Sirigineedi added a comment - Merged all the other code changes and did a few fixes. Updated code is available at: https://bitbucket.org/jaya-sravani/integrated-genome-browser/branch/IGBF-3640 . Tested the code and it is working as expected. Below are screenshots: Selected single track with extra properties: Selected single track without extra properties: Selected two tracks, both have similar extra properties: Selected two tracks, one has similar extra properties and the other one doesn't have: To Test: Build the jar file for the branch. Start the application in debug mode, select any species, and select any one of the ucsc rest tracks that has extra properties (bed, bigBed, bedDetail, and bigGenePRed track types have extra properties). Now while loading the empty track, an API call is made and the featureProps are loaded. Load data for the track and right-click on the track name to see the options menu Click on Color by option and select property from the dropdown items Now, in the property dropdown list, you should be able to see the extra properties that are present in that track
            jsirigin Jaya Sravani Sirigineedi made changes -
            Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Assignee Jaya Sravani Sirigineedi [ jsirigin ]
            nfreese Nowlan Freese made changes -
            Link This issue relates to IGBF-3722 [ IGBF-3722 ]
            Hide
            nfreese Nowlan Freese added a comment -

            Built and tested Sravani's branch locally on my Mac.

            Following the testing above, the additional properties show up for the appropriate files from UCSC REST. I am able to filter and color the data appropriately. If multiple tracks are selected, only the shared/common properties appear.

            Did code review with Sravani.

            Ready for pull request.

            Show
            nfreese Nowlan Freese added a comment - Built and tested Sravani's branch locally on my Mac. Following the testing above, the additional properties show up for the appropriate files from UCSC REST. I am able to filter and color the data appropriately. If multiple tracks are selected, only the shared/common properties appear. Did code review with Sravani. Ready for pull request.
            nfreese Nowlan Freese made changes -
            Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
            nfreese Nowlan Freese made changes -
            Status First Level Review in Progress [ 10301 ] Ready for Pull Request [ 10304 ]
            nfreese Nowlan Freese made changes -
            Assignee Jaya Sravani Sirigineedi [ jsirigin ]
            Hide
            jsirigin Jaya Sravani Sirigineedi added a comment - - edited

            Created a single Pull request for both tickets, https://jira.bioviz.org/browse/IGBF-3660 and https://jira.bioviz.org/browse/IGBF-3640. Nowlan Freese already reviewed it, Kaushik Gopu Please review it and let me know if find any issues. Added the test case fix for windows as well in the commit and added a try catch block as per Kaushik Gopu suggestion. Here is the updated PR: https://bitbucket.org/lorainelab/integrated-genome-browser/pull-requests/1008

            Show
            jsirigin Jaya Sravani Sirigineedi added a comment - - edited Created a single Pull request for both tickets, https://jira.bioviz.org/browse/IGBF-3660 and https://jira.bioviz.org/browse/IGBF-3640 . Nowlan Freese already reviewed it, Kaushik Gopu Please review it and let me know if find any issues. Added the test case fix for windows as well in the commit and added a try catch block as per Kaushik Gopu suggestion. Here is the updated PR: https://bitbucket.org/lorainelab/integrated-genome-browser/pull-requests/1008
            jsirigin Jaya Sravani Sirigineedi made changes -
            Status Ready for Pull Request [ 10304 ] Pull Request Submitted [ 10101 ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Assignee Jaya Sravani Sirigineedi [ jsirigin ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Status Pull Request Submitted [ 10101 ] Reviewing Pull Request [ 10303 ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Status Reviewing Pull Request [ 10303 ] To-Do [ 10305 ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Status To-Do [ 10305 ] In Progress [ 3 ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Assignee Jaya Sravani Sirigineedi [ jsirigin ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Status First Level Review in Progress [ 10301 ] Ready for Pull Request [ 10304 ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Status Ready for Pull Request [ 10304 ] Pull Request Submitted [ 10101 ]
            jsirigin Jaya Sravani Sirigineedi made changes -
            Assignee Jaya Sravani Sirigineedi [ jsirigin ]
            Hide
            kgopu Kaushik Gopu added a comment -

            IGB was built without skipping test cases; all test cases passed on Windows, and no issues occurred during the build process.
            Jaya Sravani Sirigineedi incorporated suggested changes in the code.

            Show
            kgopu Kaushik Gopu added a comment - IGB was built without skipping test cases; all test cases passed on Windows, and no issues occurred during the build process. Jaya Sravani Sirigineedi incorporated suggested changes in the code.
            kgopu Kaushik Gopu made changes -
            Status Pull Request Submitted [ 10101 ] Reviewing Pull Request [ 10303 ]
            kgopu Kaushik Gopu made changes -
            Assignee Kaushik Gopu [ kgopu ]
            kgopu Kaushik Gopu made changes -
            Assignee Kaushik Gopu [ kgopu ]
            ann.loraine Ann Loraine made changes -
            Sprint Spring 5, Spring 6, Spring 7, Spring 8, Spring 9 [ 189, 190, 191, 192, 193 ] Spring 5, Spring 6, Spring 7, Spring 8, Spring 9, Spring 10 [ 189, 190, 191, 192, 193, 194 ]
            ann.loraine Ann Loraine made changes -
            Rank Ranked higher

              People

              • Assignee:
                Unassigned
                Reporter:
                nfreese Nowlan Freese
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: