Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-1220

Ensure that IGB is loading sequences according to load priority

    Details

    • Sprint:
      B - Summer 2018

      Description

      Ensure that IGB is loading sequences appropriately according to load priority.

      Ann encountered an error when she tried to load a genome after moving the IGB Quickload data provider down on the priority list.

      Find out how IGB determines where to pull the genomic sequence from, namely which data provider to use. This should be based on the data provider's load priority. If there is only one data provider that has the sequence, then it should not matter what the load priority is. IGB should retrieve the sequence without troubling the user with errors.

        Attachments

          Issue Links

            Activity

            Hide
            ieclabau Ivory Blakley (Inactive) added a comment - - edited

            Some notes on load priority:

            Data providers have a load priority attribute given in igbDefaultPrefs.json.
            This load priority is saved to user preferences.
            The order of the data providers in preferences does not match the json file, or the load priority. Looks random?

            I tried moving SoySeq node (load priority 4) up within the json file. The order of data providers in the configure window matched the load priority, not the order in the json file.
            I moved the Blueberry dp up in the configure window (to the first position). In the saved user preferences, its load priority is now 0 (in the json file it was 3). IGB quickload (load priority of 0 in the json file) has a load priority of 1. Load priority of RNA-Seq is 1 in json, and its now 3 in prefs. SoySeq is 4 in prefs, and 4 in the json file. There is no dp with load priority of 2 in the json file, or the prefs file.

            Show
            ieclabau Ivory Blakley (Inactive) added a comment - - edited Some notes on load priority: Data providers have a load priority attribute given in igbDefaultPrefs.json. This load priority is saved to user preferences. The order of the data providers in preferences does not match the json file, or the load priority. Looks random? I tried moving SoySeq node (load priority 4) up within the json file. The order of data providers in the configure window matched the load priority, not the order in the json file. I moved the Blueberry dp up in the configure window (to the first position). In the saved user preferences, its load priority is now 0 (in the json file it was 3). IGB quickload (load priority of 0 in the json file) has a load priority of 1. Load priority of RNA-Seq is 1 in json, and its now 3 in prefs. SoySeq is 4 in prefs, and 4 in the json file. There is no dp with load priority of 2 in the json file, or the prefs file.
            Hide
            ieclabau Ivory Blakley (Inactive) added a comment - - edited

            I created a set of local quickloads to test with. Each has a 2bit genome sequence. One matches the correct A thaliana sequence, one has a bunch of bases in the first chromosome changed to A, and one has a bunch of bases changed to G.

            In adding these, and changing their load priority order I found that:

            1 - I think IGB is getting the genome sequence from the correct source based on load priority. So this issue can close.

            2 - All new data providers are added with load priority of 1, even if there is already a -1. This a wrench in the works for changing the load priorities. This should be made into a separate issue-see IGBF-1225.

            And we need to think on how to handle the version transition for load priorities. If we want to respect user assigned load priorities, but we change the defaults, what should the outcome look like?

            3 - There is no 3rd point. At one point, there was----I just made a mistake.

            For a detailed record of my tests, see the attached text file: Testing_IGBF-1220.txt

            Show
            ieclabau Ivory Blakley (Inactive) added a comment - - edited I created a set of local quickloads to test with. Each has a 2bit genome sequence. One matches the correct A thaliana sequence, one has a bunch of bases in the first chromosome changed to A, and one has a bunch of bases changed to G. In adding these, and changing their load priority order I found that: 1 - I think IGB is getting the genome sequence from the correct source based on load priority. So this issue can close. 2 - All new data providers are added with load priority of 1, even if there is already a -1. This a wrench in the works for changing the load priorities. This should be made into a separate issue -see IGBF-1225 . And we need to think on how to handle the version transition for load priorities. If we want to respect user assigned load priorities, but we change the defaults, what should the outcome look like? 3 - There is no 3rd point. At one point, there was----I just made a mistake. For a detailed record of my tests, see the attached text file: Testing_ IGBF-1220 .txt
            Hide
            ieclabau Ivory Blakley (Inactive) added a comment - - edited

            Sneha found that the lack of any data provider with load priority 2 was the cause of some of the odd behavior we were seeing. When we talked, we decided that we would need to find out what code was tripping up on non-consecutive load priorities. If there was a valid reason, then we should ensure (through clear documentation and through tests) that there are always consecutive load priority values. If there is no good reason to trip up, we should re-write the code to tolerate gaps in the load priority values.

            We should also review this now that we know all new DP are assigned load priority of -1 to make sure these aren't different interpretations of the same underlying issue.

            ---> We discussed this and we think the lack of a '2' does not matter after all.

            Show
            ieclabau Ivory Blakley (Inactive) added a comment - - edited Sneha found that the lack of any data provider with load priority 2 was the cause of some of the odd behavior we were seeing. When we talked, we decided that we would need to find out what code was tripping up on non-consecutive load priorities. If there was a valid reason, then we should ensure (through clear documentation and through tests) that there are always consecutive load priority values. If there is no good reason to trip up, we should re-write the code to tolerate gaps in the load priority values. We should also review this now that we know all new DP are assigned load priority of -1 to make sure these aren't different interpretations of the same underlying issue. ---> We discussed this and we think the lack of a '2' does not matter after all.
            Hide
            ieclabau Ivory Blakley (Inactive) added a comment -

            At this point, I think this issue is done. IGB is taking the genomic sequence from the correct data provider based on load priority. No code change needed.

            However, there are related issues that need to be addressed, namely IGBF-1225.

            IGB passed up a data provider that had a fasta sequence file to go to one that had a 2bit. We may want to examine that more and potentially make an issue for it.

            I am reassigning this to Ann.
            She can choose to close the issue or to expand it to require more testing/documenting/coding.

            Show
            ieclabau Ivory Blakley (Inactive) added a comment - At this point, I think this issue is done. IGB is taking the genomic sequence from the correct data provider based on load priority. No code change needed. However, there are related issues that need to be addressed, namely IGBF-1225 . IGB passed up a data provider that had a fasta sequence file to go to one that had a 2bit. We may want to examine that more and potentially make an issue for it. I am reassigning this to Ann. She can choose to close the issue or to expand it to require more testing/documenting/coding.
            Hide
            ann.loraine Ann Loraine added a comment -

            I added a couple comments to the latest commit. Please take a look and let me know what you think. Also, I'm in the lab right now, so feel free to stop by if you'd like to talk in person.

            Show
            ann.loraine Ann Loraine added a comment - I added a couple comments to the latest commit. Please take a look and let me know what you think. Also, I'm in the lab right now, so feel free to stop by if you'd like to talk in person.

              People

              • Assignee:
                sneha Sneha Ramesh Watharkar (Inactive)
                Reporter:
                sneha Sneha Ramesh Watharkar (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: