Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2099

Fix: "works_with" field not getting set (always 9.1)

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Story Points:
      1
    • Sprint:
      Fall 6 : 28 Oct to 8 Nov, Fall 7 : 11 Nov to 22 Nov, Fall 8 : 25 Nov to 6 Dec

      Description

      When a user uploads a new App, it gets placed in the "pending apps" table in the database, which has a field "works_with".

      This field is meant to contain a string indicating which version or version range of the IGB platform the App requires.

      Currently, this value is being set to a default value of 9.1.0, the upcoming release of IGB.

      Instead, this value should be set using the IGB platform version indicated in the App's repository.xml and MANIFEST.MF files. (App repository.xml and App MANIFEST.MF should contain the same information regarding imported dependencies.)

      Potentially useful suggestions and comments:

      • Seems like the repository.xml file would be much easier to parse because the data are more structured. Since we now require the repository.xml file to be included in the uploaded jar file, let's parse repository.xml to get the platform versions. Why not?
      • For now, let's use the version syntax word-for-word as provided by the repository.xml file, e.g., [10.0.0,11.0.0) or whatever. We will report that to the user as-is. It's not very user-friendly, but I have a feeling biologists and users will "get" what it means. And later on we can come up with a more user-friendly format. For now, we just need to report the platform version to the user – and to us, for testing.
      • We can use the repository.xml "require" tag with "name" equal to "package" to identify IGB platform dependencies.

      Probably most packages that an IGB App can legally import from the platform has package structure of org.lorainelab.igb.*.

      So we could maybe write a regex that would extract the version strings for every imported org.lorainelab.igb package used by an App.

      We should look at all of them, and require that all such packages use the same version strings.

        Attachments

          Issue Links

            Activity

            Hide
            noor91zahara Noor Zahara (Inactive) added a comment - - edited

            Is there a case where the versions of org.lorainelab.igb packages might be different from one another? Is it necessary to look into all the packages?

            Code changes = https://bitbucket.org/nzahara/appstore-local/branch/IGBF-2099#diff

            Show
            noor91zahara Noor Zahara (Inactive) added a comment - - edited Is there a case where the versions of org.lorainelab.igb packages might be different from one another? Is it necessary to look into all the packages? Code changes = https://bitbucket.org/nzahara/appstore-local/branch/IGBF-2099#diff
            Hide
            ann.loraine Ann Loraine added a comment -

            IGB platform released by us will always have the same bundle versions as eachother.

            Show
            ann.loraine Ann Loraine added a comment - IGB platform released by us will always have the same bundle versions as eachother.
            Hide
            noor91zahara Noor Zahara (Inactive) added a comment -

            [~aloraine] - Kindly confirm if the attached screenshot looks fine or should the IGB version format be changed?

            Show
            noor91zahara Noor Zahara (Inactive) added a comment - [~aloraine] - Kindly confirm if the attached screenshot looks fine or should the IGB version format be changed?
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            If the import statements for platform bundles specify [9.1.0,10,0.0], then change the user-facing text to look like:

            "Works with IGB [9.1.0,10.0.0)"

            Please:

            • Omit the word "version" here – I think it's clear from the context that we're talking about a version.
            • Indicate the allowed range of versions. I can't think of a better way to this than the syntax [9.1.0,10.0.0). I've explained what this means dozens of times in many different settings. People seem to instantly know what it means just from looking at the string. So I think it will work fine to communicate the basic idea. If I'm wrong this about, we will find out from usability testing.
            • Make sure that version strings that don't indicate a range are also supported. Please review: https://jira.transvar.org/browse/IGBF-1906

            I recommend:

            • Store the version string from the XML itself, along with platform major, minor, and micro (patch) versions extracted from same.
            • Store the parsed platform version upper bound major, minor, micro numbers and lower bound major, minor, and micro numbers
            • If a developer does not specify an upper bound (it's possible) then leave these fields as null values
            • Lower bound should be specified with no defaults. The code that processes the bundle and prepares it for release should warn the developer and reject the bundle if we can't figure out what lower bound version the App depends on
            • Handle cases where the App developers does not indicate an upper bound. In that case, we assume the range is open-ended. Indicate that with lower-bound+, e.g., "Works with IGB 9.1.0+"
            Show
            ann.loraine Ann Loraine added a comment - - edited If the import statements for platform bundles specify [9.1.0,10,0.0] , then change the user-facing text to look like: "Works with IGB [9.1.0,10.0.0)" Please: Omit the word "version" here – I think it's clear from the context that we're talking about a version. Indicate the allowed range of versions. I can't think of a better way to this than the syntax [9.1.0,10.0.0). I've explained what this means dozens of times in many different settings. People seem to instantly know what it means just from looking at the string. So I think it will work fine to communicate the basic idea. If I'm wrong this about, we will find out from usability testing. Make sure that version strings that don't indicate a range are also supported. Please review: https://jira.transvar.org/browse/IGBF-1906 I recommend: Store the version string from the XML itself, along with platform major, minor, and micro (patch) versions extracted from same. Store the parsed platform version upper bound major, minor, micro numbers and lower bound major, minor, and micro numbers If a developer does not specify an upper bound (it's possible) then leave these fields as null values Lower bound should be specified with no defaults. The code that processes the bundle and prepares it for release should warn the developer and reject the bundle if we can't figure out what lower bound version the App depends on Handle cases where the App developers does not indicate an upper bound. In that case, we assume the range is open-ended. Indicate that with lower-bound+, e.g., "Works with IGB 9.1.0+"
            Hide
            noor91zahara Noor Zahara (Inactive) added a comment - - edited

            I have made the changes as per the above comment.
            Kindly check the screenshots to view the changes

            Code changes - https://bitbucket.org/nzahara/appstore-local/branch/IGBF-2099#diff

            Show
            noor91zahara Noor Zahara (Inactive) added a comment - - edited I have made the changes as per the above comment. Kindly check the screenshots to view the changes Code changes - https://bitbucket.org/nzahara/appstore-local/branch/IGBF-2099#diff
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Scope changes from discussion earlier today:

            • Please change field named "works_with" to "platform_compatibility"
            • Please save the compatibility string exactly as reported in the XML. (Not sure if this already been done?)

            For example, if the string is: "[9.1.0,10.0.0]" then save exactly that.
            Or, if the string is "9.1.0", then save exactly that.

            • Within the app's page itself, simply extract this string from and show it as is.
            • (minor change: change "verison" to "version")
            Show
            ann.loraine Ann Loraine added a comment - - edited Scope changes from discussion earlier today: Please change field named "works_with" to "platform_compatibility" Please save the compatibility string exactly as reported in the XML. (Not sure if this already been done?) For example, if the string is: " [9.1.0,10.0.0] " then save exactly that. Or, if the string is "9.1.0", then save exactly that. Within the app's page itself, simply extract this string from and show it as is. (minor change: change "verison" to "version")
            Hide
            ann.loraine Ann Loraine added a comment -

            Please review IGB packages. Unfortunately, there are some that do not match the pattern in the regular expression being used.

            To avoid increasing the scope much more, I think we should modify the _create_pending in apps/views.py code as follows:

            First assemble a list of all artifact id's in IGB (see top-level pom for these).
            Then, build a set of regular expressions that can match each of these.
            Modify the views.py _create_pending code to extract the platform compatibility string for each platform artifactId. Get the version compatibility string for each platform bundle.
            Add code to check that all of these are the same. If some are different, use the Django logging system to add a warning to the log.
            Pick the version compatibility string that appears most often and use that.

            We will build a warning to the user later on as part of different ticket – possibly after the refactoring of App and Release models in apps/apps/models.py.

            Show
            ann.loraine Ann Loraine added a comment - Please review IGB packages. Unfortunately, there are some that do not match the pattern in the regular expression being used. To avoid increasing the scope much more, I think we should modify the _create_pending in apps/views.py code as follows: First assemble a list of all artifact id's in IGB (see top-level pom for these). Then, build a set of regular expressions that can match each of these. Modify the views.py _create_pending code to extract the platform compatibility string for each platform artifactId. Get the version compatibility string for each platform bundle. Add code to check that all of these are the same. If some are different, use the Django logging system to add a warning to the log. Pick the version compatibility string that appears most often and use that. We will build a warning to the user later on as part of different ticket – possibly after the refactoring of App and Release models in apps/apps/models.py.
            Hide
            noor91zahara Noor Zahara (Inactive) added a comment -

            The only other valid IGB package I see is com.affymetrix. Could you verify the same? (I am a bit confused.)

            Also my suggestion would be changing "works_with" to "platform_compatibility" as a part of refactoring ticket.

            Show
            noor91zahara Noor Zahara (Inactive) added a comment - The only other valid IGB package I see is com.affymetrix. Could you verify the same? (I am a bit confused.) Also my suggestion would be changing "works_with" to "platform_compatibility" as a part of refactoring ticket.
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Yes, I agree. Changing "works_with" to "platform_compatibility" should be done as a part of refactoring ticket.

            Regarding getting a list of packages in the platform:
            The only way I can think of to do it would be to look at all the pom.xml files and parse out the exported packages. Or manually go through every module in the project and look at all the package names. That would take a lot of time and would probably be hard to maintain. There is surely a better way.

            Here is an example of a package that does not contain "igb" in the name:
            https://bitbucket.org/lorainelab/integrated-genome-browser/src/master/core/common/src/main/java/com/affymetrix/common/

            To close this, maybe best approach is:

            • add the logic for searching for with more than one regular expression
            • add a regex to catch the packages in the link above but don't spend a lot of time digging through the entire code base and all the pom's
            • create a new ticket where the task is to identify all packages
            • create a new ticket to find a more robust approach that does not require us to update App Store every time we add or refactor platform packages
            Show
            ann.loraine Ann Loraine added a comment - - edited Yes, I agree. Changing "works_with" to "platform_compatibility" should be done as a part of refactoring ticket. Regarding getting a list of packages in the platform: The only way I can think of to do it would be to look at all the pom.xml files and parse out the exported packages. Or manually go through every module in the project and look at all the package names. That would take a lot of time and would probably be hard to maintain. There is surely a better way. Here is an example of a package that does not contain "igb" in the name: https://bitbucket.org/lorainelab/integrated-genome-browser/src/master/core/common/src/main/java/com/affymetrix/common/ To close this, maybe best approach is: add the logic for searching for with more than one regular expression add a regex to catch the packages in the link above but don't spend a lot of time digging through the entire code base and all the pom's create a new ticket where the task is to identify all packages create a new ticket to find a more robust approach that does not require us to update App Store every time we add or refactor platform packages
            Hide
            noor91zahara Noor Zahara (Inactive) added a comment - - edited

            As of now I have added a logic to support multiple IGB packages.

            Creating tickets for finding out all the IGB packages and also a way to avoid updating Appstore each time a new IGB package is added.

            Kindly review the code.

            Show
            noor91zahara Noor Zahara (Inactive) added a comment - - edited As of now I have added a logic to support multiple IGB packages. Creating tickets for finding out all the IGB packages and also a way to avoid updating Appstore each time a new IGB package is added. Kindly review the code.
            Hide
            ann.loraine Ann Loraine added a comment -

            Merged into the master branch and deployed to https://dev-appstore-2.bioviz.org for testing.

            Show
            ann.loraine Ann Loraine added a comment - Merged into the master branch and deployed to https://dev-appstore-2.bioviz.org for testing.
            Hide
            shamika Shamika Gajanan Kulkarni (Inactive) added a comment -

            Testing on hold, since Sameer Shanbhag is working on a ticket related to 'Improve box-release-info on apps page'

            Show
            shamika Shamika Gajanan Kulkarni (Inactive) added a comment - Testing on hold, since Sameer Shanbhag is working on a ticket related to 'Improve box-release-info on apps page'
            Hide
            sameer Sameer Shanbhag (Inactive) added a comment -

            Logic is not working for Command Socket App Moving it back to To-Do

            Show
            sameer Sameer Shanbhag (Inactive) added a comment - Logic is not working for Command Socket App Moving it back to To-Do
            Hide
            noor91zahara Noor Zahara (Inactive) added a comment -

            This is fixed as a part of IGBF-2139

            Show
            noor91zahara Noor Zahara (Inactive) added a comment - This is fixed as a part of IGBF-2139
            Show
            noor91zahara Noor Zahara (Inactive) added a comment - Code changes - https://bitbucket.org/nzahara/appstore-local/branch/IGBF-2099-1#diff Related to IGBF-2148
            Hide
            ann.loraine Ann Loraine added a comment -

            Please rebase onto the master branch and deploy on an EC2 for pre-merge testing.
            Noor Zahara

            Show
            ann.loraine Ann Loraine added a comment - Please rebase onto the master branch and deploy on an EC2 for pre-merge testing. Noor Zahara
            Hide
            noor91zahara Noor Zahara (Inactive) added a comment - - edited

            This has been rebased with master. Kindly review the change.

            Show
            noor91zahara Noor Zahara (Inactive) added a comment - - edited This has been rebased with master. Kindly review the change.
            Hide
            ann.loraine Ann Loraine added a comment -

            Change request:

            Please change error_message (line 105) to:

            "Please note: We read your App's repository.xml file but could not determine the IGB version it requires. Not to worry! You can enter this information manually after the App is released."

            Show
            ann.loraine Ann Loraine added a comment - Change request: Please change error_message (line 105) to: "Please note: We read your App's repository.xml file but could not determine the IGB version it requires. Not to worry! You can enter this information manually after the App is released."
            Hide
            noor91zahara Noor Zahara (Inactive) added a comment -

            I have changed the error message to "Please note: We read your App's repository.xml file but could not determine the IGB version it requires. Not to worry! You can enter this information manually after the App is released."

            Show
            noor91zahara Noor Zahara (Inactive) added a comment - I have changed the error message to "Please note: We read your App's repository.xml file but could not determine the IGB version it requires. Not to worry! You can enter this information manually after the App is released."
            Show
            noor91zahara Noor Zahara (Inactive) added a comment - https://bitbucket.org/lorainelab/appstore/pull-requests/171/igbf-2099-1/diff - PR raised
            Hide
            ann.loraine Ann Loraine added a comment -

            Merged and deployed to https://dev-appstore-2.bioviz.org.

            Show
            ann.loraine Ann Loraine added a comment - Merged and deployed to https://dev-appstore-2.bioviz.org .
            Hide
            prutha Prutha Kulkarni (Inactive) added a comment -

            Tested on dev-appstore-2. The works with field along with whichever IGB version it is compatible with is shown properly on the app page.
            Moving it to DONE.

            Show
            prutha Prutha Kulkarni (Inactive) added a comment - Tested on dev-appstore-2. The works with field along with whichever IGB version it is compatible with is shown properly on the app page. Moving it to DONE.

              People

              • Assignee:
                noor91zahara Noor Zahara (Inactive)
                Reporter:
                ann.loraine Ann Loraine
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: