Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2545

Improve Genome Dashboard interaction with quickload sites

    Details

    • Type: Improvement
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Story Points:
      1.5
    • Sprint:
      Fall 2: 28 Sep - 9 Oct, Fall 3: Oct 12 - Oct 23, Fall 4 Oct 26 - Nov 6

      Description

      Genome Dashboard retrieves data from all IGB Quickload sites by parsing the master branch's igb preferences json file.

      However, it breaks if one of those sites is not available.

      Modify the code so that if a site is not available, Genome Dashboard will not crash.

      Also, modify the code so that if the primary site is available, it tries to use the backup site.

      Lastly, add a configuration to bioviz-playbooks that ensures that the IGB preferences file for the current release branch is used and not the latest master branch.

      bioviz playbooks repo: https://bitbucket.org/lorainelab/bioviz-playbooks/src/master/

        Attachments

          Activity

          ann.loraine Ann Loraine created issue -
          ann.loraine Ann Loraine made changes -
          Field Original Value New Value
          Epic Link IGBF-1765 [ 17855 ]
          Hide
          ann.loraine Ann Loraine added a comment -

          Sameer Shanbhag - please take a look

          Show
          ann.loraine Ann Loraine added a comment - Sameer Shanbhag - please take a look
          ann.loraine Ann Loraine made changes -
          Description Genome Dashboard retrieves data from all IGB Quickload sites by parsing the master branch's igb preferences json file.

          However, it breaks if one of those sites is not available.

          Modify the code so that if a site is not available, Genome Dashboard will not crash.

          Also, modify the code so that if the primary site is available, it tries to use the backup site.

          Lastly, add a configuration to bioviz-playbooks that ensures that the IGB preferences file for the current release branch is used and not the latest master branch.

          Genome Dashboard retrieves data from all IGB Quickload sites by parsing the master branch's igb preferences json file.

          However, it breaks if one of those sites is not available.

          Modify the code so that if a site is not available, Genome Dashboard will not crash.

          Also, modify the code so that if the primary site is available, it tries to use the backup site.

          Lastly, add a configuration to bioviz-playbooks that ensures that the IGB preferences file for the current release branch is used and not the latest master branch.

          bioviz playbooks repo: https://bitbucket.org/lorainelab/bioviz-playbooks/src/master/
          sameer Sameer Shanbhag (Inactive) made changes -
          Status To-Do [ 10305 ] In Progress [ 3 ]
          sameer Sameer Shanbhag (Inactive) made changes -
          Assignee Sameer Shanbhag [ sameer ]
          Hide
          sameer Sameer Shanbhag (Inactive) added a comment - - edited

          Hi Professor,

          I have updated the Bioviz Playbook to point to my feature branch, Pull Request:
          Updated PR

          Improvement to deal with unavailable Quickload website has been added to the following commit:

          c.c. Prof. [~aloraine] Please have a look at the update PR for Bioviz playbook and let me know if that looks right

          Show
          sameer Sameer Shanbhag (Inactive) added a comment - - edited Hi Professor, I have updated the Bioviz Playbook to point to my feature branch, Pull Request: Updated PR https://bitbucket.org/lorainelab/bioviz-playbooks/pull-requests/7/igbf-2545-add-feature-branch-to-bioviz Improvement to deal with unavailable Quickload website has been added to the following commit: https://bitbucket.org/sshanbh1/genome-dashboard/commits/07734cd8ee1a8c609c0e2bf015fd90bbde02234f c.c. Prof. [~aloraine] Please have a look at the update PR for Bioviz playbook and let me know if that looks right
          sameer Sameer Shanbhag (Inactive) made changes -
          Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
          sameer Sameer Shanbhag (Inactive) made changes -
          Assignee Sameer Shanbhag [ sameer ]
          Hide
          ann.loraine Ann Loraine added a comment - - edited

          Please do not add bioviz_vars.yml to the repository. This actually should never be checked in because once properly configured the file contains AWS secret keys and whatnot.

          What we actually need to do is add a configuration to bioviz-playbooks that ensures that the IGB preferences file for the current release branch of Integrated Genome Browser is used and not the latest master branch.

          The bioviz-playbooks have a configuration that specifies the IGB release version being used. When deploying genome dashboard code, we can include a play that edits deployed genome dashboard code to ensure that the release branch specified in the playbooks is also the same version of the IGB preferences file being used to retrieve the igbquickload sites and their backups. This can be done using the "file" module in ansible.

          I will show how this works tomorrow. It is a bit easier to show how it works than explain it. So let's hold off on this until then.

          Thank you Sameer Shanbhag! Have a great rest of your day!

          Show
          ann.loraine Ann Loraine added a comment - - edited Please do not add bioviz_vars.yml to the repository. This actually should never be checked in because once properly configured the file contains AWS secret keys and whatnot. What we actually need to do is add a configuration to bioviz-playbooks that ensures that the IGB preferences file for the current release branch of Integrated Genome Browser is used and not the latest master branch. The bioviz-playbooks have a configuration that specifies the IGB release version being used. When deploying genome dashboard code, we can include a play that edits deployed genome dashboard code to ensure that the release branch specified in the playbooks is also the same version of the IGB preferences file being used to retrieve the igbquickload sites and their backups. This can be done using the "file" module in ansible. I will show how this works tomorrow. It is a bit easier to show how it works than explain it. So let's hold off on this until then. Thank you Sameer Shanbhag ! Have a great rest of your day!
          ann.loraine Ann Loraine made changes -
          Sprint Fall 2: 28 Sep - 9 Oct [ 104 ] Fall 2: 28 Sep - 9 Oct, Fall 3: Oct 12 - Oct 23 [ 104, 106 ]
          ann.loraine Ann Loraine made changes -
          Rank Ranked higher
          Hide
          sameer Sameer Shanbhag (Inactive) added a comment -

          Hi, Prof. [~aloraine],

          I have updated the pull request:

          Please let me know if the changes look good.

          Show
          sameer Sameer Shanbhag (Inactive) added a comment - Hi, Prof. [~aloraine] , I have updated the pull request: https://bitbucket.org/lorainelab/bioviz-playbooks/pull-requests/8/igbf-2545-implement-a-play-to-replace Please let me know if the changes look good.
          ann.loraine Ann Loraine made changes -
          Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
          ann.loraine Ann Loraine made changes -
          Status First Level Review in Progress [ 10301 ] Ready for Pull Request [ 10304 ]
          ann.loraine Ann Loraine made changes -
          Status Ready for Pull Request [ 10304 ] Pull Request Submitted [ 10101 ]
          ann.loraine Ann Loraine made changes -
          Status Pull Request Submitted [ 10101 ] Reviewing Pull Request [ 10303 ]
          ann.loraine Ann Loraine made changes -
          Status Reviewing Pull Request [ 10303 ] Merged Needs Testing [ 10002 ]
          Hide
          ann.loraine Ann Loraine added a comment - - edited

          To test:

          • Run the playbooks and confirm the site is properly deployed.
          • Make a branch of IGB and modify the preferences JSON file to include a new quickload repository with species and/or genome version not present in the current release of IGB; confirm the new QL species appears in Genome Dashboard deployed on the test site
          • Include non-functional primary, non-functional secondary pairs in the JSON file to check that Genome Dashboard still works and also that it can use a working secondary site when its primary site is non-functional

          Keep good note on exactly what you tested. Ask Sameer for clarification if any of this does not make sense.

          Show
          ann.loraine Ann Loraine added a comment - - edited To test: Run the playbooks and confirm the site is properly deployed. Make a branch of IGB and modify the preferences JSON file to include a new quickload repository with species and/or genome version not present in the current release of IGB; confirm the new QL species appears in Genome Dashboard deployed on the test site Include non-functional primary, non-functional secondary pairs in the JSON file to check that Genome Dashboard still works and also that it can use a working secondary site when its primary site is non-functional Keep good note on exactly what you tested. Ask Sameer for clarification if any of this does not make sense.
          noor91zahara Noor Zahara (Inactive) made changes -
          Status Merged Needs Testing [ 10002 ] Post-merge Testing In Progress [ 10003 ]
          noor91zahara Noor Zahara (Inactive) made changes -
          Assignee Noor Zahara [ noor91zahara ]
          Hide
          ann.loraine Ann Loraine added a comment - - edited

          Message from Noor Zahara:

          I am getting an error - "msg": "Instance creation failed: Value (Bioviz) for parameter iamInstanceProfile.name is invalid. Invalid IAM Instance Profile name" Is there something that I am missing?

          Yes, this needs to be set up in your AWS account. The setup for this ought to be should be included in the bioviz playbook, but it might not be.

          Please ask Sameer Shanbhag - Did he encounter the same problem?

          Show
          ann.loraine Ann Loraine added a comment - - edited Message from Noor Zahara : I am getting an error - "msg": "Instance creation failed: Value (Bioviz) for parameter iamInstanceProfile.name is invalid. Invalid IAM Instance Profile name" Is there something that I am missing? Yes, this needs to be set up in your AWS account. The setup for this ought to be should be included in the bioviz playbook, but it might not be. Please ask Sameer Shanbhag - Did he encounter the same problem?
          noor91zahara Noor Zahara (Inactive) made changes -
          Status Post-merge Testing In Progress [ 10003 ] Merged Needs Testing [ 10002 ]
          noor91zahara Noor Zahara (Inactive) made changes -
          Assignee Noor Zahara [ noor91zahara ] Ann Loraine [ aloraine ]
          noor91zahara Noor Zahara (Inactive) made changes -
          Status Merged Needs Testing [ 10002 ] Post-merge Testing In Progress [ 10003 ]
          noor91zahara Noor Zahara (Inactive) made changes -
          Status Post-merge Testing In Progress [ 10003 ] To-Do [ 10305 ]
          ann.loraine Ann Loraine made changes -
          Status To-Do [ 10305 ] In Progress [ 3 ]
          ann.loraine Ann Loraine made changes -
          Sprint Fall 2: 28 Sep - 9 Oct, Fall 3: Oct 12 - Oct 23 [ 104, 106 ] Fall 2: 28 Sep - 9 Oct, Fall 3: Oct 12 - Oct 23, Fall 4 Oct 26 - Nov 6 [ 104, 106, 107 ]
          ann.loraine Ann Loraine made changes -
          Rank Ranked higher
          ann.loraine Ann Loraine made changes -
          Assignee Ann Loraine [ aloraine ]
          ann.loraine Ann Loraine made changes -
          Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
          ann.loraine Ann Loraine made changes -
          Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
          ann.loraine Ann Loraine made changes -
          Status First Level Review in Progress [ 10301 ] Ready for Pull Request [ 10304 ]
          ann.loraine Ann Loraine made changes -
          Status Ready for Pull Request [ 10304 ] Pull Request Submitted [ 10101 ]
          ann.loraine Ann Loraine made changes -
          Status Pull Request Submitted [ 10101 ] Reviewing Pull Request [ 10303 ]
          ann.loraine Ann Loraine made changes -
          Status Reviewing Pull Request [ 10303 ] Merged Needs Testing [ 10002 ]
          ann.loraine Ann Loraine made changes -
          Status Merged Needs Testing [ 10002 ] Post-merge Testing In Progress [ 10003 ]
          ann.loraine Ann Loraine made changes -
          Resolution Done [ 10000 ]
          Status Post-merge Testing In Progress [ 10003 ] Closed [ 6 ]
          ann.loraine Ann Loraine made changes -
          Assignee Sameer Shanbhag [ sameer ]

            People

            • Assignee:
              sameer Sameer Shanbhag (Inactive)
              Reporter:
              ann.loraine Ann Loraine
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: