Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2421

Write role to replicate mysql database for appstore redeployment

    Details

    • Type: New Feature
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      The current App Store deployed at https://apps.bioviz.org uses an out-of-date codebase.

      We need to update it, but we also cannot risk (much) down-time.

      To update the code, I would like to set up an entirely new EC2 with new code, but using a copy of the current production database.

      For this task, let's a new role to appstore-playbooks that creates a copy of an existing mysql database to given host.

      Assumptions:

      • RDS hosts for the original and the copy are already configured
      • RDS hosts may be different for original and copy
      • Ansible control nodes can connect to both RDS hosts using mysql ansible modules

      User must supply:

      • admin username and passwords for both hosts
      • name of database to copy
      • name of copied database

      Don't do anything to the database – just copy it.

        Attachments

          Issue Links

            Activity

            Hide
            cdias1 Chester Dias (Inactive) added a comment -

            The below steps Summarize the flow order of tasks
            a. Take a manual snapshot of the DB
            b. Download the original DB copy.
            c. Create a new DB from the copy if not existing, if existing drop and recreate
            d. Load the data into the new DB.
            e. (optional) Make changes to new DB code.
            f. Set new DB as the Primary.

            The above idea will help to follow the Blue-Green Code Deployment strategy(Minimal downtime)
            As part of this, one DB(called Blue) will continue to be active during deployment while we ready the second DB(Green) with the code of Blue and additional code change of the release.

            we can then just modify the EC2 code to point to the new DB. Both DB will be present on the RDS but only 1 will remain active, while second remains as a backup.

            This way the Production DB will toggle between Blue and Green DBs and there always will be a backup present on the other DB.
            The snapshot can be dropped once the activities are done and complete.
            [~aloraine] Please let me know your thoughts on this idea https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html#:~:text=Blue%2Dgreen%20deployment%20is%20a,live%20and%20Green%20is%20idle.

            Show
            cdias1 Chester Dias (Inactive) added a comment - The below steps Summarize the flow order of tasks a. Take a manual snapshot of the DB b. Download the original DB copy. c. Create a new DB from the copy if not existing, if existing drop and recreate d. Load the data into the new DB. e. (optional) Make changes to new DB code. f. Set new DB as the Primary. The above idea will help to follow the Blue-Green Code Deployment strategy(Minimal downtime) As part of this, one DB(called Blue) will continue to be active during deployment while we ready the second DB(Green) with the code of Blue and additional code change of the release. we can then just modify the EC2 code to point to the new DB. Both DB will be present on the RDS but only 1 will remain active, while second remains as a backup. This way the Production DB will toggle between Blue and Green DBs and there always will be a backup present on the other DB. The snapshot can be dropped once the activities are done and complete. [~aloraine] Please let me know your thoughts on this idea https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html#:~:text=Blue%2Dgreen%20deployment%20is%20a,live%20and%20Green%20is%20idle .
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            With our current playbooks, all we need to do (I think) is add a new role that duplicates the production database as described above.

            When we want to deploy a new version of appstore, we will duplicate the database first. Next, we'll run the "setup.yml" playbook to create a new EC2.

            The database-related roles as currently written will detect that the database for same already exists. All they will do to the database is add a new user for the new appstore EC2 to connect to the database.

            Modifying the ticket title to include writing the role. Also I increased the story points to 2.

            Chester Dias - please read again and also update the story points as needed.

            Show
            ann.loraine Ann Loraine added a comment - - edited With our current playbooks, all we need to do (I think) is add a new role that duplicates the production database as described above. When we want to deploy a new version of appstore, we will duplicate the database first. Next, we'll run the "setup.yml" playbook to create a new EC2. The database-related roles as currently written will detect that the database for same already exists. All they will do to the database is add a new user for the new appstore EC2 to connect to the database. Modifying the ticket title to include writing the role. Also I increased the story points to 2. Chester Dias - please read again and also update the story points as needed.
            Hide
            cdias1 Chester Dias (Inactive) added a comment -

            Following code role has been made
            This role will create a copy of the database on a different RDS host, If the second hostname is provided.
            If the second hostname is not provided the new DB is created on the same RDS host.
            The second DB name is mandatory.

            The second DB is loaded with all the data dump of the first DB.

            A user is created with privilege over the second DB so it can be accessed from the ec2 server.
            Note: This role must be run after ec2 is created

            [~aloraine]Please review: https://bitbucket.org/chesterdias/chester-local-appstore-playbooks/branch/IGBF-2421#diff

            Show
            cdias1 Chester Dias (Inactive) added a comment - Following code role has been made This role will create a copy of the database on a different RDS host, If the second hostname is provided. If the second hostname is not provided the new DB is created on the same RDS host. The second DB name is mandatory. The second DB is loaded with all the data dump of the first DB. A user is created with privilege over the second DB so it can be accessed from the ec2 server. Note: This role must be run after ec2 is created [~aloraine] Please review: https://bitbucket.org/chesterdias/chester-local-appstore-playbooks/branch/IGBF-2421#diff
            Hide
            ann.loraine Ann Loraine added a comment -

            Great first draft! Thanks for this! Please submit PR.

            Show
            ann.loraine Ann Loraine added a comment - Great first draft! Thanks for this! Please submit PR.
            Show
            cdias1 Chester Dias (Inactive) added a comment - PR: https://bitbucket.org/lorainelab/appstore-playbooks/pull-requests/29/igbf-2421/diff
            Hide
            ann.loraine Ann Loraine added a comment -

            Testing will be done as part of https://jira.transvar.org/browse/IGBF-2426.
            Moving to closed and re-assigning to original developer.

            Show
            ann.loraine Ann Loraine added a comment - Testing will be done as part of https://jira.transvar.org/browse/IGBF-2426 . Moving to closed and re-assigning to original developer.

              People

              • Assignee:
                cdias1 Chester Dias (Inactive)
                Reporter:
                ann.loraine Ann Loraine
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: