Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2446

Clone current Jira using backup or other approach as makes sense

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Story Points:
      3.5
    • Sprint:
      Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug, Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep, Fall 2: 28 Sep - 9 Oct

      Description

      For this task, clone our Jira onto a new VM using the backups or other method as appropriate.
      Take very careful notes to ensure that you (or someone else) can easily make a new clone in a short period of time.
      Also, develop testing instructions needed to validate that the clone is correct.
      Once you have learned all the tricks, make another clone! And then another one

      Goal: Decrease the time to create and deploy and test a clone to 1 hour or less.

        Attachments

          Issue Links

            Activity

            Hide
            ann.loraine Ann Loraine added a comment -

            Created folder in Google Drive containing:

            • mysql dump of Jira database from 6/23/20
            • jira home directory "tarball" from 6/23/20
            • saved Jira system information page (logged in as "admin" user)

            Also, see:

            Suggested protocol:

            • Use ansible playbooks to install dependencies (MySQL, Java, Tomcat)
            • Use ansible playbook (if possible) to install Jira version 6.4.8
            • Use ansible playbook to modify Apache configuration files to run Jira via a reverse proxy
            Show
            ann.loraine Ann Loraine added a comment - Created folder in Google Drive containing: mysql dump of Jira database from 6/23/20 jira home directory "tarball" from 6/23/20 saved Jira system information page (logged in as "admin" user) Also, see: https://community.atlassian.com/t5/Jira-Questions/Fully-automated-installation-configuration-of-JIRA/qaq-p/147413 Suggested protocol: Use ansible playbooks to install dependencies (MySQL, Java, Tomcat) Use ansible playbook (if possible) to install Jira version 6.4.8 Use ansible playbook to modify Apache configuration files to run Jira via a reverse proxy
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            None of the MySQL databases was getting backed up.

            The command that fails as a cron job but which seems to work fine as a manual command looks like this:

            docker exec -it jira-db /usr/local/mysql/bin/mysqldump -u auser -pnotactuallythepassword databasename | gzip -9 > daily/jira/mysql/jira_$(date +%F).sql.gz
            

            Our Jira is deployed in a docker container, so to get the database out of it, you have to run docker exec as in the above command.

            I don't know yet what the problem is, but I bet it has something to do with the shell used by cron being different somehow from the interactive shell.

            Update:

            Environment was a good guess but it's probably the "-i" flag according to this: https://stackoverflow.com/questions/37089033/docker-exec-is-not-working-in-cron.

            Show
            ann.loraine Ann Loraine added a comment - - edited None of the MySQL databases was getting backed up. The command that fails as a cron job but which seems to work fine as a manual command looks like this: docker exec -it jira-db /usr/local/mysql/bin/mysqldump -u auser -pnotactuallythepassword databasename | gzip -9 > daily/jira/mysql/jira_$(date +%F).sql.gz Our Jira is deployed in a docker container, so to get the database out of it, you have to run docker exec as in the above command. I don't know yet what the problem is, but I bet it has something to do with the shell used by cron being different somehow from the interactive shell. Update: Environment was a good guess but it's probably the "-i" flag according to this: https://stackoverflow.com/questions/37089033/docker-exec-is-not-working-in-cron .
            Hide
            ann.loraine Ann Loraine added a comment -

            Please use this repository for new playbooks and deployment code: https://bitbucket.org/lorainelab/jira-playbooks

            cc: Chester Dias

            Show
            ann.loraine Ann Loraine added a comment - Please use this repository for new playbooks and deployment code: https://bitbucket.org/lorainelab/jira-playbooks cc: Chester Dias
            Hide
            cdias1 Chester Dias (Inactive) added a comment -

            I noticed the existing Jira is running on t2 large
            Atlassian has suggested we use M or C series since the performance of t series is not known to be great and they are not optimised
            https://confluence.atlassian.com/jirakb/avoid-the-use-of-micro-instances-for-jira-server-in-aws-600115398.html
            cc: [~aloraine]
            Please let me know the version of mysql connector jar currently in use

            the jar should be present under: $JIRA_HOME/lib

            Please note there are 2 home folders which are similar but one is a Jira data folder...The exact folder name should be present in the .profile

            Show
            cdias1 Chester Dias (Inactive) added a comment - I noticed the existing Jira is running on t2 large Atlassian has suggested we use M or C series since the performance of t series is not known to be great and they are not optimised https://confluence.atlassian.com/jirakb/avoid-the-use-of-micro-instances-for-jira-server-in-aws-600115398.html cc: [~aloraine] Please let me know the version of mysql connector jar currently in use the jar should be present under: $JIRA_HOME/lib Please note there are 2 home folders which are similar but one is a Jira data folder...The exact folder name should be present in the .profile
            Hide
            ann.loraine Ann Loraine added a comment -

            System info screen in Jira reports the following Database driver:

            • MySQL Connector Java mysql-connector-java-5.1.31 ( Revision: alexander.soklakov@oracle.com-20140520065950-groqzzbvxprqdmnz )
            Show
            ann.loraine Ann Loraine added a comment - System info screen in Jira reports the following Database driver: MySQL Connector Java mysql-connector-java-5.1.31 ( Revision: alexander.soklakov@oracle.com-20140520065950-groqzzbvxprqdmnz )
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Current host for Jira and Confluence:

            • Amazon Linux AMI 2016.03.3 x86_64 HVM GP2
            • Image Size - 8GB

            Instance type:

            • t2.large

            Drives:

            [root@ip-172-31-11-196 etc]# df -h
            Filesystem Size Used Avail Use% Mounted on
            /dev/xvda1 99G 91G 7.9G 93% /
            devtmpfs 3.9G 144K 3.9G 1% /dev
            tmpfs 3.9G 0 3.9G 0% /dev/shm

            Show
            ann.loraine Ann Loraine added a comment - - edited Current host for Jira and Confluence: Amazon Linux AMI 2016.03.3 x86_64 HVM GP2 Image Size - 8GB Instance type: t2.large Drives: [root@ip-172-31-11-196 etc] # df -h Filesystem Size Used Avail Use% Mounted on /dev/xvda1 99G 91G 7.9G 93% / devtmpfs 3.9G 144K 3.9G 1% /dev tmpfs 3.9G 0 3.9G 0% /dev/shm
            Hide
            cdias1 Chester Dias (Inactive) added a comment -

            [~aloraine]
            I have used the latest connector of MySQL for now. Cloning is complete. The playbooks can be used to replicate the Jira infrastructure on a different host within a few minutes.
            Please Review: https://bitbucket.org/chesterdias/jira-playbooks-chester-local/branch/IGBF-2446#diff

            Just note that the URL will start on port 8080

            Show
            cdias1 Chester Dias (Inactive) added a comment - [~aloraine] I have used the latest connector of MySQL for now. Cloning is complete. The playbooks can be used to replicate the Jira infrastructure on a different host within a few minutes. Please Review: https://bitbucket.org/chesterdias/jira-playbooks-chester-local/branch/IGBF-2446#diff Just note that the URL will start on port 8080
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Two queries:

            • Can you modify this to allow the user to provide an unpacked (tar.gz) Jira home directory instead of having to uncompress it?
            • Could you show us the site running live from your account?
            Show
            ann.loraine Ann Loraine added a comment - - edited Two queries: Can you modify this to allow the user to provide an unpacked (tar.gz) Jira home directory instead of having to uncompress it? Could you show us the site running live from your account?
            Hide
            cdias1 Chester Dias (Inactive) added a comment -

            It wont be a good idea to allow user to provide an unpacked tar.gz.This would be present on the control node, so the time taken to copy data as the individual file will slow down the process.
            In the existing method, we just give the JIRA home tar.gz or a tar to ansible for an easier copy. Then Ansible will untar it.
            [~aloraine] Let me know if you still want an unpacked tar. I will tweak code a little

            Show
            cdias1 Chester Dias (Inactive) added a comment - It wont be a good idea to allow user to provide an unpacked tar.gz.This would be present on the control node, so the time taken to copy data as the individual file will slow down the process. In the existing method, we just give the JIRA home tar.gz or a tar to ansible for an easier copy. Then Ansible will untar it. [~aloraine] Let me know if you still want an unpacked tar. I will tweak code a little
            Hide
            cdias1 Chester Dias (Inactive) added a comment - - edited

            http://ec2-23-20-237-204.compute-1.amazonaws.com:8080/secure/Dashboard.jspa

            this is running on M series server
            you can try logging in

            Show
            cdias1 Chester Dias (Inactive) added a comment - - edited http://ec2-23-20-237-204.compute-1.amazonaws.com:8080/secure/Dashboard.jspa this is running on M series server you can try logging in
            Hide
            ann.loraine Ann Loraine added a comment -

            So sorry! I wrote the wrong thing! I should have said "compressed" and not "unpacked." Also, I did not thoroughly read the playbook.

            Keeping the existing method as you have written it is the correct thing, in my view. We should identify the "tarball" file using a playbook variable and simply have ansible unpack it and deploy it.

            I see now that is exactly what you are doing with:

            - name: Copy the existing Jira home folder
              copy:
                src: "/home/ec2-user/{{ old_jira_data }}"
                dest: /home/ec2-user/jirahomedir
                owner: ec2-user
                group: ec2-user
                mode: '0755'
            

            and

            - name: Extract Jira Data tar into {{ jira_data }}
              unarchive:
                src: "/home/ec2-user/jirahomedir/{{ old_jira_data }}"
                dest: "{{jira_data}}"
                copy: no
            

            Two requests:

            • Could you change this variable name? Instead of: "old_jira_data" can we call it "jira_data_tarball" ?
            • Please re-factor this shell command to use a handler so that it only runs if previous tasks have a "changed" status:
            - name: Start Jira server
              shell: sudo nohup {{jira_home}}/bin/start-jira.sh &
            
            Show
            ann.loraine Ann Loraine added a comment - So sorry! I wrote the wrong thing! I should have said "compressed" and not "unpacked." Also, I did not thoroughly read the playbook. Keeping the existing method as you have written it is the correct thing, in my view. We should identify the "tarball" file using a playbook variable and simply have ansible unpack it and deploy it. I see now that is exactly what you are doing with: - name: Copy the existing Jira home folder copy: src: "/home/ec2-user/{{ old_jira_data }}" dest: /home/ec2-user/jirahomedir owner: ec2-user group: ec2-user mode: '0755' and - name: Extract Jira Data tar into {{ jira_data }} unarchive: src: "/home/ec2-user/jirahomedir/{{ old_jira_data }}" dest: "{{jira_data}}" copy: no Two requests: Could you change this variable name? Instead of: "old_jira_data" can we call it "jira_data_tarball" ? Please re-factor this shell command to use a handler so that it only runs if previous tasks have a "changed" status: - name: Start Jira server shell: sudo nohup {{jira_home}}/bin/start-jira.sh &
            Hide
            ann.loraine Ann Loraine added a comment -

            Looks like the clone you stood up works - attachments are there!
            Have you tried logging in with your Jira account credentials?

            Show
            ann.loraine Ann Loraine added a comment - Looks like the clone you stood up works - attachments are there! Have you tried logging in with your Jira account credentials?
            Hide
            cdias1 Chester Dias (Inactive) added a comment - - edited

            Yes it logs in. I have verified that. Also checked to transition a ticket from one state to another.

            Some reading up has given me an idea for an approach to a version upgrade, check the below table.
            https://confluence.atlassian.com/adminjiraserver074/upgrading-jira-applications-using-the-installer-881683273.html?_ga=2.219711518.684639599.1593546483-821977391.1584022985#UpgradingJIRAapplicationsusingtheinstaller-1.Determineyourupgradepath

            Looks like a safe path to upgrade will be
            6.3.15->7.0.11->8.X.X

            I will change the variable name old_jira_data to jira_data_tarball.
            Converting from the shell script to service. I am not very certain on process but I can add a when statement which will determine if the previous task has changed.
            This is Jira local tomcat. Not installed as a service. Please let me know if you still need it as a service, I could look into it

            cc: [~aloraine]

            Show
            cdias1 Chester Dias (Inactive) added a comment - - edited Yes it logs in. I have verified that. Also checked to transition a ticket from one state to another. Some reading up has given me an idea for an approach to a version upgrade, check the below table. https://confluence.atlassian.com/adminjiraserver074/upgrading-jira-applications-using-the-installer-881683273.html?_ga=2.219711518.684639599.1593546483-821977391.1584022985#UpgradingJIRAapplicationsusingtheinstaller-1.Determineyourupgradepath Looks like a safe path to upgrade will be 6.3.15->7.0.11->8.X.X I will change the variable name old_jira_data to jira_data_tarball. Converting from the shell script to service. I am not very certain on process but I can add a when statement which will determine if the previous task has changed. This is Jira local tomcat. Not installed as a service. Please let me know if you still need it as a service, I could look into it cc: [~aloraine]
            Hide
            cdias1 Chester Dias (Inactive) added a comment -

            The direct upgrade to 7.0.11 has thrown an error that has left the cloned jira in a nonresponsive state, I am attempting to try to upgrade to 7.0.0 first.

            Nowlan Freese, Logan Weidenhammer, Chaitanya Kintali, [~aloraine], Jay Chamma, Philip Badzuh
            Please use the link below to check and test out the cloned Jira version. Please add any comments to the bugs observed
            http://ec2-34-200-84-199.compute-1.amazonaws.com:8080/secure/Dashboard.jspa

            Show
            cdias1 Chester Dias (Inactive) added a comment - The direct upgrade to 7.0.11 has thrown an error that has left the cloned jira in a nonresponsive state, I am attempting to try to upgrade to 7.0.0 first. Nowlan Freese , Logan Weidenhammer , Chaitanya Kintali , [~aloraine] , Jay Chamma , Philip Badzuh Please use the link below to check and test out the cloned Jira version. Please add any comments to the bugs observed http://ec2-34-200-84-199.compute-1.amazonaws.com:8080/secure/Dashboard.jspa
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Suggestion:

            If "service" module does not work, I think it will be OK. We can get some advice from Atlassian on our setup if we need it. Also, we will probably run Apache on the server in a reverse proxy setup and delegate to tomcat. This is how we are doing it on our current host.

            FYI: The current host has the services running in Docker containers managed by Docker compose. There are Docker files and images, but I don't know enough about Docker to understand how to clone the setup.

            Show
            ann.loraine Ann Loraine added a comment - - edited Suggestion: Move the "shell" task to a handler, as with https://bitbucket.org/lorainelab/bioviz-playbooks/src/master/roles/apache/handlers/main.yml . If "service" module does not work, I think it will be OK. We can get some advice from Atlassian on our setup if we need it. Also, we will probably run Apache on the server in a reverse proxy setup and delegate to tomcat. This is how we are doing it on our current host. FYI: The current host has the services running in Docker containers managed by Docker compose. There are Docker files and images, but I don't know enough about Docker to understand how to clone the setup.
            Hide
            cdias1 Chester Dias (Inactive) added a comment -

            I have moved the Shell task to handlers, verified that it is working.

            Show
            cdias1 Chester Dias (Inactive) added a comment - I have moved the Shell task to handlers, verified that it is working.
            Show
            cdias1 Chester Dias (Inactive) added a comment - https://bitbucket.org/lorainelab/jira-playbooks/pull-requests/1/igbf-2446/diff
            Hide
            ann.loraine Ann Loraine added a comment -

            PR merged.

            Show
            ann.loraine Ann Loraine added a comment - PR merged.
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            To test, run the playbooks.
            Note: You will need sql and jira data files to test.

            Show
            ann.loraine Ann Loraine added a comment - - edited To test, run the playbooks. Note: You will need sql and jira data files to test.
            Hide
            ann.loraine Ann Loraine added a comment -

            Moving to backlog.

            Show
            ann.loraine Ann Loraine added a comment - Moving to backlog.
            Hide
            ann.loraine Ann Loraine added a comment -

            First drafts are done and are being tested and improved. Moving to Done.

            Show
            ann.loraine Ann Loraine added a comment - First drafts are done and are being tested and improved. Moving to Done.

              People

              • Assignee:
                cdias1 Chester Dias (Inactive)
                Reporter:
                ann.loraine Ann Loraine
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: