Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Story Points:
      6
    • Sprint:
      Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep, Fall 2: 28 Sep - 9 Oct, Fall 3: Oct 12 - Oct 23

      Description

      The IGB users and developers guides are hosted on https://wiki.transvar.org, which is running version 5.6.3 of Confluence.

      Spaces on this site which we need to maintain include:

      • IGB Developer's Guide
      • IGB Release Documentation
      • IGB User's Guide

      For this task, set up a new mirror site (using address wiki.bioviz.org) to host the above spaces only using the latest version of Confluence. In addition, develop a backup strategy that will allow us to set up a new Confluence site if required.
      https://community.atlassian.com/t5/Confluence-questions/How-do-we-turn-off-the-blog/qaq-p/169157

        Attachments

          Activity

          ann.loraine Ann Loraine created issue -
          ann.loraine Ann Loraine made changes -
          Field Original Value New Value
          Epic Link IGBF-2323 [ 18477 ]
          cdias1 Chester Dias (Inactive) made changes -
          Assignee Chester Dias [ cdias1 ]
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          The current confluence version used is v5.6.3
          Latest Confluence version is 7.7.2

          Initial read up shows we can directly upgrade from existing version to v7.0
          https://confluence.atlassian.com/doc/upgrading-confluence-manually-255363437.html

          [~aloraine] Could you please upload the latest copy of the confluence backups for me to start planning the playbooks.

          Show
          cdias1 Chester Dias (Inactive) added a comment - The current confluence version used is v5.6.3 Latest Confluence version is 7.7.2 Initial read up shows we can directly upgrade from existing version to v7.0 https://confluence.atlassian.com/doc/upgrading-confluence-manually-255363437.html [~aloraine] Could you please upload the latest copy of the confluence backups for me to start planning the playbooks.
          cdias1 Chester Dias (Inactive) made changes -
          Status To-Do [ 10305 ] In Progress [ 3 ]
          Hide
          ann.loraine Ann Loraine added a comment -

          Please see drive folder for confluence backups.

          Show
          ann.loraine Ann Loraine added a comment - Please see drive folder for confluence backups.
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          After running into several errors best possible solution to upgrade confluence seems to be the following.
          1. Backup the spaces that are needed following the steps on link below.
          https://confluence.atlassian.com/display/CONF43/Exporting+Confluence+Pages+and+Spaces+to+XML?_ga=2.64998630.81440753.1598312975-1876299205.1595446590

          2. Setup New Confluence Instance using Ansible(I have already started working on developing the playbooks).

          3. Load the zip file using the steps mentioned in the
          https://confluence.atlassian.com/display/CONF43/Restoring+a+Space

          Additional Notes
          There are differences in the mysql compatibilities and java versions used in the latest version and the existing in-use version of the confluence.

          Show
          cdias1 Chester Dias (Inactive) added a comment - After running into several errors best possible solution to upgrade confluence seems to be the following. 1. Backup the spaces that are needed following the steps on link below. https://confluence.atlassian.com/display/CONF43/Exporting+Confluence+Pages+and+Spaces+to+XML?_ga=2.64998630.81440753.1598312975-1876299205.1595446590 2. Setup New Confluence Instance using Ansible(I have already started working on developing the playbooks). 3. Load the zip file using the steps mentioned in the https://confluence.atlassian.com/display/CONF43/Restoring+a+Space Additional Notes There are differences in the mysql compatibilities and java versions used in the latest version and the existing in-use version of the confluence.
          Hide
          ann.loraine Ann Loraine added a comment - - edited

          Exported the "IGB User's Guide" space to a zipped XML file and saved to shared drive.
          Chester Dias - please check to make sure it contains everything needed, esp. image files.
          file name is: Confluence-space-export-135549-135472.xml.zip

          Exported the "Developers Guide" space to a zipped XML file and saved to the above shared drive.
          file name is: Confluence-space-export-140140-135475.xml.zip

          Show
          ann.loraine Ann Loraine added a comment - - edited Exported the "IGB User's Guide" space to a zipped XML file and saved to shared drive. Chester Dias - please check to make sure it contains everything needed, esp. image files. file name is: Confluence-space-export-135549-135472.xml.zip Exported the "Developers Guide" space to a zipped XML file and saved to the above shared drive. file name is: Confluence-space-export-140140-135475.xml.zip
          Hide
          cdias1 Chester Dias (Inactive) added a comment - - edited

          Notes about Java version
          Incorrect Java version can throw compatibility issues with commands in the execution scripts that are not recognized. some commands are deprecated and removed to the point being no longer supported or recognized. The Java execution commands are located in the bin folder inside setenv.sh, catalina.sh, start-confluence.sh.

          Notes about MySQL
          The new mysql5.7 has a slightly different connection string as compared to the earlier version. storage_engine is no longer recognized

          Notes about instance size
          Although this should not be an issue during testing, but Confluence demands the requirement of having a large type server for its memory requirements while the disk space initially can be less.

          Notes about License Key
          The license key for Confluence unlike in the case of Jira is located in the Confluence Home folder in the file confluence.cfg.xml. This also has details like DB connections, Server ID, etc.

          Notes about the Mysql Java Driver
          The mysql Java driver name is being flagged as changed to com.mysql.cj.jdbc.Driver from earlier com.mysql.jdbc.Driver . Possible reason being use of a recent mysql connector jar.
          The confluence setup does not come with a connector jar. This Jar has to be downloaded manually from a maven repository.

          Show
          cdias1 Chester Dias (Inactive) added a comment - - edited Notes about Java version Incorrect Java version can throw compatibility issues with commands in the execution scripts that are not recognized. some commands are deprecated and removed to the point being no longer supported or recognized. The Java execution commands are located in the bin folder inside setenv.sh, catalina.sh, start-confluence.sh. Notes about MySQL The new mysql5.7 has a slightly different connection string as compared to the earlier version. storage_engine is no longer recognized Notes about instance size Although this should not be an issue during testing, but Confluence demands the requirement of having a large type server for its memory requirements while the disk space initially can be less. Notes about License Key The license key for Confluence unlike in the case of Jira is located in the Confluence Home folder in the file confluence.cfg.xml. This also has details like DB connections, Server ID, etc. Notes about the Mysql Java Driver The mysql Java driver name is being flagged as changed to com.mysql.cj.jdbc.Driver from earlier com.mysql.jdbc.Driver . Possible reason being use of a recent mysql connector jar. The confluence setup does not come with a connector jar. This Jar has to be downloaded manually from a maven repository.
          Hide
          cdias1 Chester Dias (Inactive) added a comment - - edited

          [~aloraine] The SQL file in the Drive does not seem to contain any data. Could you please check that?.

          Please also provide me the Repository for adding playbooks.

          Also, Would you like me to look into Cloudformation here? This seems like the smallest use case since we don't have multiple services involved as such(Just EC2 instance no involvement of much security group changes).

          Show
          cdias1 Chester Dias (Inactive) added a comment - - edited [~aloraine] The SQL file in the Drive does not seem to contain any data. Could you please check that?. Please also provide me the Repository for adding playbooks. Also, Would you like me to look into Cloudformation here? This seems like the smallest use case since we don't have multiple services involved as such(Just EC2 instance no involvement of much security group changes).
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          Directly uploading the backup zip file of a space on the latest version works as expected.
          I have checked a few child pages, a few images, and links to be in working condition.

          [~aloraine] This is a temporary link for testing only.
          http://100.25.10.47:8090/display/igbman/About+the+User%27s+Guide

          Show
          cdias1 Chester Dias (Inactive) added a comment - Directly uploading the backup zip file of a space on the latest version works as expected. I have checked a few child pages, a few images, and links to be in working condition. [~aloraine] This is a temporary link for testing only. http://100.25.10.47:8090/display/igbman/About+the+User%27s+Guide
          Hide
          ann.loraine Ann Loraine added a comment -

          request:

          • remove the "test" and "demo" spaces (do these show up by default?)
          • modify the side panel (on left) to look like the production version (no "blogs" link or "pages" link)
          Show
          ann.loraine Ann Loraine added a comment - request: remove the "test" and "demo" spaces (do these show up by default?) modify the side panel (on left) to look like the production version (no "blogs" link or "pages" link)
          ann.loraine Ann Loraine made changes -
          Sprint Summer 6: 17 Aug - 28 Aug [ 101 ] Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep [ 101, 102 ]
          ann.loraine Ann Loraine made changes -
          Rank Ranked higher
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          http://52.45.117.194:8090/display/igbman/Home

          Those seem to be showing up by default, They are configurable to be removed. I have removed them from the link

          Show
          cdias1 Chester Dias (Inactive) added a comment - http://52.45.117.194:8090/display/igbman/Home Those seem to be showing up by default, They are configurable to be removed. I have removed them from the link
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          The MySQL database does not seem to get populated with the user details.
          [~aloraine] Could you please verify that the existing Mysql DB used by the confluence is getting backed up correctly?
          [~aloraine] Please also provide me a repository for the confluence playbooks.

          Show
          cdias1 Chester Dias (Inactive) added a comment - The MySQL database does not seem to get populated with the user details. [~aloraine] Could you please verify that the existing Mysql DB used by the confluence is getting backed up correctly? [~aloraine] Please also provide me a repository for the confluence playbooks.
          cdias1 Chester Dias (Inactive) made changes -
          Description The IGB users and developers guides are hosted on https://wiki.transvar.org, which is running version 5.6.3 of Confluence.

          Spaces on this site which we need to maintain include:

          * IGB Developer's Guide
          * IGB Release Documentation
          * IGB User's Guide

          For this task, set up a new mirror site (using address wiki.bioviz.org) to host the above spaces only using the latest version of Confluence. In addition, develop a backup strategy that will allow us to set up a new Confluence site if required.
          The IGB users and developers guides are hosted on https://wiki.transvar.org, which is running version 5.6.3 of Confluence.

          Spaces on this site which we need to maintain include:

          * IGB Developer's Guide
          * IGB Release Documentation
          * IGB User's Guide

          For this task, set up a new mirror site (using address wiki.bioviz.org) to host the above spaces only using the latest version of Confluence. In addition, develop a backup strategy that will allow us to set up a new Confluence site if required.
          https://community.atlassian.com/t5/Confluence-questions/How-do-we-turn-off-the-blog/qaq-p/169157
          Hide
          ann.loraine Ann Loraine added a comment -

          I am working on the MySQL problem mentioned.
          attn: Chester Dias

          Show
          ann.loraine Ann Loraine added a comment - I am working on the MySQL problem mentioned. attn: Chester Dias
          Hide
          ann.loraine Ann Loraine added a comment -

          Please see the google drive

          • a new SQL file is uploaded
          • a new "zip" file is uploaded with "IGB Release Documentation" space
          Show
          ann.loraine Ann Loraine added a comment - Please see the google drive a new SQL file is uploaded a new "zip" file is uploaded with "IGB Release Documentation" space
          cdias1 Chester Dias (Inactive) made changes -
          Status In Progress [ 3 ] To-Do [ 10305 ]
          Show
          ann.loraine Ann Loraine added a comment - Repository: https://bitbucket.org/lorainelab/confluence-playbooks
          cdias1 Chester Dias (Inactive) made changes -
          Status To-Do [ 10305 ] In Progress [ 3 ]
          Hide
          cdias1 Chester Dias (Inactive) added a comment - - edited
          Show
          cdias1 Chester Dias (Inactive) added a comment - - edited http://ec2-23-20-156-126.compute-1.amazonaws.com:8090/display/igbman Please let me know if all looks fine
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          [~aloraine] Can you provide me all the httpd settings files from the existing server?

          Show
          cdias1 Chester Dias (Inactive) added a comment - [~aloraine] Can you provide me all the httpd settings files from the existing server?
          Show
          cdias1 Chester Dias (Inactive) added a comment - https://bitbucket.org/chesterdias/chester-local-confluence-playbooks/branch/IGBF-2513#diff
          Hide
          ann.loraine Ann Loraine added a comment -

          Will do.
          Please note:
          The existing server consists of multiple Docker containers, including one for Apache. Keep it in mind when looking at the settings.

          Show
          ann.loraine Ann Loraine added a comment - Will do. Please note: The existing server consists of multiple Docker containers, including one for Apache. Keep it in mind when looking at the settings.
          Hide
          ann.loraine Ann Loraine added a comment -

          Sorry for the delay. Configuration files sent via email.

          Show
          ann.loraine Ann Loraine added a comment - Sorry for the delay. Configuration files sent via email.
          ann.loraine Ann Loraine made changes -
          Sprint Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep [ 101, 102 ] Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep [ 101, 102, 103 ]
          ann.loraine Ann Loraine made changes -
          Rank Ranked higher
          cdias1 Chester Dias (Inactive) made changes -
          Story Points 4 6
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          All changes completed. Reverse proxy setup is working as expected.
          Please review: https://bitbucket.org/chesterdias/chester-local-confluence-playbooks/branch/IGBF-2513#diff

          Show
          cdias1 Chester Dias (Inactive) added a comment - All changes completed. Reverse proxy setup is working as expected. Please review: https://bitbucket.org/chesterdias/chester-local-confluence-playbooks/branch/IGBF-2513#diff
          cdias1 Chester Dias (Inactive) made changes -
          Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
          cdias1 Chester Dias (Inactive) made changes -
          Assignee Chester Dias [ cdias1 ] Ann Loraine [ aloraine ]
          ann.loraine Ann Loraine made changes -
          Sprint Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep [ 101, 102, 103 ] Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep, Fall 2: 28 Sep - 9 Oct [ 101, 102, 103, 104 ]
          ann.loraine Ann Loraine made changes -
          Rank Ranked higher
          ann.loraine Ann Loraine made changes -
          Sprint Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep, Fall 2: 28 Sep - 9 Oct [ 101, 102, 103, 104 ] Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep, Fall 2: 28 Sep - 9 Oct, Fall 3: Oct 12 - Oct 23 [ 101, 102, 103, 104, 106 ]
          ann.loraine Ann Loraine made changes -
          Rank Ranked higher
          ann.loraine Ann Loraine made changes -
          Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
          ann.loraine Ann Loraine made changes -
          Status First Level Review in Progress [ 10301 ] Ready for Pull Request [ 10304 ]
          ann.loraine Ann Loraine made changes -
          Assignee Ann Loraine [ aloraine ] Chester Dias [ cdias1 ]
          Hide
          ann.loraine Ann Loraine added a comment -

          Chester Dias - please submit PR.

          Show
          ann.loraine Ann Loraine added a comment - Chester Dias - please submit PR.
          Show
          cdias1 Chester Dias (Inactive) added a comment - https://bitbucket.org/chesterdias/chester-local-confluence-playbooks/pull-requests/1/igbf-2513/diff
          cdias1 Chester Dias (Inactive) made changes -
          Status Ready for Pull Request [ 10304 ] Pull Request Submitted [ 10101 ]
          cdias1 Chester Dias (Inactive) made changes -
          Assignee Chester Dias [ cdias1 ] Ann Loraine [ aloraine ]
          Hide
          ann.loraine Ann Loraine added a comment -

          Attn Chester Dias:

          The target of the pull request needs to be master branch on the upstream repository. Submit a new PR.

          Show
          ann.loraine Ann Loraine added a comment - Attn Chester Dias : The target of the pull request needs to be master branch on the upstream repository. Submit a new PR.
          ann.loraine Ann Loraine made changes -
          Status Pull Request Submitted [ 10101 ] Reviewing Pull Request [ 10303 ]
          ann.loraine Ann Loraine made changes -
          Status Reviewing Pull Request [ 10303 ] To-Do [ 10305 ]
          ann.loraine Ann Loraine made changes -
          Assignee Ann Loraine [ aloraine ] Chester Dias [ cdias1 ]
          Show
          cdias1 Chester Dias (Inactive) added a comment - Done: https://bitbucket.org/lorainelab/confluence-playbooks/pull-requests/1/igbf-2513/diff
          cdias1 Chester Dias (Inactive) made changes -
          Status To-Do [ 10305 ] In Progress [ 3 ]
          cdias1 Chester Dias (Inactive) made changes -
          Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
          cdias1 Chester Dias (Inactive) made changes -
          Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
          cdias1 Chester Dias (Inactive) made changes -
          Status First Level Review in Progress [ 10301 ] Ready for Pull Request [ 10304 ]
          cdias1 Chester Dias (Inactive) made changes -
          Status Ready for Pull Request [ 10304 ] Pull Request Submitted [ 10101 ]
          cdias1 Chester Dias (Inactive) made changes -
          Assignee Chester Dias [ cdias1 ] Ann Loraine [ aloraine ]
          Hide
          ann.loraine Ann Loraine added a comment -

          Merged.

          Show
          ann.loraine Ann Loraine added a comment - Merged.
          ann.loraine Ann Loraine made changes -
          Status Pull Request Submitted [ 10101 ] Reviewing Pull Request [ 10303 ]
          ann.loraine Ann Loraine made changes -
          Status Reviewing Pull Request [ 10303 ] Merged Needs Testing [ 10002 ]
          ann.loraine Ann Loraine made changes -
          Assignee Ann Loraine [ aloraine ]
          Hide
          ann.loraine Ann Loraine added a comment -

          Closing for now. Will migrate next month.

          Show
          ann.loraine Ann Loraine added a comment - Closing for now. Will migrate next month.
          ann.loraine Ann Loraine made changes -
          Status Merged Needs Testing [ 10002 ] Post-merge Testing In Progress [ 10003 ]
          ann.loraine Ann Loraine made changes -
          Resolution Done [ 10000 ]
          Status Post-merge Testing In Progress [ 10003 ] Closed [ 6 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              ann.loraine Ann Loraine
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: