Details
-
Type: Task
-
Status: Closed (View Workflow)
-
Priority: Major
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: None
-
Labels:None
-
Story Points:1.5
-
Epic Link:
-
Sprint:Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug, Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep, Fall 2: 28 Sep - 9 Oct
Description
When we stand up a new appstore with all new code, we often want to copy data (msyql database and s3 bucket contents) to a new database and new s3 bucket to be used by the new appstore.
Currently, we do not have a good way to copy the s3 bucket contents. (We recently developed code for copying the mysql database, however - see IGBF-2421.)
For this task, we'll write some code that copies data from an existing s3 bucket into a newly created one for the new ec2 to use.
Let's add some new tasks that accomplish this goal to the end of main.yml in role S3 in appstore playbooks.
The new tasks should check to see if a variable first_s3_bucket_name exists. This is to ensure that if the user is not actually trying to copy over some S3 contents, the tasks will not run.
If variable first_s3_bucket_name exists and first_s3_bucket_name does not equal s3_bucket_name and s3_bucket_name is empty, copy the contents of first_s3_bucket_name into s3_bucket_name.
Note: I'm entirely sure that the mysql database contains references to particular s3 buckets or if it is using relative paths of some type. Someone needs to check this. If it contains references to the s3 bucket name, we will need to add some extra logic to deal with that.
Attachments
Issue Links
- blocks
-
IGBF-2426 Write next draft database migration code
- Closed
Activity
Field | Original Value | New Value |
---|---|---|
Epic Link | IGBF-2323 [ 18477 ] |
Rank | Ranked higher |
Assignee | Ann Loraine [ aloraine ] |
Story Points | 1 | 2 |
Sprint | Summer 1: 8 Jun - 19 Jun [ 96 ] |
Assignee | Chester Dias [ cdias1 ] |
Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Summary | Add new tasks to role S3 to copy contents of first_s3_bucket to the newly created on | Add new tasks to role S3 to copy contents of first_s3_bucket to newly created one |
Summary | Add new tasks to role S3 to copy contents of first_s3_bucket to newly created one | Add new tasks to role S3 to copy contents of first_s3_bucket_name to newly created s3_bucket_name |
Description |
When we stand up a new appstore with all new code, we often want to migrate data (msyql database and s3 bucket contents) over to be used by the appstore.
Currently, we do not have a good way to do this. For this task, we'll write some code that copies data from the existing s3 bucket into a newly created one. Add some new tasks to the end of main.yml in role S3 in appstore playbooks. The new tasks should check to see if a variable first_s3_bucket_name exists. If variable first_s3_bucket_name exists *and* first_s3_bucket_name does not equal s3_bucket_name *and* s3_bucket_name is empty, copy the contents of first_s3_bucket_name into s3_bucket_name. Note: I'm entirely sure that the mysql database contains references to particular s3 buckets or if it is using relative paths of some type. Someone needs to check this. If it contains references to the s3 bucket name, we will need to add some extra logic to deal with that. |
When we stand up a new appstore with all new code, we often want to copy data (msyql database and s3 bucket contents) to a new database and new s3 bucket to be used by the new appstore.
Currently, we do not have a good way to copy the s3 bucket contents. (We recently developed code for copying the mysql database, however - see For this task, we'll write some code that copies data from an existing s3 bucket into a newly created one for the new ec2 to use. Let's add some new tasks that accomplish this goal to the end of main.yml in role S3 in appstore playbooks. The new tasks should check to see if a variable first_s3_bucket_name exists. This is to ensure that if the user is not actually trying to copy over some S3 contents, the tasks will not run. If variable first_s3_bucket_name exists *and* first_s3_bucket_name does not equal s3_bucket_name *and* s3_bucket_name is empty, copy the contents of first_s3_bucket_name into s3_bucket_name. Note: I'm entirely sure that the mysql database contains references to particular s3 buckets or if it is using relative paths of some type. Someone needs to check this. If it contains references to the s3 bucket name, we will need to add some extra logic to deal with that. |
Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
Assignee | Chester Dias [ cdias1 ] | Ann Loraine [ aloraine ] |
Story Points | 2 | 1.5 |
Story Points | 1.5 | 1 |
Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
Status | First Level Review in Progress [ 10301 ] | To-Do [ 10305 ] |
Assignee | Ann Loraine [ aloraine ] | Chester Dias [ cdias1 ] |
Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
Assignee | Chester Dias [ cdias1 ] | Ann Loraine [ aloraine ] |
Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
Status | First Level Review in Progress [ 10301 ] | To-Do [ 10305 ] |
Assignee | Ann Loraine [ aloraine ] | Chester Dias [ cdias1 ] |
Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
Assignee | Chester Dias [ cdias1 ] | Ann Loraine [ aloraine ] |
Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
Status | First Level Review in Progress [ 10301 ] | To-Do [ 10305 ] |
Assignee | Ann Loraine [ aloraine ] | Chester Dias [ cdias1 ] |
Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
Assignee | Chester Dias [ cdias1 ] | Ann Loraine [ aloraine ] |
Assignee | Ann Loraine [ aloraine ] | Chester Dias [ cdias1 ] |
Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
Status | First Level Review in Progress [ 10301 ] | Ready for Pull Request [ 10304 ] |
Status | Ready for Pull Request [ 10304 ] | Pull Request Submitted [ 10101 ] |
Status | Pull Request Submitted [ 10101 ] | Reviewing Pull Request [ 10303 ] |
Status | Reviewing Pull Request [ 10303 ] | Merged Needs Testing [ 10002 ] |
Assignee | Chester Dias [ cdias1 ] |
Link | This issue blocks IGBF-2371 [ IGBF-2371 ] |
Sprint | Summer 1: 8 Jun - 19 Jun [ 96 ] | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 July [ 96, 97 ] |
Rank | Ranked higher |
Assignee | Ann Loraine [ aloraine ] |
Sprint | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul [ 96, 97 ] | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 3 Jul - 14 Jul [ 96, 97, 98 ] |
Rank | Ranked higher |
Status | Merged Needs Testing [ 10002 ] | Post-merge Testing In Progress [ 10003 ] |
Sprint | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul [ 96, 97, 98 ] | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul [ 96, 97, 98, 99 ] |
Rank | Ranked higher |
Status | Post-merge Testing In Progress [ 10003 ] | To-Do [ 10305 ] |
Assignee | Ann Loraine [ aloraine ] | Chester Dias [ cdias1 ] |
Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
Assignee | Chester Dias [ cdias1 ] | Ann Loraine [ aloraine ] |
Story Points | 1 | 1.5 |
Sprint | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul [ 96, 97, 98, 99 ] | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 29 Jul - 12 Aug [ 96, 97, 98, 99, 100 ] |
Rank | Ranked higher |
Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
Status | First Level Review in Progress [ 10301 ] | To-Do [ 10305 ] |
Assignee | Ann Loraine [ aloraine ] | Chester Dias [ cdias1 ] |
Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Assignee | Chester Dias [ cdias1 ] | Ann Loraine [ aloraine ] |
Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
Status | First Level Review in Progress [ 10301 ] | Ready for Pull Request [ 10304 ] |
Assignee | Ann Loraine [ aloraine ] | Chester Dias [ cdias1 ] |
Status | Ready for Pull Request [ 10304 ] | Pull Request Submitted [ 10101 ] |
Status | Pull Request Submitted [ 10101 ] | Reviewing Pull Request [ 10303 ] |
Status | Reviewing Pull Request [ 10303 ] | To-Do [ 10305 ] |
Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
Assignee | Chester Dias [ cdias1 ] | Ann Loraine [ aloraine ] |
Assignee | Ann Loraine [ aloraine ] | Chester Dias [ cdias1 ] |
Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
Status | First Level Review in Progress [ 10301 ] | To-Do [ 10305 ] |
Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
Assignee | Chester Dias [ cdias1 ] | Ann Loraine [ aloraine ] |
Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
Status | First Level Review in Progress [ 10301 ] | Ready for Pull Request [ 10304 ] |
Status | Ready for Pull Request [ 10304 ] | Pull Request Submitted [ 10101 ] |
Status | Pull Request Submitted [ 10101 ] | Reviewing Pull Request [ 10303 ] |
Status | Reviewing Pull Request [ 10303 ] | Merged Needs Testing [ 10002 ] |
Assignee | Ann Loraine [ aloraine ] |
Sprint | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug [ 96, 97, 98, 99, 100 ] | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug, Summer 6: 17 Aug - 28 Aug [ 96, 97, 98, 99, 100, 101 ] |
Rank | Ranked higher |
Status | Merged Needs Testing [ 10002 ] | Post-merge Testing In Progress [ 10003 ] |
Status | Post-merge Testing In Progress [ 10003 ] | To-Do [ 10305 ] |
Assignee | Chester Dias [ cdias1 ] |
Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
Assignee | Chester Dias [ cdias1 ] | Ann Loraine [ aloraine ] |
Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
Status | First Level Review in Progress [ 10301 ] | Ready for Pull Request [ 10304 ] |
Status | Ready for Pull Request [ 10304 ] | Pull Request Submitted [ 10101 ] |
Status | Pull Request Submitted [ 10101 ] | Reviewing Pull Request [ 10303 ] |
Status | Reviewing Pull Request [ 10303 ] | Merged Needs Testing [ 10002 ] |
Assignee | Ann Loraine [ aloraine ] |
Sprint | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug, Summer 6: 17 Aug - 28 Aug [ 96, 97, 98, 99, 100, 101 ] | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug, Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep [ 96, 97, 98, 99, 100, 101, 102 ] |
Rank | Ranked higher |
Sprint | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug, Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep [ 96, 97, 98, 99, 100, 101, 102 ] | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug, Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep [ 96, 97, 98, 99, 100, 101, 102, 103 ] |
Rank | Ranked higher |
Status | Merged Needs Testing [ 10002 ] | Post-merge Testing In Progress [ 10003 ] |
Status | Post-merge Testing In Progress [ 10003 ] | To-Do [ 10305 ] |
Assignee | Chester Dias [ cdias1 ] |
Comment | [ Temporary policy is not getting removed. ] |
Assignee | Chester Dias [ cdias1 ] | Ann Loraine [ aloraine ] |
Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Sprint | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug, Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep [ 96, 97, 98, 99, 100, 101, 102, 103 ] | Summer 1: 8 Jun - 19 Jun, Summer 2: 22 Jun - 3 Jul, Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug, Summer 6: 17 Aug - 28 Aug, Summer 7: 31 Aug - 11 Sep, Fall 1: 14 Sep - 25 Sep, Fall 2: 28 Sep - 9 Oct [ 96, 97, 98, 99, 100, 101, 102, 103, 104 ] |
Rank | Ranked higher |
Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
Status | First Level Review in Progress [ 10301 ] | Ready for Pull Request [ 10304 ] |
Status | Ready for Pull Request [ 10304 ] | Pull Request Submitted [ 10101 ] |
Status | Pull Request Submitted [ 10101 ] | Reviewing Pull Request [ 10303 ] |
Status | Reviewing Pull Request [ 10303 ] | Merged Needs Testing [ 10002 ] |
Status | Merged Needs Testing [ 10002 ] | Post-merge Testing In Progress [ 10003 ] |
Resolution | Done [ 10000 ] | |
Status | Post-merge Testing In Progress [ 10003 ] | Closed [ 6 ] |
Assignee | Ann Loraine [ aloraine ] | Chester Dias [ cdias1 ] |
Suggestion: Look at Ansible Galaxy for code we can import that does this. I bet some-one has written something to do this. Basically, the job is to mirror an s3 bucket. Surely this has been written already!