Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2307

Investigate: Configure playbooks to ensure control node's IP address is added to the security group for the RDS host

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Ansible control node needs to run database management commands on the RDS host used by app store instances.

      To enable this, the control node's private IP address must be added to the RDS host's security group. In addition, the app store EC2's private IP address must also be added to the RDS host's security group.

      Currently, this has to be done manually when provisioning the control node.

      For this task, investigate what would be required to automate this step. Could we include the task of adding the control node's private IP address to the RDS's security group?

      Note: When adding a new inbound rule to the security group, we should assign a description to it. Please use the "Name" tag of the Ansible Control node in the description assigned to the inbound rule so that we can easily recognize it when inspecting security groups for the RDS host.

        Attachments

          Activity

          Hide
          ann.loraine Ann Loraine added a comment -

          Un-assigning this for now as this task should be done after we finalize workflow for setting up the control node.

          Show
          ann.loraine Ann Loraine added a comment - Un-assigning this for now as this task should be done after we finalize workflow for setting up the control node.
          Hide
          ann.loraine Ann Loraine added a comment - - edited

          See:

          https://bitbucket.org/aloraine/appstore-playbooks/branch/IGBF-2307#diff

          I wrote a sample playbook (rds.yml) that discovers the security group attached to an RDS instance, tries to get information about it, and also tries to add an EC's private IP address to it. (The EC2 IP address is also discovered.)

          However, this failed. See attached sample output.

          Maybe there is a bug with how my system is configured?

          Moving this to "needs first level review" for Chester Dias to take a look at.

          For now, as a work-around, we'll use aws command line to add the new inbound rule to the security group.

          Show
          ann.loraine Ann Loraine added a comment - - edited See: https://bitbucket.org/aloraine/appstore-playbooks/branch/IGBF-2307#diff I wrote a sample playbook (rds.yml) that discovers the security group attached to an RDS instance, tries to get information about it, and also tries to add an EC's private IP address to it. (The EC2 IP address is also discovered.) However, this failed. See attached sample output. Maybe there is a bug with how my system is configured? Moving this to "needs first level review" for Chester Dias to take a look at. For now, as a work-around, we'll use aws command line to add the new inbound rule to the security group.
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          Issues Noticed
          Incorrect quotes we used to reference the json attributes
          for example
          incorrect quotes on line 53 resulted in text returned with single quotes(encasing the sec group text) acting as a single string value that made no meaning that is used to get another variable that gets corrupted due to the quote characters.

          The var couldnt be assigned a whole Json for some reason(not too sure why?)I dont think vars are assigned that way. In order to assign and use variable set_fact has to be used.
          https://docs.ansible.com/ansible/latest/modules/set_fact_module.html
          where data is stored as key value pair

          with all those changes I have the working code for this funcitonality

          The group_id attribute of module ec2_group can only be used when a delete operation is being used to remove rules on all other cases the name of the sec group is needed

          purge_rules, purge_rules_egress needs to be set to 'no' To prevent the existing rules from being dropped.

          Show
          cdias1 Chester Dias (Inactive) added a comment - Issues Noticed Incorrect quotes we used to reference the json attributes for example incorrect quotes on line 53 resulted in text returned with single quotes(encasing the sec group text) acting as a single string value that made no meaning that is used to get another variable that gets corrupted due to the quote characters. The var couldnt be assigned a whole Json for some reason(not too sure why?)I dont think vars are assigned that way. In order to assign and use variable set_fact has to be used. https://docs.ansible.com/ansible/latest/modules/set_fact_module.html where data is stored as key value pair with all those changes I have the working code for this funcitonality The group_id attribute of module ec2_group can only be used when a delete operation is being used to remove rules on all other cases the name of the sec group is needed purge_rules, purge_rules_egress needs to be set to 'no' To prevent the existing rules from being dropped.
          Hide
          ann.loraine Ann Loraine added a comment -

          Thank you I will take a look!

          cc: Chester Dias

          Show
          ann.loraine Ann Loraine added a comment - Thank you I will take a look! cc: Chester Dias
          Hide
          ann.loraine Ann Loraine added a comment -

          Using recommended changes I am able to get the code to run.
          For the next steps, I will add a new play to control node playbook to add the control node's private IP to the RDS host.
          Thanks Chester Dias!

          Show
          ann.loraine Ann Loraine added a comment - Using recommended changes I am able to get the code to run. For the next steps, I will add a new play to control node playbook to add the control node's private IP to the RDS host. Thanks Chester Dias !
          Hide
          ann.loraine Ann Loraine added a comment -

          Because the basic code has been reviewed already, I have the changes into the master branch.
          This ticket is now ready for final testing.

          Suggestions for how to test it:

          • Create an RDS host in your AWS account
          • Attach a security group to it
          • Create control_node_vars.yml following instructions in example_vars.yml
          • Run the playbook control_node.yml - see instructions at the top of the playbook file

          Note that the first time you run the playbook, it may fail because the ssh daemon needs a bit of time to start up before ansible can proceed with installing software on the target VM.

          If that happens, wait a minute and run it a second time.

          To check that the security group was properly added, check the AWS console. If it is there, you can mark this as complete.

          However, please note that it will again will fail (by design) after this step when it gets to the task of cloning the private ansible playbooks repository onto the control node vm. At that point, you would need to copy the newly provisioned control node VM's public key into the bitbucket account of the user whose repository you are cloning. (You define the bitbucket user and repository in control_node_vars.yml.) Once the key is copied, you should be able to run the playbook once again and observe no errors.

          However it is not necessary to do this as this ticket relates only to the security group modification. It is fine to skip the remaining plays.

          Show
          ann.loraine Ann Loraine added a comment - Because the basic code has been reviewed already, I have the changes into the master branch. This ticket is now ready for final testing. Suggestions for how to test it: Create an RDS host in your AWS account Attach a security group to it Create control_node_vars.yml following instructions in example_vars.yml Run the playbook control_node.yml - see instructions at the top of the playbook file Note that the first time you run the playbook, it may fail because the ssh daemon needs a bit of time to start up before ansible can proceed with installing software on the target VM. If that happens, wait a minute and run it a second time. To check that the security group was properly added, check the AWS console. If it is there, you can mark this as complete. However, please note that it will again will fail (by design) after this step when it gets to the task of cloning the private ansible playbooks repository onto the control node vm. At that point, you would need to copy the newly provisioned control node VM's public key into the bitbucket account of the user whose repository you are cloning. (You define the bitbucket user and repository in control_node_vars.yml.) Once the key is copied, you should be able to run the playbook once again and observe no errors. However it is not necessary to do this as this ticket relates only to the security group modification. It is fine to skip the remaining plays.

            People

            • Assignee:
              ann.loraine Ann Loraine
              Reporter:
              ann.loraine Ann Loraine
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: