Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2309

Make storing encrypted keys more convenient

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Currently, the ansible playbooks we are using to deploy App Store instances use a shared variables file "all.yml" to store variables used in multiple roles.

      This file contains values for things like secret keys that need to be encrypted. It also contains other variables that do not require encryption.

      Combining these two categories of variables into a single file requires us to interleave encrypted and unencrypted values. This makes it awkward to update encrypted values when required. For example, if we delete an AWS access credential and replace it with something new, we will then need to manually encrypt the replacement values using ansible-vault and copy the cypher-text into all.yml. We must do this multiple times for each variable, individually.

      To make this process less error-prone, let's create two files:

      all.yml - unencrypted variables
      secrets.yml - encrypted variables

      In this scheme, the entire file "secrets.yml" will be encrypted, not individual values one-by-one as we are doing now.

      The file "secrets.yml" will never reside in the git repository itself. It will only be created upon provisioning the control node, which we will do manually. To protect the file, we will add "secrets.yml" as an entry to the .gitignore file for the reposiotry.

      To remind us which variables need to be defined, we will create another file called "protected_variables.txt" that simple lists each variable, e.g.,

      AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY
      ... (and son on)

      To provision the control node, the user will create the "secrets.yml" file using "protected_variables.txt" as a guide. Next, the user will encrypt the entire file and transfer it to the control node.

      As a reminder, to encrypt an entire file using ansible-vault, all we need to do is run:

      • ansible-vault encrypt secrets.yml

      To decrypt it (for editing), run:

      • ansible-vault decrypt secrets.yml

      To run the playbook, the user can provide a file containing a password and indicate it the location of same in an ansible settings file. Alternatively, the user can enter the password manually by running ansible-playbook with --ask-vault-pass flag.

      For a quick refresher on how this works, watch:

      https://linuxacademy.com/cp/courses/lesson/course/4489/lesson/7/module/366

        Attachments

          Activity

          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          I have an idea of this approach... will try it out...
          --ask-vault-pass has to be passed as command line argument will ask user to enter vault password everytime a playbook is executed

          Please let me know if that is what is needed. In that case I can rewerite and override the existing ansible.cfg and that will solve both this ticket and https://jira.transvar.org/browse/IGBF-2311

          Show
          cdias1 Chester Dias (Inactive) added a comment - I have an idea of this approach... will try it out... --ask-vault-pass has to be passed as command line argument will ask user to enter vault password everytime a playbook is executed Please let me know if that is what is needed. In that case I can rewerite and override the existing ansible.cfg and that will solve both this ticket and https://jira.transvar.org/browse/IGBF-2311
          Hide
          ann.loraine Ann Loraine added a comment -

          I think it is OK to run playbooks using --ask-vault-pass option.

          Show
          ann.loraine Ann Loraine added a comment - I think it is OK to run playbooks using --ask-vault-pass option.
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          this makes it simple for me then... I will complete this ticket and https://jira.transvar.org/browse/IGBF-2311 at the same time on thursday

          Show
          cdias1 Chester Dias (Inactive) added a comment - this makes it simple for me then... I will complete this ticket and https://jira.transvar.org/browse/IGBF-2311 at the same time on thursday
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          The format for the secrets.yml is

          ##############################AWS EC2 CREATION PARAMETES############################

          1. DONOT MODIFY THESE UNLESS NEEDED
            AWS_ACCESS_KEY_ID:
            AWS_SECRET_ACCESS_KEY:
            rds_admin_user:
            rds_admin_password:
            settings_SECRET_KEY:
            settings_EMAIL_HOST_PASSWORD:
            ...
          Show
          cdias1 Chester Dias (Inactive) added a comment - The format for the secrets.yml is — ##############################AWS EC2 CREATION PARAMETES############################ DONOT MODIFY THESE UNLESS NEEDED AWS_ACCESS_KEY_ID: AWS_SECRET_ACCESS_KEY: rds_admin_user: rds_admin_password: settings_SECRET_KEY: settings_EMAIL_HOST_PASSWORD: ...
          Hide
          cdias1 Chester Dias (Inactive) added a comment - - edited

          To encrypt run
          ansible-vault encrypt secrets.yml

          To decrypt it (for editing), run:
          ansible-vault decrypt secrets.yml

          There is no need to decrypt the secrets file explicitly before the execution of the ansible playbooks.

          In order to pass the password use the below command
          ansible-playbook main.yml --extra-vars "ec2_name=DEV-AppStore123" --ask-vault-pass

          It will throw a prompt to enter the password

          Show
          cdias1 Chester Dias (Inactive) added a comment - - edited To encrypt run ansible-vault encrypt secrets.yml To decrypt it (for editing), run: ansible-vault decrypt secrets.yml There is no need to decrypt the secrets file explicitly before the execution of the ansible playbooks. In order to pass the password use the below command ansible-playbook main.yml --extra-vars "ec2_name=DEV-AppStore123" --ask-vault-pass It will throw a prompt to enter the password
          Hide
          ann.loraine Ann Loraine added a comment -

          Merged into master branch.
          Added .gitignore separately.

          Show
          ann.loraine Ann Loraine added a comment - Merged into master branch. Added .gitignore separately.

            People

            • Assignee:
              cdias1 Chester Dias (Inactive)
              Reporter:
              ann.loraine Ann Loraine
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: