Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2531

Investigate: Lambda function to shuts down EC2s at 18:30 every day

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      We have many EC2's used for development purposes.

      EC2's with tag "Stack" equal to "Develop" are all used for development purposes only.

      To save money, we should change their status to "Stopped" at the end of very workday - 6:30 pm Easter Standard Time.

      Currently, Dr. Loraine manually does this - when she remembers to do it.

      We need an automated approach instead.

      A lambda function attached to the Loraine Lab account would probably be the best approach - it is simple and relatively easy to set up.

      For example, according to this documentation – https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/ – you can create a lambda function that would then be triggered by a timer configured in cloudwatch.

      Alternatively, you can do something like this - read: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/UsingAlarmActions.html#AddingStopActions

      For this task, investigate the best way to automate stopping ECs with Stack equal to Develop. It should be super simple and easy for Dr. Loraine to manage and maintain.

      Give an overview of the solution here.

        Attachments

          Issue Links

            Activity

            Hide
            ann.loraine Ann Loraine added a comment -

            For review, can you ask one of the other students to try doing this in their account?

            Show
            ann.loraine Ann Loraine added a comment - For review, can you ask one of the other students to try doing this in their account?
            Hide
            chirag24 Chirag Chandrahas Shetty (Inactive) added a comment -

            Link: https://github.com/chirag06/auto_shutdown_ec2_instances
            Steps for testing:
            1) Login to your own AWS account where you have full admin privileges
            2) Create two ec2 instances and add tag with key: development and value: True
            3) Follow the readme file of the GitHub repository.

            Show
            chirag24 Chirag Chandrahas Shetty (Inactive) added a comment - Link: https://github.com/chirag06/auto_shutdown_ec2_instances Steps for testing: 1) Login to your own AWS account where you have full admin privileges 2) Create two ec2 instances and add tag with key: development and value: True 3) Follow the readme file of the GitHub repository.
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Suggestions:

            • Under "Step 1" put "Add Tags...to skip this" after "Select AmazonEC2...and click Next" (tags configuration comes after selecting a policy
            • Under "Step 2", item 3, I would recommend using a bulleted sub-list to show the 3 configurations that need to be done (function name, run-time, IAM role selection)
            • Under "Step 2", should item number be "Choose Deploy" instead of "Choose Save" ?
            • Add some tips on how to run the lambda function from the console just to check that it is working properly (click "Test" and don't worry about editing the boiler-plate JSON string shown)

            I followed the instructions but the function did not shut down the Develop instances. The code I used was:

            import boto3
            
            def lambda_handler(event, context):
                ec2 = boto3.resource('ec2')
                filters = [{
                        'Name': 'tag:Stack',
                        'Values': ['Develop']
                    },
                    {
                        'Name': 'instance-state-name', 
                        'Values': ['running']
                    }
                ]
                instances = ec2.instances.filter(Filters=filters)
                RunningInstances = [instance.id for instance in instances]
                for instance_id in RunningInstances:
                    ec2.instances.stop(InstanceIds=[instance_id])
            
            Show
            ann.loraine Ann Loraine added a comment - - edited Suggestions: Under "Step 1" put "Add Tags...to skip this" after "Select AmazonEC2...and click Next" (tags configuration comes after selecting a policy Under "Step 2", item 3, I would recommend using a bulleted sub-list to show the 3 configurations that need to be done (function name, run-time, IAM role selection) Under "Step 2", should item number be "Choose Deploy" instead of "Choose Save" ? Add some tips on how to run the lambda function from the console just to check that it is working properly (click "Test" and don't worry about editing the boiler-plate JSON string shown) I followed the instructions but the function did not shut down the Develop instances. The code I used was: import boto3 def lambda_handler(event, context): ec2 = boto3.resource('ec2') filters = [{ 'Name': 'tag:Stack', 'Values': ['Develop'] }, { 'Name': 'instance-state-name', 'Values': ['running'] } ] instances = ec2.instances.filter(Filters=filters) RunningInstances = [instance.id for instance in instances] for instance_id in RunningInstances: ec2.instances.stop(InstanceIds=[instance_id])
            Hide
            ann.loraine Ann Loraine added a comment -

            This worked:

            import boto3
            ec2 = boto3.resource('ec2')
            def lambda_handler(event, context):
                filters = [{
                        'Name': 'tag:Stack',
                        'Values': ['Develop']
                    },
                    {
                        'Name': 'instance-state-name', 
                        'Values': ['running']
                    }
                ]
                instances = ec2.instances.filter(Filters=filters)
                RunningInstances = [instance.id for instance in instances]
                for instance_id in RunningInstances:
                    ec2.instances.stop(InstanceIds=[instance_id])
            

            The only different is placement of this line:

            ec2 = boto3.resource('ec2')
            
            Show
            ann.loraine Ann Loraine added a comment - This worked: import boto3 ec2 = boto3.resource('ec2') def lambda_handler(event, context): filters = [{ 'Name': 'tag:Stack', 'Values': ['Develop'] }, { 'Name': 'instance-state-name', 'Values': ['running'] } ] instances = ec2.instances.filter(Filters=filters) RunningInstances = [instance.id for instance in instances] for instance_id in RunningInstances: ec2.instances.stop(InstanceIds=[instance_id]) The only different is placement of this line: ec2 = boto3.resource('ec2')
            Hide
            ann.loraine Ann Loraine added a comment -

            Request:

            • Please devise cron string(s) that handle daylight savings time in the eastern US.

            During the winter months, GMT is five hours ahead of eastern US (Charlotte) time. During the summer months (e.g., now), GMT is four hours ahead of eastern US time.

            Show
            ann.loraine Ann Loraine added a comment - Request: Please devise cron string(s) that handle daylight savings time in the eastern US. During the winter months, GMT is five hours ahead of eastern US (Charlotte) time. During the summer months (e.g., now), GMT is four hours ahead of eastern US time.
            Hide
            chirag24 Chirag Chandrahas Shetty (Inactive) added a comment - - edited

            Hi Dr. [~aloraine],
            Thank you for the suggestions. I have done the changes in the readme file. I do not know why the lambda function is not consistent with the placement of that line. Will look into it today.

            Regarding the cron strings, the problem is that the dates daylight saving dates are not fixed, so it won't be possible to do that by only using cron strings. We can have one more lambda function that will be changing the Cloudwatch schedule every day according to the local time of our console. But this would incur an extra cost as cloud-watch, and lambda function would be running every day.

            Thank you!

            Show
            chirag24 Chirag Chandrahas Shetty (Inactive) added a comment - - edited Hi Dr. [~aloraine] , Thank you for the suggestions. I have done the changes in the readme file. I do not know why the lambda function is not consistent with the placement of that line. Will look into it today. Regarding the cron strings, the problem is that the dates daylight saving dates are not fixed, so it won't be possible to do that by only using cron strings. We can have one more lambda function that will be changing the Cloudwatch schedule every day according to the local time of our console. But this would incur an extra cost as cloud-watch, and lambda function would be running every day. Thank you!
            Hide
            ann.loraine Ann Loraine added a comment -

            Thank you very much for the explanation. It is very helpful!

            Show
            ann.loraine Ann Loraine added a comment - Thank you very much for the explanation. It is very helpful!
            Hide
            ann.loraine Ann Loraine added a comment -

            Confirming: EC2s in the loraine lab account got shut down at the proper time via cloudwatch.
            Thank you Chirag Chandrahas Shetty for the very helpful code!

            Show
            ann.loraine Ann Loraine added a comment - Confirming: EC2s in the loraine lab account got shut down at the proper time via cloudwatch. Thank you Chirag Chandrahas Shetty for the very helpful code!
            Hide
            ann.loraine Ann Loraine added a comment -

            need to update cloudwatch for daylight savings time

            Show
            ann.loraine Ann Loraine added a comment - need to update cloudwatch for daylight savings time
            Hide
            chirag24 Chirag Chandrahas Shetty (Inactive) added a comment -

            [~aloraine] We can change the string in step 3 from "30 22 * * ? *" to "30 23 * * ? *".

            Show
            chirag24 Chirag Chandrahas Shetty (Inactive) added a comment - [~aloraine] We can change the string in step 3 from "30 22 * * ? *" to "30 23 * * ? *".
            Hide
            ann.loraine Ann Loraine added a comment -

            Thank you - it is done!

            Show
            ann.loraine Ann Loraine added a comment - Thank you - it is done!

              People

              • Assignee:
                chirag24 Chirag Chandrahas Shetty (Inactive)
                Reporter:
                ann.loraine Ann Loraine
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: