Chester Dias (Inactive) added a comment - 24/Sep/20 9:06 AM Cloudwatch allows monitoring of NetworkIn and NetworkOut for EC2 instances. Based on that we can implement a lambda function that if triggered based on a threshold breach and would halt the http access by temporarily adding a restriction on the security group and starting a timer. on timer completion we can undo the changes on the security group. Looking into some more options

Ann Loraine added a comment - 24/Sep/20 9:22 AM Thanks! Please focus on limiting access to resources stored in S3 buckets.

Chester Dias (Inactive) added a comment - 24/Sep/20 9:30 AM I assumed the ticket was for limiting only EC2 but will look into S3 also

Chester Dias (Inactive) added a comment - 25/Sep/20 9:32 AM S3 by default populates only 2 below monitoring metrics for free in cloudwatch BucketSizeBytes NumberOfObjects Http monitoring is possible if we get the paid feature where `Request Metrics` will also be recorded on a small monthly charge parameters include below GetRequests PutRequests ListRequests AllRequests PostRequests DeleteRequests HeadRequests 4xxErrors 5xxErrors SelectRequests Costing information listed on the link: https://aws.amazon.com/cloudwatch/pricing/ cc: [~aloraine] I am looking into how to proceed with setting it up

Chester Dias (Inactive) added a comment - 02/Oct/20 9:47 AM Process There will be one alarm with 2 types of Notification. Alarm state when downloadbyte threshold is breached OK state when it falls During Alarm state below lambda will be triggered for that bucket using SNS import json import boto3 def lambda_handler(event, context): client = boto3.client('s3') policy = client.get_bucket_policy(Bucket='testbucket-monitorin') jsond=json.loads(policy[ "Policy" ]) jsond[ "Statement" ][0][ "Effect" ] = "Deny" response = client.put_bucket_policy(Bucket='testbucket-monitorin', Policy=str(json.dumps(jsond)),) After the bucket download is reduced in OK state import json import boto3 def lambda_handler(event, context): client = boto3.client('s3') policy = client.get_bucket_policy(Bucket='testbucket-monitorin') jsond=json.loads(policy[ "Policy" ]) jsond[ "Statement" ][0][ "Effect" ] = "Allow" response = client.put_bucket_policy(Bucket='testbucket-monitorin', Policy=str(json.dumps(jsond)),) Working on making this more generic to be used for any bucket

Chester Dias (Inactive) added a comment - 02/Oct/20 4:40 PM - edited Follow the steps in the order listed Steps to enable the download monitoring 1. Click on the S3 bucket to view objects 2. Click on Management Tab inside the bucket 3. Click on Metrics 4. Click on Edit icon(small Pencil icon) under the metrics filters 5. Enable Checkbox on the feature "Request metrics(10)" Note: This will take time to populate new monitoring data into the cloudwatch, This is also a paid feature Steps to create IAM role for AWS Lambda to have access to S3 1. Open IAM 2. Roles 3. Click Create Role Button 4. In Choose a use case select 'Lambda'.This Allows Lambda functions to call AWS services on your behalf. 5. Click next button to set permissions 6. Attach below 2 policies a. AmazonS3FullAccess b. AWSLambdaBasicExecutionRole Policy b is only for monitoring/logging the Lambda function execution. This can be removed if not needed 7. Ensure to give a valid name. Steps to create 2 Topics 1. Open Amazon SNS 2. Click on Topics 3. Create Topic with a name: Topic-Restrict-Bucket 4. Create Topic with a name: Topic-Normalise-Bucket-Condition Steps to create 2 Lambda Functions 1. Open AWS Lambda 2. Click on the 'Create function' Button 3. Radio Button Author from scratch 4. a. Give the function name: Restrict-Bucket b. Runtime: Python3.8 c. Permissions i. In Execution Role, click use an existing role: select the role you had created earlier above mentioned steps in IAM 5. Click Create Function Button 6. In the conifguration tab that opens for the newly created function goto Designer and "Add Trigger" a. Select trigger as SNS b. Select SNS Topic: Topic-Restrict-Bucket c. Enabe the trigger and add 7. In the Designer click on the Restrict-Bucket Lambda Icon 8. Add the following in the "Function Code" import json import boto3 def lambda_handler(event, context): client = boto3.client('s3') snsjson=event[ "Records" ][0][ "Sns" ] messagejson=json.loads(snsjson[ "Message" ]) bucket_name=messagejson[ "Trigger" ][ "Dimensions" ][0][ "value" ] policy = client.get_bucket_policy(Bucket=bucket_name) jsond=json.loads(policy[ "Policy" ]) jsond[ "Statement" ][0][ "Effect" ] = "Deny" response = client.put_bucket_policy(Bucket=bucket_name, Policy=str(json.dumps(jsond)),) 9. Click on Deploy (This is a save and apply button) 10. Do the steps 2 to 9 for unrestricting the bucket access lambda function. with the below changes in the names 4 a. Give the function name: Unrestrict-Bucket 6 b. Select SNS Topic: Topic-Normalise-Bucket-Condition import json import boto3 def lambda_handler(event, context): client = boto3.client('s3') snsjson=event[ "Records" ][0][ "Sns" ] messagejson=json.loads(snsjson[ "Message" ]) bucket_name=messagejson[ "Trigger" ][ "Dimensions" ][0][ "value" ] policy = client.get_bucket_policy(Bucket=bucket_name) jsond=json.loads(policy[ "Policy" ]) jsond[ "Statement" ][0][ "Effect" ] = "Allow" response = client.put_bucket_policy(Bucket=bucket_name, Policy=str(json.dumps(jsond)),) 11. Click deploy after copying the python code Steps for Adding 2 subscription to the topic 1. Go to Amazon SNS 2. Navigate to Subscriptions 3. Click on Create Subscriptions 4. a. Topic arn: search for Topic-Restrict-Bucket it will give the exact arn b. Protocol: AWS Lambda c. Endpoint: Search for Restrict-Bucket it will give the exact arn 5. Create Subscription 6. Do steps 3 to 5 withe the following changes 4 a. Topic arn: search for Topic-Normalise-Bucket-Condition it will give the exact arn b. Protocol: AWS Lambda c. Endpoint: Search for Unrestrict-Bucket it will give the exact arn The above will provide ability to edit the bucket policy automatically when triggered in a situation of download threshold breach. The above can be used by any bucket having the same type of policy.

Chester Dias (Inactive) added a comment - 02/Oct/20 4:41 PM - edited The Below steps will have to be done for each bucket The below steps are for setting up The threshold triggers per bucket. 1. Go to AWS CloudWatch 2. Alarms 3. Create Alarms 4. Select metric 5. In the All metrics select S3 6. For a bucket name to be monitored select the metrics BytesDownloaded click the "Select metric" button 7. Set the Period for monitoring. This would be the duration of increased Set it to 5 min for testing. This can be edited later 8. Conditions a. Static b. Greater c. Add the threshold limit in Bytes. 9. Additional configuration a. Missing data treatment: Treat missing data as good(Since if no one performed download we dont want threshold alarm to trigger) 10. Next a. In alarm i. Select existing SNS topic. ii. Send a notification to...: Topic-Restrict-Bucket b. Click Add notification for adding another OK state i. Click OK radio button ii. Select existing SNS topic. iii. Send a notification to...: Topic-Normalise-Bucket-Condition 11. Give alarm name as: ALARM_FOR_BUCKETNAME Next. DONOT give a description 12. Next 13. Create Alarm

Chester Dias (Inactive) added a comment - 02/Oct/20 5:15 PM [~aloraine] Please Review. The steps mentioned in the above 2 comments. I have mentioned the exact Process in the image attached

Ann Loraine added a comment - 10/Oct/20 9:57 AM Thank you for the investigation Chester Dias . We now have the background information needed to proceed.