About Me

DevOps@RajeshKumar.XYZ

Course Introduction

Module Summary

• Auditing IAM
• Authentication and Authorization
• Users
• Groups
• Roles
• Policies
• Combined and summary CLI examples

Authentication and Authorization in AWS

Objects in IAM

Users in IAM

• Represent actual person or application
• Can be member of multiple groups
• Can have inline or managed policies
• Different methods of authentication
  • User/pass for console
  • Up to 2 apikeys for CLI/SDK
  • MFA
• User Access Advisor

Auditing Users Using CLI

User Access Advisor

Groups in IAM

Useful for aggregating permissions for multiple users

Can have managed or inline policies

Group Access Advisor

Auditing Groups Using CLI

Group Access Advisor

Roles in IAM

• Cross-service access
• Cross account access
• Trust Relationship
• Role Access Advisor

Auditing Roles Using CLI

Role Access Advisor

Policies in IAM

• Managed policies
• Policy version
• Inline policies
• Policy Access Advisor

Auditing Policies Using CLI

Policy Access Advisor

Combined example

 for i in `aws iam list-users --query Users[*].UserName`
 do
   echo "username $i”
   aws iam list-user-policies --user-name $i
 done
					

• Loop through a dynamically generated list of usernames
• Print the username
• Print inline policies attached to each user (which is against best practices –use group policies instead)

Summary commands

 aws iam get-account-authorization-details
 aws iam get-account-summary
 aws iam get-account-password-policy
					

• List all IAM objects and relationships to each other
• Return numeric count of each object
• Describe the IAM password policy for console credentials

• Analysis of an IAM Credential Report
• What can we learn?

Summary

• IAM Objects
  • How they relate to each other
  • Audit individually and together
• Methods for Audit
  • CLI
  • AWS Console
  • Credential Report

Module Overview

• Auditing VPC
• Network configuration
• VPC Options
• Ingress/Egress Points
• Network security

What Is a VPC?

• Data center in the cloud
• Services placed inside
  • EC2
  • RDS
  • ELB
  • And more!

VPC Network Configuration

Question

• Where do I put my VPC?
• How big is my VPC network?
• How do I segment my network?
• How can I keep my resources private?
  

Choices

• Pick an AWS region
• CIDR network up to /16 (65536 addrs)
• Create subnets
• Route tables, NACLs, Security Groups

Create a VPC inventory

for i in `aws ec2 describe-regions --query
Regions[*].RegionName`
do
 echo “region $i”
 aws ec2 describe-vpcs --region $i
done
					

• Loop through a dynamically generated list of regions
• Print the region
• Print vpc id, cid rrange, and other details

Check VPCs for CIDR overlap

for i in `aws ec2 describe-regions --query
Regions[*].RegionName`
do
 echo “region $i”
 aws ec2 describe-vpcs--region $i--query Vpcs[*].CidrBlock
done
					

• Loop through a dynamically generated list of regions
• Print the region
• Print cidr range only, to visually check for overlap

VPC Options

• DNS Support
• DNS Hostnames
• DHCP Options
  • Domain
  • DNS servers
  • NTP servers
  • NetBIOS servers

Check DHCP option sets

for i in `aws ec2 describe-regions --query
Regions[*].RegionName`
do
 echo “region $i”
 aws ec2 describe-dhcp-options--region $i–output text
done
					

• Loop through a dynamically generated list of regions
• Print the region
• Print DHCP options. Look for custom domains or DNS servers when auditing.

VPC Ingress/Egress Points

• Internet Gateway (IGW)
• VPN Gateway (VGW)
• AWS Direct Connect
• VPC Peering Connections
• VPC Endpoints
• EC2 ClassicLink

Internet Gateway

• 2-way traffic
• Route table
• Public IP
• No firewall

aws ec2 describe-internet-gateways --query 
'InternetGateways[].{GwId:InternetGatewayId,
VpcId:Attachments[].VpcId}'--output text --region <region>
					

Create report of IGW and attached VPCs

Print the ID of each Internet Gateway in the region and which VPC it is attached to. Use JMES query format to customize output of the command.

VPN Gateway

• 2-way traffic
• Route table
• Private IP
• No firewall
• IPSEC Encryption

AWS Direct Connect

• 2-way traffic
• Route table
• Private IP
• No firewall
• 801.Q VLAN and BGP

VPC Peering

• 2-way traffic
• Route table
• Private IP
• No firewall
• Private, Point to Point

Describe VPC Peering Connections

for i in `aws ec2 describe-regions --query Regions[*].RegionName`
do
 echo “region $i”
 aws ec2 describe-vpc-peering-connections --region us-west-2 -
 -query 
'VpcPeeringConnections[].[AccepterVpcInfo.VpcId,RequesterVpcInfo.VpcId]’
done
					

• Loop through a dynamically generated list of regions
• Print the region
• Print the accepter and requester VPC Ids for each connection

VPC Endpoints

• S3 only
• 1-way traffic
• Route table
• Endpoint Policies

Describe VPC Endpoints

for i in `aws ec2 describe-regions --query Regions[*].RegionName`
do
 echo “region $i”
 aws ec2 describe-vpc-endpoints --query 'VpcEndpoints[].[VpcId,ServiceName]'
done
					

• Loop through a dynamically generated list of regions
• Print the region
• Print each endpoint with the VPC ID and service name.

EC2 ClassicLink

• 1-way traffic
• Route table
• Private IP
• No firewall

Discover VPCs with Classic Link Enabled

for i in `aws ec2 describe-regions --query Regions[*].RegionName`
do
 echo “region $i”
 awse c2 describe-vpc-classic-link --output json--query Vpcs[].[VpcId,ClassicLinkEnabled]
done
					

• Loop through a dynamically generated list of regions
• Print the region
• Print each VPC ID and whether ClassicLinkis enabled

Discovering Ingress/Egress points on a VPC

VPC Security

• Route Tables
• Network Access Control Lists (NACLs)
• VPC Flow Logs

VPC Route Tables

aws ec2 describe-route-tables --query
 'RouteTables[].[[VpcId,Routes[].GatewayId][]]’ --output text
					

List all route tables in the region with target ID

Print each VPC ID, and all targets used by route tables in the VPC.

Useful for discovering possible egress points from the VPC.

Network Access Control Lists

aws ec2 describe-network-acls--output table --region 
<region>
					

Print NACLs in human-readable form

Print easy-to-read text output of all NACLs in a region, including the associated VPC ID and Subnet ID

VPC Flow Logs

Explore VPC Flow Logs and Route Tables

Summary

• VPC can be customized in many different ways
• VPC environments have many options for ingress and egress
• Resources in the VPC can have security mechanisms separate from the VPC itself
• Next Up: Auditing EC2

Module Overview

Auditing EC2

• Security Features
• Inventory And Best Practices

EC2 Security

• Shared Responsibility Security Model
• Hypervisor
• EC2 keypairs
• Security groups
• Demo: Auditing Security Groups with AWS Config

Shared Responsibility Security Model

Security Model Applied to EC2

EC2 Hypervisor

• Xen Hypervisor (heavily modified)
• Virtualization types
  • HVM is full virtualization
  • PV is “lighter virtualization”
• VM Isolation on all resources
  • CPU
  • Memory
  • Disk
  • Network

EC2 Keypairs

Security Groups -Function

• Stateful firewall
• Applied to network interface
• Multiple allowed per interface
  • Rulesetsapplied as a whole
  • Least restrictive rule wins
• Inbound rules default deny
• Outbound rules default allow

Security Groups –Best Practices

Auditing Security Groups Script Part 1

#!/bin/bash
REGION=us-west-1
SGOUT="/tmp/sginfo”
aws ec2 describe-security-groups --region $REGION --output text > $SGOUT
IFS=$'\n'
cat $SGOUT | while read line
do
					

Auditing Security Groups Input File

SECURITYGROUPS registration frontend sg-c21df0a7 vpc1w2-reg2.0-prod-frontend 168369983848 vpc-0ef6e36c
IPPERMISSIONS    80  tcp  80
IPRANGES        0.0.0.0/0
IPPERMISSIONS     -1
IPRANGES       10.0.0.0/18
IPRANGES       10.0.128.0/18
IPRANGES       10.0.64.0/18
IPPERMISSIONS   22 tcp  22
IPRANGES       208.76.0.0/22
IPPERMISSIONS   443  tcp  443
IPRANGES       0.0.0.0/0
IPPERMISSIONSEGRESS   -1
IPRANGES       0.0.0.0/0
					

Auditing Security Groups Script Part 2

case $line in
SECURITYGROUPS*)
GID=(`echo $line | awk–F”\t” '{print $3}'`)
GNAME=(`echo $line | awk–F”\t” '{print $4}'`)
;;
IPPERMISSIONSEGRESS*)
PROTO=“EGRESS”
;;
					

Auditing Security Groups Script Part 3

IPPERMISSIONS*)
FROMPORT=(`echo $line | awk–F”\t” '{print $2}'`)
PROTO=(`echo $line | awk–F”\t” '{print $3}'`)
TOPORT=(`echo $line | awk–F”\t” '{print $4}'`)
;;
					

Auditing Security Groups Script Part 4

IPRANGES*)
CIDR=(`echo $line | awk–F”\t” '{print $2}'`)
if [[ "$CIDR" = "0.0.0.0/0" && "$PROTO" != ”EGRESS" ]]; then
  echo "$GNAME,$GID,$CIDR,$PROTO,$FROMPORT,$TOPORT"
fi
;;
esac
Done
rm $SGOUT
					

Auditing Security Groups Script Output

tu-prod-gateway,sg-3a05610a,0.0.0.0/0,tcp,22,22
vpc2w2-reg2.0-loadtest-frontend,sg-e318f586,0.0.0.0/0,tcp,80,80
vpc2w2-reg2.0-loadtest-frontend,sg-e318f586,0.0.0.0/0,tcp,443,443
vpc2w2-wp-stage-frontend,sg-c4abe6a3,0.0.0.0/0,tcp,80,80
vpc2w2-wp-stage-frontend,sg-c4abe6a3,0.0.0.0/0,tcp,443,443
					

Auditing Security Groups Script Summary

• CSV is easy to ingest into a database
• Python is useful for SDK work
• Task automation will make audits easier

EC2 Inventory and Tagging Best Practices

• Using CLI to inventory EC2
• Resource tagging
• Trusted advisor
• Demo: EC2 dashboard and filters

Why Use CLI to Audit EC2?

aws ec2 describe-instances --output text --region <region>
 --filter Name=instance-state-code,Values=16 --query 
 'Reservations[*].Instances[*].[InstanceId,PublicDnsName]'
					

Generate List of Instance ID and Public DNS

Print the Instance ID of each running instance (instance state code 16) and its associated Public DNS.

Great for inventory and loop-and-ssh functions.

aws ec2 describe-instances --output text --region <region> 
--query 
Reservations[].Instances[].[[InstanceId,SecurityGroups[].Gr
oupId,SecurityGroups[].GroupName][]]| tr'\t' ','
					

Generate CSV of Instance ID, SG ID, SG Name

Print each Instance ID in the region with its Security Group IDs and Security Group Names in CSV format.

The [] at the end of the query forces output to stay on the same line.

EC2 Tags

• 10 tags per instance
• Key/Value pairs, 128/255 character limit
• Integrated with CLI and billing

Tagging Strategies

Audit for Missing Tag Part 1

#!/bin/bash
REGION="us-west-2"
TAG="CostCenter"
TAGFILE="/tmp/ec2tags"
aws ec2 describe-tags --region $REGION --filters=Name=resource-
type,Values=instance--output text > $TAGFILE
ec2instances=`awsec2 describe-instances --region $REGION --query 
Reservations[].Instances[].InstanceId--output text |tr'\t' '\n'`
					

Audit for Missing Tag Part 2

for i in `echo $ec2instances`; do
if [[ `grep $i $TAGFILE |grep "$TAG"` = "" ]]; then
echo "$i is missing $TAG"
fi
done
rm $TAGFILE
					

AWS Trusted Advisor

• Free “Lite” version
• Types of predefined reports
  • Cost optimization
  • Performance
  • Security
  • Fault tolerance

EC2 dashboard and filters

Summary

• EC2 security involves AWS and customer
• Security groups can require extra effort to audit from the CLI
• Tags and filters are important
• Next Up: Auditing EBS

Module Overview

• EBS Basics
• EBS Security
• EBS Inventory
• EBS Best Practices

EBS Basics

• Network attached block storage
• Availability Zone specific
• Frequent Snapshots Decrease AFR

Volume Types

EBS Security

• NOT shared storage
• Hardware Decommissioning
• Volume encryption
• Demo: Add encryption to a volume

Storage Decommissioning

https://d0.awsstatic.com/whitepapers/aws-security-whitepaper.pdf

AWS uses the techniques detailed in DoD5220.22 -M (“National Industrial Security Program Operating Manual “) or NIST 800 -88 (“Guidelines for Media Sanitization”) to destroy data as part of the decommissioning process. All decommissioned magnetic storage devices are degaussed and physically destroyed in accordance with industry -standard practices.

EBS Volume Encryption

• Boot volumes
• Data volumes
• Encrypts traffic and data
• Not available for all instance types
  • Newer generations of EC2 only
  • Not for M1, C1, T1, etc
• Uses KMS (Key Management Service)

EBS Encryption Integration Points

Demo: Add Encryption To A Volume

EBS Inventory

Volume Inventory: Find Detached Volumes

	aws ec2 describe-volumes --region <region> \
	--query Volumes[].VolumeId\
	--filters Name=status,Values=available
					

Snapshot Inventory: Purge All But N Part 1

	#!/bin/bash
	VOLUMEID=$1
	REGION=$2
	NUMKEEP=$3
	TAIL=$(( $NUMKEEP + 1 ))
					

Snapshot Inventory: Purge All But N Part 2

SNAPIDS=`aws ec2 describe-snapshots \
--output text --region $REGION \
--query Snapshots[].[SnapshotId,StartTime] \
--filters Name=volume-id,Values=$VOLUMEID \
|sort -r -k2 |cut -f1 |tail -n +$TAIL`
					

Snapshot Inventory: Purge All But N Part 3

if [ "$SNAPIDS" != "" ]; then
  for SNAP in $SNAPIDS; do
   echo "Deleting snapshot $SNAP"
   aws ec2 delete-snapshot --region $REGION --snapshot-id $SNAP
  done
else
echo "Not enough snapshots, exiting"
fi
					

EBS Best Practices: Volume Sizing

Best Practices: Volume Performance

Demo Scenario: Compromised Instance

• How can we perform Root Cause Analysis safely?
• VPC, EC2, EBS

Summary

• EBS Volumes give many choices for performance
• Encryption is simple to deploy
• Snapshot management is critical
• Next Up: Auditing S3

Module Overview

• S3 Basics
• S3 Security
• S3 Inventory
• S3 Best Practices

S3 Basics

• Object Store
• Natively Available Online
• Fully Managed Service

S3 Hierarchy

S3 Storage Classes

S3 Security

• Access Control
  • IAM
  • ACLs
  • Bucket Policy
• Encryption
  • Server Side
  • Client Side

Access Control

S3 Object Encryption –Server Side with KMS

S3 Object Encryption –Client Side

Demo: Uploading into S3 Using SSE-KMS

S3 Inventory

S3 Inventory: Sort Buckets by Region Part 1

PRESORTFILE="/tmp/notsorted"
POSTSORTFILE="/tmp/sorted"
rm $PRESORTFILE $POSTSORTFILE
BUCKETLIST=`aws s3api list-buckets --query 'Buckets[].Name' |tr'\t' '\n'`
					

S3 Inventory: Sort Buckets by Region Part 2

for BUCKET in $BUCKETLIST; do
LOCATION=`aws s3api get-bucket-location --bucket $BUCKET`
echo "$LOCATION,$BUCKET" >> $PRESORTFILE
done
					

S3 Inventory: Sort Buckets by Region Part 3

sort -n $PRESORTFILE > $POSTSORTFILE
cat $POSTSORTFILE
					

Snapshot Inventory: List Bucket Sizes Part 1

#!/bin/bash
BUCKETLIST="/tmp/sorted"
for BUCKETINFO in `cat $BUCKETLIST`; do
REGION=`echo $BUCKETINFO |cut -f1 -d,`
if [[ $REGION == "None" ]]; then
REGION="us-east-1 --endpoint-urlhttps://s3.amazonaws.com"
fi
					

Snapshot Inventory: List Bucket Sizes Part 2

BUCKET=`echo $BUCKETINFO |cut -f2 -d,`
SIZE=`aws s3api list-objects --region $REGION --bucket $BUCKET --output json--query '[sum(Contents[].Size)]'`
echo "$BUCKET,$SIZE"
done
					

S3 Best Practices: Lifecycle Configuration

Best Practices: Naming Conventions

Summary

• S3 is a managed service. Customer only responsible for data protection and access.
• Several options for protecting objects
• Bucket audit actions can be complex

Audit Results

Questions?

Thanks for You!

---