Books / AWS in Bullet Points - Exam Prep Study Notes / Chapter 4

Compute - EC2 (Elastic Compute Cloud)

In this chapter

Basic tasks
Security Groups
EC2 User Data
EC2 Instance Launch Types
- Tenancy
EC2 Instance Types
EC2 AMIs
- Common tasks
EC2 Placement Groups

Some capabilities/integrations:
- EC2: Renting virtual machines
- EBS: Storing data on virtual drives
- ELB: Distributing load across machines
- ASG Scaling the services using an auto-scaling group
Pricing
- Billed only when in running state.
  - And stopping state only if it’s hibernating.
  - Also stopped state for reserved instance.
- Billed by the hour or the second, depending on which AMI you’re running
- t2.micro is free tier.
- Data transfer costs
  - Free between EC2 and other services.
  - Free between EC2 instances in same availability zone using private IP addresses only.
  - $0.01/GB for transfer between instances in different availability zones
  - See also data transfer costs
Elastic Network Interface (ENI)
- Each EC2 has one default ENI.
- Each ENI have
  - one primary private IPv4, can have additional private IPv4 (optional)
  - one Elastic IP (static IP) per private IPv4 (optional)
  - one public IPv4
  - one or more IPv6
  - one or more security group
  - a mac address
  - source destination check flag
  - description of ENI
- Can attach a secondary network interface
- You can attach a network interface to an EC2 instance in the following ways:
  - Hot attach: When it’s running
  - Warm attach: When it’s stopped
  - Cold attach: When the instance is being launched
- Termination with instance termination
  - Default interfaces are terminated with instance termination.
  - Manually added interfaces are not terminated by default
Type describes how powerful the machine is (vCPUs, Memory, Network performance etc.)
EC2 Tenancy allows you to choose a tenant for the instance
- Run a shared hardware - Default
- Dedicated - Run a Dedicated instance, shared within different instance (non-dedicated) from same AWS account.
- Dedicated host - You have visibility and control over how instances are placed on the server
Status Checks checks physical machine (underlying hyper-visor) and returns pass & fail.
- You can set-up Status Checks alarms.
- Auto Scaling Group uses this check by default.
  - Can set to use Elastic Load Balancer’s health check.
You can enable termination protection (termination = deletion)
- Prevents an instance from being accidentally terminated by requiring that you disable the protection before terminating the instance
EC2 Instance Metadata
- Allows AWS EC2 instances to learn about themselves without using an IAM Role / permission.
- The internal URLs are accessible only by EC2
- You can retrieve the IAM Role name from the metadata
  - ❗ You can not retrieve the IAM policy
- Meta data = Info about the EC2 instance at http://169.254.169.254/latest/meta-data
  - Your EC2 instance has always credentials to the role that usually expires in 1 hour
    - Get IAM credentials: http://169.254.169.254/latest/meta-data/iam/security-credentials/<iam-role-name>
- User data = launch script of the EC2 instance http://169.254.169.254/latest/user-data
Enhanced Networking can be enabled on specific instances with a driver to reduce networking overhead + faster networking.
- 📝Single root I/O virtualization (SR-IOV)
- Utilizes Elastic Network Adapter (high Performance Network Interface for Amazon EC2)
- Free
- ❗ Limitations
  - Instances must be launched in a VPC
  - Instances must be launched from a HVM AMI
- Elastic Network Adapter: High performance
- Elastic Fabric Adapter: High Performance Computing
  - ❗ Linux machines only
Monitoring
- ❗📝 CloudWatch does not monitor memory usage as well as disk space utilization
  - 💡 Requires installation of CloudWatch agent or use custom scripts to send custom metrics.
Launch templates
- Store launch parameters so that you don’t specify them every time you launch an instance.
- E.g. AMI ID, instance type, key pairs and Security Groups.t
💡 An instance may immediately terminate if:
- You’ve reached your EBS volume limit
- An EBS snapshot is corrupt
- The root EBS volume is encrypted and you do not have permissions to access the KMS key for decryption
- The instance store-backed AMI that is attached is missing a required part (an image.part.xx file)

Basic tasks

Hibernating an instance means persisting all of its data and RAM so you can pre-warm an instance before resuming.
Stopping and starting an instance migrate EC2 to a most likely new underlying host.
Retiring: An instance is scheduled to be retired when AWS detects irreparable failure of the underlying hardware hosting the instance.
- When an instance reaches its scheduled retirement date
  - It is stopped (if it has EBS root volume) or
  - Terminated (if it has instance instance store root volume) by AWS
📝Get meta data
- Request to EC2 metadata service from your EC2 instance
  - http://169.254.169.254/latest/meta-data
  - It returns the properties (recursively)
  - Get data further e.g. http://169.254.169.254/latest/meta-data/placement/availability-zone
- You can also use Instance Metadata Query Tool which is a simple bash script that does the curl.
Launch virtual server using AWS Console
1. Go to EC2 -> Launch an instance -> Choose Amazon Machine Image (AMI)
2. Choose type for machine (e.g. t2.nano, t2.micro)
3. Configurations
  - Storage: EBS is where the data will be stored (SSD, HDD etc.)
  - You can have different tags
    - Name is important as it appears in UI.
4. Launch
  - For SSH you need to have key pair, you can generate new or use existing.
Connect to EC2 through SSH
- SSH with e.g. ssh -i privatekey.pem <user-name>@<ip-address>
  - Username can be ec2-user (Amazon Linux, RHEL, SUSE, FreeBSD), ubuntu (Ubuntu, NanoStack), admin (Debian), root (RHEL, TurnKey, OmniOS), fedora (Fedora), centos (Centos), ``bitnami` (BitNami)
  - IP address is the public IP address of the machine, you cannot use private IP to connect.
  - Private key privatekey.pem is enough as AWS puts public key automatically to ~/.ssh/authorized_keys
- 💡 When you first download key-pair file permission is 0644 which is to open -> private key can leak as it’s accessible by others.
  - 📝Fix with chmod 0400 awsEc2.pem -> only root user can see it.
  - 📝chmod 400 is for Linux only
- On Linux / Mac we use SSH, on Windows we use PuTTy (on Windows 10 SSH can be used as well)
Configuring Windows machines
- EC2 Launch is PowerShell scripts that makes the computer AWS-friendly
  - It sends instance information to EC2 console, sets computer name & wallpaper, adds DNS suffixes, executes user data, persistence static routes to reach the metadata & KMS services, dynamically extends OS partition to include any unpartitioned space and more…

Security Groups

Firewall on EC2 instances.
Associated to ENI and not with instance
Controls authorized IP ranges (IPv4 and IPv6) and ports with:
- Inbound rules
  - From other to the instance
  - All inbound rules are denied by default
  - You define protocol (e.g. TCP) & port range & source
    - Source
      - 📝Refer another security group (type in Security Group ID)
      - An IPv4 or IPv6 CIDR block
      - A single IPv4 or IPv6 address (e.g. anywhere: 0.0.0.0/0 for IPv4 and ::/0 for IPv6)
- Outbound rules
  - From the instance to other
  - Open to everywhere by default
  - You define protocol (e.g. TCP) & port range & destination.
    - Destination
      - 📝Another security group
      - An IPv4 or IPv6 CIDR block
      - A single IPv4 or IPv6 address
      - An AWS service (i.e. prefix list ID)
Can be attached to multiple instances
📝Security groups are sateful(once allowed inbound, connection passes through outbound and vice versa)
An instance can have multiple security groups
Locked down to a region / VPC combination
- ❗ You need to recreate security groups if you change region / VPC.
- You can create a snapshot => Create AMI and use AMI to launch in other AZ or copy AMI to new region and launch there.
💡 Good practice: maintain one separate security group for SSH access.
📝Any timeout errors means a misconfiguration of your security groups
If your application gives “connection refused” error, then it’s an application error or it’s not launched
Being able to reference security groups allows:
- Easier maintenance as you don’t need to work with IP’s.
- E.g. allow EC2 instances to connect with each other by authorizing their attached security groups.
❗ You can only delete security group after you delete all EC2 instances associated with the security group.
❗📝 You cannot block single IP/IP-range with Security Groups, you’ll need NACL (Network Access Control Lists) for it.
- Security groups can only “allow” services through a default deny

EC2 User Data

Enables to bootstrap instances using EC2 data script.
- bootstrapping means launching commands when a machine starts
Script is only run once at the instance first start
Enables automating boot tasks such as installing updates, installing software, downloading common files from internet ….
Runs with the root user (sudo rights)
📝Get user data -> Request to http://169.254.169.254/latest/user-data from your EC2 instance
Flow
- Create EC2 -> Configure Instance -> Advanced Details -> User data -> Paste commands
- 💡 In first line add #!/bin/bash -> tells Unix what program to use to run it
For faster deployment without bootstrapping use Golden AMIs:
1. Create a golden AMI with the static installation components already setup
2. Use EC2 user data to customize the dynamic installation parts at boot time.
  - 🤗 Elastic Beanstalk is a good example for mixing golden AMIs and EC2 user data.

EC2 Instance Launch Types

On demand instances
- Start whenever you want & stop without any long time commitment.
- Pricing
  - Pay for what you use (billing per second, after the first minute)
  - Has highest cost but no upfront payment
- 💡 Recommended for short-term and uninterrupted workload, where you can’t predict how the application will behave.
  - E.g. when doing auto-scaling
Reserved instances
- Standard Reserved instances
  - Reserve an instance for 1 to 3 years
  - Enables you to modify Availability Zone, scope, network platform, and instance size (within the same instance type)
    - Up to 75% discount compared to On-demand
    - Payment options are “no upfront” (pay monthly), “partial upfront”, “all upfront” (cheaper).
  - You can sell sell your unused instances on the AWS Reserved Instance Marketplace.
  - 💡 Recommended when e.g. when running a database over a year
  - 💡 Reserve minimum capacity you’ll use in year e.g. 1 EC2 instance in each AZ in multi-AZ architecture.
  - 💡 Can be sold in Reserved Instance Marketplace
- Convertible Reserved Instances
  - Convertible means you have option to change:
    - Instance family, instance type, platform, scope, and tenancy
    - Only if the creation of new instance is equal or greater value
  - Pricing: Up to 54% discount
  - 💡 Workloads are likely to change
- Scheduled Reserved Instances
  - Launch within time window you reserve
  - 💡 You know something will happen e.g. every week but only during one hour or one day.
Spot instances
- Cheapest (up to 90% discount compared to on demand) but risk of losing instance
  - Spot instance is charged per second.
  - If AWS shuts down your instance in first hour you don’t pay, but you have to pay if you terminate.
- You bid a maximum price and get the instance as long as it’s under the price
  - You lose the instance if you get outbid
    - Instances are reclaimed with a 2 minute notification warning when the spot price goes above your bid.
    - Persistence request allows instances to come back when the price drops to the right pricing.
  - Price varies based on offer and demand at times
- 📝Spot instance shut down priority: The one who had longer time (e.g. 50 min) will be shut down faster than the shorter time (e.g. 5 min)
- 💡 For short & fault tolerant workloads e.g. batch jobs, Big Data analysis.
- In the event of an interruption, you can set instances to be terminated or hibernated.
- You may get insufficient capacity error
  - Stop all instances & start again to end up in a new hardware.
❗ 20 On-Demand & 20 Reserved & 20 Spot instances per region (soft limit)
Spot Block
- Allows you to request Spot instances for 1 to 6 hours at a time to avoid being interrupted while your job completes

Tenancy

Each instance that you launch into a VPC has a tenancy attribute
- Default: Your instance runs on shared hardware.
- Dedicated: Your instance runs on single-tenant hardware.
- Host: Your instance runs on a Dedicated Host, which is an isolated server with configurations that you can control.
❗ Cannot change tenancy after launching instances when
- From default to dedicated or host.
- From dedicated or host to default.
- 💡 You only switch between dedicated instances i.e. from host to dedicated and from host to dedicated.
- A VPC tenancy can be changed to any but it only affects new instances.
Dedicated instances
- Instances running on hardware that’s dedicated to you
  - You don’t control your hardware.
  - May share hardware with other instances in same account
- No control over instance placement (can move hardware after Stop / Start)

Dedicated Hosts

Physical dedicated EC2 server for your use
Full control of EC2 instance placement
Visibility into the underlying sockets / physical cores of the hardware
- You have direct access to the CPU
Allocated for your account for a 3 year period reservation
More expensive
You can allow instance auto-placement where untargeted EC2s will be launched in dedicated host(s).
Useful for
- Complicated licenses such as BYOL - Bring Your Own License, or you pay based on CPU cores etc.
  - ❗ A Dedicated Host is required if you’d like to use your existing Windows Server licenses.
- Regulatory & compliance needs.

Dedicated Instances vs Dedicated hosts

Characteristics	Dedicated Instances	Dedicated Hosts
Enables the use of dedicated physical servers	X	X
Per instance billing (subject to a $2 per region fee)	X
Per host billing		X
Visibility of sockets, cores, host ID		X
Affinity between a host and instance		X
Targeted & automatic instance placement		X
Add capacity using an allocation request		X

EC2 Instance Types

📝Main types

Type	Mnemonic	Category	Description	Use-cases
M	(M)ain / (M)edium	General purpose	Balance of compute, memory, network resources	General, mid-size databases, web apps
C	(C)ompute	Compute optimized	Advanced CPUs	Modeling, analytics, databases
H	(H)DD	Storage optimized	Local HDD storage	Map reduce
R	(R)am	Memory Optimized	More ram per $	In-memory caching
X	(X)treme	Memory Optimized	Terabytes of RAM and SSD	In-memory databases
I	(I)ops	IO optimized	Local SSD storage, High IOPS	NoSQL databases
G	(G)PU	GPU Graphics	GPUs with video encoders	3D rendering
P	(P)ictures	GPU Compute	GPUs with tensor cores	Machine learning
F	(F)ield	Accelerated Computing	Field Programmable Gate Array, custom hardware accelerations	Genomics, financial analytics
T	(T)iny / (T)urbo	Burstable, Shared CPUs, lowest cost	Web servers

Remember all: PHD McGift Rx

Numbers change over time and indicate generation e.g. t2., t3..
Burstable instances
- Good performance for a burst / short while e.g. spike and your CPU goes to 100%
- Burst credits
  - Bursting consumes burst credits and low CPU gets you credits.
  - You get more credits when your CPU is low.
T2 / T3 unlimited provides unlimited burst but you pay extra if you go over your credit balance.
You can also have Bare Metal instances to eliminate virtualization overhead.
X1e is part of Memory Optimized instance family and recommended for databases.

EC2 AMIs

AMI = Amazon Machine Images
AWS has base images for Ubuntu, Fedora, Red Hat, Windows, Amazon Linux etc.
AMI can be found and published on the Amazon Marketplace
Two virtualization types
- HVM VM - Hardware virtual machine
  - Utilizes special hardware extensions as it’s on bare metal hardware.
  - Enhanced Networking
  - Only option for Windows.
- PV VM - Paravirtual virtual machines, only available for Linux
- 💡 HVM is recommended because same or better performance than paravirtual guests.
  - Before: Better storage & network in PV VM because of special I/O drivers.
  - Now: PV drivers are available for HVM VM (instead of emulating hardware)
Custom AMIs
- AMIs can be built for Linux/Windows machines.
- Removes need for EC2 user data with faster boot time.
- Can be created from volumes or snapshots.
- Can use pre-configured & maintained public AMIs from other people by paying by hour.
- ❗ AMI are built for a specific AWS region
  - An AMI created for a region can only be seen in that region.
  - 📝 Copying makes it available in different regions for e.g. disaster recovery.
- ❗ Do not use AMI you don’t trust, they may be unsecure or contain malware.
- VM Import/Export allows you to import OVA file (or VMDK, VHD, or RAW) as AMI then create EC2 based on it.
  - AWS SMS - Server Migration Service
    - Enhances VM import & export
    - Migrate thousands (concurrently 50) of Hyper-V and VMware workloads to AWS.
AMI Storage
- S3 but you won’t see them in the S3 console
- By default, your AMIs are private and locked for your account / region.
  - You can make AMIs public in order to share with other accounts or sell in AMI Marketplace.
AMI Pricing: Cheap as you only pay for space it takes in S3.

Common tasks

Copying an AMI
AWS does not batch copy following, need manual work:
- User-defined tags
- Launch permissions
- S3 bucket permissions
Create AMI from EC2
- Right click on EC2 -> Image -> Create Image
- When it’s done it appears on EC2 dashboard -> Images -> AMIs
- You can right click and
  - Copy it to another region
  - Deregister it in order to remove it.
  - In image permissions:
    - Make it publicly available, or share it with other AWS account numbers.
    - Create volume permissions option allows others accessing AMI to make a copy of the AMI.
  - Launch a new EC2 based on it.
📝Cross account / cross region AMI Copy
- Share AMI with another AWS account
  - Sharing an AMI does not affect the ownership of the AMI.
    - ❗ However if you copy an AMI that has been shared with your account, you are the owner of the target AMI in your account.
  - Flow:
    1. Right click on EC2 -> Image Permissions
    2. Keep it private & add account number.
    3. Select Add “create volume permissions to snapshot to allow other accounts to make a copy.
- Ensure you have read permissions for AMI storage:
  - Associated EBS snapshot (for an Amazon EBS-backed AMI)
  - Associated S3 bucket (for an instance store-backed AMI)
- ❗ You can’t copy:
  - Encrypted AMI shared from another account
    - 💡 Instead you can copy the snapshot (if it’s shared with encryption key) and re-encrypt it & register as a new AMI you own.
  - AMI with an associated billingProduct shared from another account
    - They’re Windows AMIs and AMIs from the AWS marketplace
    - 💡 You can launch an EC2 instance in your account using the shared AMI -> Create an AMI from the instance

EC2 Placement Groups

Use to control how EC2 instances will be placed within AWS infrastructure.
❗ Works only for certain types of instances
❗ You can’t merge placement group
❗ Placement group must have unique name within AWS account.
❗ You can’t move existing EC2 to placement group -> you can create AMI and launch new.
💡 AWS recommends homogenous instances within placement groups e.g. using same instance types.
Creation
- Console -> EC2 -> Placement Groups -> Create Placement Group -> Select a strategy
- You can now launch new EC2 instances in the placement group.
Placement Group Strategies
- Cluster
  - Clusters instances into a low-latency group in a single Availability Zone
    - ❗ Cannot span multiple AZ
  - High performance, high risk
    - High network throughput, low-latency (10 Gbps bandwidth between instances)
    - Same rack & same AZ -> If the rack fails, all instances fails at the same time.
  - 💡 Use cases
    - Big Data job that needs to complete fast
    - Application that needs extremely low latency and high network throughput.
- Spread
  - Spreads instances across underlying hardware (❗ max 7 instances per group per AZ)
    - Ensures all instances are located in different hardware.
  - Can span across multi AZ.
  - Minimizes failure risks.
- Partition
  - Spreads instances across many different partitions (which rely on different sets of racks) within an AZ.
  - Partitions are isolated from each other from hardware failures in different racks but not VMs.
  - Can span across multi AZ.
  - Scales to 100s of EC2 instances per group.
  - ❗ Up to 7 partitions per AZ
  - EC2 instances get access to the partition information as metadata
  - 💡 Use cases: HDFS, HBase, Cassandra, Kafka

Licenses and Attributions

Speak Your Mind Cancel reply

-->

Related Books

Books / AWS in Bullet Points - Exam Prep Study Notes / Chapter 4

Compute - EC2 (Elastic Compute Cloud)

In this chapter

Basic tasks

Security Groups

EC2 User Data

EC2 Instance Launch Types

Tenancy

EC2 Instance Types

EC2 AMIs

Common tasks

EC2 Placement Groups

Licenses and Attributions

Original Content License

Speak Your Mind Cancel reply