If you are working with AWS heavily, you should look into a powerful tool called CloudFormation. Taking the time to learn CloudFormation is an investment that is easily returned to you in the form of powerful automation. In this guide, we’ll provide a gentle introduction to CloudFormation, and by the end, you’ll have the skills to start using it in your own AWS workflow.

The examples in this article are all available on Github.

What is AWS CloudFormation?

CloudFormation is a powerful tool from AWS that allows you to provision practically any resource you want. Here’s how I like to explain CloudFormation to people:

1. You tell CloudFormation what to provision.
2. You press a button.
3. Then CloudFormation magically creates the resources for you.

You tell CloudFormation what to provision in these blueprint documents called “templates.” These blueprint documents are written in either JSON or YAML form. Using these templates, you create “instances” of your blueprint called “stacks.” So if you are an engineer, you can think of templates as classes and stacks as instances of that class.

CloudFormation Terms and Template Anatomy

AWS CloudFormation docs cover the sections you define inside a template in the Template Anatomy of CloudFormation template section. Here’s the list condensed:

• AWSTemplateFormatVersion
• Description
• Metadata
• Parameters
• Mappings
• Conditions
• Transform
• Resources
• Outputs

Though these terms are provided in the proper structural order, explaining the terms in the same order is not best way to learn; especially if you are new to CloudFormation. It’s the same with your keyboard keys – having them in alphabetical order is not ideal for typing. Instead, the keys are laid out with the most common letters conveniently located to facilitate faster typing. Similarly, we will focus on and explain the most common CloudFormation sections first to promote learning.

Resources and Parameters Introduction

The Resources and Parameters sections are the most commonly used sections. Resources is the only required section in your template. After all, CloudFormation cannot spin up resources if you don’t tell it what to create. CloudFormation cannot magically read your mind (yet).

Here’s an example of a CloudFormation template that contains a single resource: an EC2 instance.

ec2.yml:

---
Resources:
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: t2.micro
      KeyName: 'default' # keypair must already exist
      ImageId: ami-6869aa05 # AMI us-east-1

Here’s how you use the template to create a stack using the AWS command line interface (CLI).

$ aws cloudformation create-stack --stack-name ec2 --template-body file://ec2.yml
{
    "StackId": "arn:aws:cloudformation:us-east-1:160619113767:stack/ec2/fbbfc9f0-c026-11e7-b32e-503aca2616d1"
}
$

To verify that the stack was created, you can use the CloudFormation Console or the CLI. Here’s the Console:

You can also use the describe-stacks command. Here’s confirmation with the CLI:

$ aws cloudformation describe-stacks --stack-name ec2
{
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:160619113767:stack/ec2/fbbfc9f0-c026-11e7-b32e-503aca2616d1",
            "Tags": [],
            "CreationTime": "2017-11-02T23:38:52.308Z",
            "StackName": "ec2",
            "NotificationARNs": [],
            "StackStatus": "CREATE_COMPLETE",
            "DisableRollback": false
        }
    ]
}
$

The next section is Parameters. Parameters are a way to provide inputs to a template and changes what the template does. Think about them as runtime arguments you can provide when running the create-stack command. Let’s add an option to change the instance type.

---
Parameters:
  InstanceType:
    Description: EC2 instance type
    Type: String
    Default: t2.micro
Resources:
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: !Ref InstanceType
      KeyName: 'default' # keypair must already exist
      ImageId: ami-6869aa05 # AMI us-east-1

You can now launch the stack and specify the InstanceType at runtime.

aws cloudformation create-stack --stack-name ec2-$(date +%s) --template-body file://ec2-param.yml --parameters "ParameterKey=InstanceType,ParameterValue=t2.small"

Note that I’m using the $(date +%s) command to add a random timestamp at the end of the stack name so I can quickly launch multiple stacks if needed. Specifying the parameters in CLI can get a little bit unwieldy, so you can use a JSON file instead. Here is the code from the JSON file and the command to launch it:

params.json:

[
  {
    "ParameterKey": "InstanceType",
    "ParameterValue": "t2.small"
  }
]

Command to launch the stack using parameters stored in a file:

aws cloudformation create-stack --stack-name ec2-$(date +%s) --template-body file://ec2-param.yml --parameters file://params.json

That covers the Resources and Parameters section of a CloudFormation template. CloudFormation can create a lot more resources than just an EC2 instance. A huge list of all the available resources is provided in the AWS Resource Type Reference docs.

Declarative vs Procedural

Before moving on to the next CloudFormation sections, it is useful to discuss the differences between the Declarative and Procedural programming paradigms.

Most developers are familiar with the procedural paradigm. In the procedural world, you write lines or code or statements, and the programming language evaluates each line of code one by one and does its bidding. The order of the statements matter. For example:

x = 1
y = x
print(y) # => prints 1

In contrast, in the declarative world, the order of the lines do not matter. This might look a little weird for folks who live more often in the procedural world. Here’s an example:

y = x
x = 1
print(y) # => prints 1

Notice that y = x comes before x = 1. Declarative programming language know how to interpret what you are trying to do just fine. Ops engineers are sometimes more familiar with the declarative model because configuration files are usually declarative. When you tweak apache.conf or nginx.conf configurations you don’t care about the order of the lines – you just care about the desired end behavior. This is the declarative frame of mind.

I’ve covered all of that so that I can now simply say that CloudFormation is declarative. It is key to understand that CloudFormation is declarative for the next section.

Conditions, Intrinsic Functions, and Mappings

With every programming language, there are constructs used to tell the software what you want it to do. Control structures like if statements, case statements, and functions are helpful. CloudFormation also has control structures, but they come in declarative form.

Conditions

A normal procedural condition looks something like this:

If env == "prod"
  # create a large m4.large instance for production environment
else
  # create a small t2.micro instance for development environment
end

With CloudFormation, the if condition looks like this:

---
Parameters:
  LargerInstance:
    Description: Whether a larger instance should be used
    Default: false
    Type: String
    AllowedValues: [true, false]
Resources:
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType:
        !If [UseLargerInstance, m4.large, t2.micro]
      KeyName: 'default' # keypair must already exist
      ImageId: ami-8c1be5f6 # AMI us-east-1
Conditions:
  UseLargerInstance: !Equals [true, !Ref LargerInstance]

The AWS If Condition documentation provides more examples with it. To launch the instance with the larger instance:

aws cloudformation create-stack --stack-name ec2-$(date +%s) --template-body file://ec2-if-env-is-prod.yml --parameters "ParameterKey=Env,ParameterValue=prod"

Intrinsic Functions

If you need more logical control, CloudFormation provides Intrinsic functions. All sort of functions are supported: Fn::Base64, Fn::FindInMap, Fn::GetAtt, Fn::GetAZs, Fn::If, Fn::Join, Fn::Select, Ref The full list is available on the official Intrinsic Functions Reference docs. Let’s go through one example. Say you wanted to join 3 variables called App, Env, and Role together. In Ruby that would look like:

[app, env, role].join('-')

In a CloudFormation template, we can do this:

---
Parameters:
  App:
    Description: "Examples: blog, api"
    Type: String
  Env:
    Description: "Examples: prod, stag, dev"
    Type: String
  Role:
    Description: "Examples: web, worker, scheduler"
    Type: String
Resources:
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: t2.micro
      KeyName: 'default' # keypair must already exist
      ImageId: ami-6869aa05 # AMI us-east-1
      Tags:
        -
          Key: Name
          Value: !Join [ '-', [!Ref App, !Ref Role, !Ref Env] ]

Here’s the command to launch the stack:

aws cloudformation create-stack --stack-name ec2-$(date +%s) --template-body file://ec2-join-tag-example.yml --parameters file://params-app-role-env.json

Mappings

Mappings are another logical construct in CloudFormation. Mappings can be thought of as a dictionary map. It’s sort of like a 2-level case statement. The mapping must be exactly 2 levels. Here’s an example:

Mappings:
  RegionMap:
    us-east-1:
      32: "ami-6411e20d"
      64: "ami-7a11e213"
    us-west-1:
      32: "ami-c9c7978c"
      64: "ami-cfc7978a"
Resources:
  myEC2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      ImageId: !FindInMap [ RegionMap, !Ref "AWS::Region", 32 ]

In Ruby, this would be:

region_map = {
  "us-east-1": {"32": "ami-6411e20d", "64": "ami-7a11e213"},
  "us-west-2": {"32": "ami-c9c7978c", "64": "ami-cfc7978a"}
}
puts region_map["us-east-1"]["32"]

If you have been thinking mostly in procedural programming land, then you might be thinking, at this point, that the CloudFormation logical constructs look weird and confusing. I would agree. Most of the time, it is easier for my brain to reason with procedural code. However, CloudFormation is implemented in a declarative manner because of the magic that I’ve been talking about.

Notice that you do not tell CloudFormation the order in which to spin up the resources. You can even move the sections around and put the in a different order and it still produces the same results. It is declarative by its very nature. Writing more advanced CloudFormation templates is about translating procedural constructs into declarative ones. You define the template, hand it over, and CloudFormation figures it all out.

Outputs, Description, Version, Metadata, Transforms

Here’s a brief description of the rest of the sections of a CloudFormation template. I’m only covering them briefly for completeness’ sake. They are not core to understanding CloudFormation on an introductory level as the previously covered sections.

• Outputs – Where you output values to make them easier to find. Some examples are instance_id or RDS DNS endpoints. Outputs are useful for nested stacks
• Description – A comments to describe your template to the world
• AWSTemplateFormatVersion – The only current and only valid value is “2010-09-09”
• Metadata – Data about your template. It is written in JSON or YAML form
• Transform – The transform syntax to use. This specifies the macro language for some resources that support it. An example is Transform: AWS::Serverless-2016-10-31

Final Thoughts

That’s it! We’ve covered all the sections of a CloudFormation template and went through a basic CloudFormation introduction.

I used to think that it is a big trade-off to have to write code declaratively. However, I quickly realized that the trade-off is fairly small, given what CloudFormation can do for you. The return you get from using CloudFormation gets larger exponentially as what you do gets more sophisticated.

Take a simple example of spinning up 10 EC2 instances and associating 10 route53 DNS record with each of them. To do this manually, you would spin up the instances, wait for each instance’s public IP address, create the route53 DNS record, and then associate the DNS record with the instance’s IP address. With CloudFormation, you simply say, “Give me 10 instances and 10 route53 DNS records.” That’s it. CloudFormation orchestrates it all. It even parallelizes it. You are offloading the hard work of coordination and optimization over to CloudFormation.

When used effectively, CloudFormation can be used to provision up entire infrastructure with VPCs, AutoScaling clusters, IAM policies, Security Groups and more with a click of a button. CloudFormation is an extremely powerful tool. For anyone using AWS often, it should be an essential part of their toolkit. CloudFormation is sometimes known for a steep learning curve, so I hope this article lowers the learning curve and has helped you. AWS Solutions Architects successfully work with CloudFormation day in and day out. It is a tool that I used often enough that the automation scripts for it eventually became a tool called lono.

A version article was originally published on Cloud Assessments.