How to provision an EC2 instance using Ansible with AWS Cloudformation template

Table of Contents

Test Environment

Fedora 32
Python 3.8.10

What is AWS Cloudformation

AWS Cloudformation is a service provided by AWS which helps to model and setup AWS resources. By using Cloudformation, we easily manage a collection of resources as a single unit. We can reuse the Cloudformation template to create the resources in a consistent and repeatable manner across multiple regions for HA. In AWS Cloudformation we work with templates and stacks. Templates describe the AWS resources and their properties. Stack provisions the AWS resources described in templates. We can work with stacks by using the Cloudformation console, API, or AWS CLI. When creating a stack, AWS Cloudformation makes underlying service calls to AWS to provision and configure your resources.

What is Ansible

Its a Configuration Management tool used to configure and manage the Linux system for all the types of operations like Installating, Patching, Deployment of Applications, File Management, Service Management, Package Management. Anisble is purely based on Python. Ansible using the SSH protocol based on the native OpenSSH package for communication with the Linux nodes which it manages.

In this article we will see how we can use Ansible AWS Cloudformation module to create and manage AWS stack using the Cloudformation templates. For this article we are going to setup our machine with Ansible and AWS CLI tools and then use the Cloudformation template from Ansible playbook to create a AWS Stack. This stack is going to create a EC2 instance in AWS cloud.

If you are interested in watch the video. Here is the youtube video on the same step by step procedure shown below.

Lets get started.

Procedure

Step1: Install and Configure Ansible

Ansible can be installed using different methods but here in this article we will install ansible from the RPM package provided by the Fedora OS.

[admin@fedser32 ec2_template]$ sudo dnf install ansible-2.9.21-1.fc32.noarch

Once ansible is installed you can updated the inventory file located at /etc/ansible/hosts with the information (ie. IP address or FQDN ) about the node that we want to manage. In our case we will using the machine on which ansible is installed to provision the AWS EC2 instance. We are not actually mananging any specfic node here but we are using the controller node on which our ansible package is installed to provision the ec2 install on AWS cloud using the Cloudformation template.

Here is my ‘hosts’ file with the loopback IP address to which i want to connect using the ansible playbook.

[admin@fedser32 ec2_template]$ cat /etc/ansible/hosts | grep 127
127.0.0.1

Now that you have the ansible installed and the inventory ready. You can quickly validate your ansible installation by executing the below ad-hoc command to make sure its working as expected.

[admin@fedser32 ec2_template]$ ansible all -m ping
[DEPRECATION WARNING]: Distribution fedora 32 on host 127.0.0.1 should use /usr/bin/python3, but is using /usr/bin/python for backward 
compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See 
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information. This feature will be removed in 
version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
127.0.0.1 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"

For more detailed information on setting up ansible, please go through the following installation guide.

Step2: Install AWS CLI tool for resource management

For managing the AWS resources using the command line, we need to get the AWS CLI tool installed on the controller machine as shown below.

[admin@fedser32 stack]$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
[admin@fedser32 stack]$ unzip awscliv2.zip
[admin@fedser32 stack]$ sudo ./aws/install
[sudo] password for admin: 
You can now run: /usr/local/bin/aws --version
[admin@fedser32 stack]$ aws --version
aws-cli/2.3.6 Python/3.8.8 Linux/5.11.22-100.fc32.x86_64 exe/x86_64.fedora.32 prompt/off

Step3: Configure the AWS CLI with credentials

Lets now configure our AWS CLI with the AWS IAM user access key id and secret key. Please make sure the IAM user has the required privileges to create and manage the AWS resources. In our case for the AWS EC2 instance.

[admin@fedser32 stack]$ aws configure
AWS Access Key ID [None]: xxxxxxxxxxxxxxxxxxxx
AWS Secret Access Key [None]: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Default region name [None]: ap-south-1
Default output format [None]: json

For creating AWS account and setting up the IAM user, please go through the following setup guide.

Step4: Install the AWS Cloudformation module

Now that we have our Ansible installed and AWS CLI configured we are ready to proceed with setting up Ansible to Automate and Manage the AWS resources using the Cloudformation templates. For this we first need to install the Ansible module named ‘amazon-aws’ which will be used to manage the AWS resources.

[admin@fedser32 awstemplates]$ ansible-galaxy collection install amazon.aws
Process install dependency map
Starting collection install process
Installing 'amazon.aws:2.1.0' to '/home/admin/.ansible/collections/ansible_collections/amazon/aws'

Also, need to install the following pip packages on the controller node that executes this module. Make sure you have python >= 2.6 installed on your machine.

[admin@fedser32 awstemplates]$ pip3 install boto boto3 botocore>=1.5.45 --user
[admin@fedser32 awstemplates]$ pip3 list | grep boto
boto               2.49.0    
boto3              1.20.6    
botocore           1.23.6

Step5: Prepare the ansible playbook to trigger the CF template

Let’s now prepare out Ansible playbook using the amazon aws Cloudformation module by passing the below parameters. Here i using a very basic Cloudformation template named ‘ec2.yml’ which creates an AWS EC2 instance in the specified region. Here are the details about the template file and ansible playbook.

Cloudformation template

[admin@fedser32 ec2_template]$ cat ec2.yml 
AWSTemplateFormatVersion: 2010-09-09
Resources:
  EC2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      ImageId: "ami-0f1fb91a596abf28d"
      InstanceType: "t2.micro"

Ansible Playbook with Cloudformation module

[admin@fedser32 ec2_template]$ cat ec2-playbook.yml 
---
- name: EC2 Cloudformation template example
  hosts: localhost
  tasks:
  - name: create a stack, pass in the template
    amazon.aws.Cloudformation:
      stack_name: "ansible-Cloudformation-stack"
      state: present
      region: ap-south-1
      disable_rollback: true
      template: "/home/admin/middleware/stack/awstemplates/ec2_template/ec2.yml"
      tags:
        Name: ansible-Cloudformation-stack

Step6: Execute the playbook

We are now ready with the Cloudformation template and the playbook to trigger it. Let go ahead and execute the playbook as shown below.

[admin@fedser32 ec2_template]$ ansible-playbook ec2-playbook.yml 

PLAY [EC2 Cloudformation template example] *******************************************************************************************************

TASK [Gathering Facts] ***************************************************************************************************************************
[DEPRECATION WARNING]: Distribution fedora 32 on host 127.0.0.1 should use /usr/bin/python3, but is using /usr/bin/python for backward 
compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See 
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information. This feature will be removed in 
version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ok: [127.0.0.1]

TASK [create a stack, pass in the template] ******************************************************************************************************
changed: [127.0.0.1]

PLAY RECAP ***************************************************************************************************************************************
127.0.0.1                  : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Step7: Validate the Stack

The playbook has been successfully executed. You can validate whether the EC2 instance has been created as shown below and also describe the details about your stack which you just created.

Describe EC2 instances

[admin@fedser32 ec2_template]$ aws ec2 describe-instances
{
    "Reservations": [
        {
            "Groups": [],
            "Instances": [
                {
                    "AmiLaunchIndex": 0,
                    "ImageId": "ami-0f1fb91a596abf28d",
                    "InstanceId": "i-01cb879e543b9bed9",
                    "InstanceType": "t2.micro",
                    "LaunchTime": "2021-11-16T08:24:55+00:00",
                    "Monitoring": {
                        "State": "disabled"
                    },
                    "Placement": {
                        "AvailabilityZone": "ap-south-1a",
                        "GroupName": "",
                        "Tenancy": "default"
...

List Cloudformation Stacks

[admin@fedser32 ec2_template]$ aws Cloudformation list-stacks
{
    "StackSummaries": [
        {
            "StackId": "arn:aws:Cloudformation:ap-south-1:xxxxxxxxx:stack/ansible-Cloudformation-stack/ab52df50-46b6-11ec-93ed-0ad2d6a913ae",
            "StackName": "ansible-Cloudformation-stack",
            "CreationTime": "2021-11-16T08:24:51.449000+00:00",
            "StackStatus": "CREATE_COMPLETE",
            "DriftInformation": {
                "StackDriftStatus": "NOT_CHECKED"
            }
        }

Hope you enjoyed reading this article. Thank you..

How to provision an EC2 instance using Ansible with AWS Cloudformation template