How to bootstrap an AWS EC2 node and manage it using Chef Server

Purpose – In this article, we will setup up an On Premise Chef server for managing our Cookbooks in a central location, setup Chef Workstation where in you author your cookbooks and upload them to Chef server and launch an ec2 instance which will be bootstrapped and configured using cookbook uploaded into Chef server.

Pre-requisites –
Chef Server node
Workstation node
Managed node

Procedure –

Step1: Install and Configure Chef Server with Management Console

In this step we are going to install Chef server and Chef Management Console and startup the components using reconfigure. Also we are creating a user and an Organization which can be managed using the user.

[root@rhel-101 ~]# yum –y install chef-server-core-12.16.2-1.el7.x86_64.rpm

[root@rhel-101 log]# chef-server-ctl user-create chefadmin chef admin chefadmin@chef.io ‘xxx’ –filename /root/chefadmin.pem

[root@rhel-101 ~]# chef-server-ctl org-create chefenv ‘Chef Admin, Inc.’ –association_user chefadmin –filename /root/chefenv-validator.pem

[root@rhel-101 ~]# chef-server-ctl install chef-manage

[root@rhel-101 ~]# chef-manage-ctl reconfigure

    [root@rhel-101 ~]# chef-server-ctl status
run: bookshelf: (pid 8937) 74s; run: log: (pid 1689) 1790s
run: nginx: (pid 8952) 73s; run: log: (pid 1691) 1790s
run: oc_bifrost: (pid 8970) 73s; run: log: (pid 1695) 1790s
run: oc_id: (pid 9001) 72s; run: log: (pid 6478) 904s
run: opscode-erchef: (pid 9049) 71s; run: log: (pid 1693) 1790s
run: opscode-expander: (pid 9056) 72s; run: log: (pid 1679) 1792s
run: opscode-solr4: (pid 9091) 72s; run: log: (pid 1680) 1791s
run: postgresql: (pid 9098) 71s; run: log: (pid 1701) 1791s
run: rabbitmq: (pid 9055) 72s; run: log: (pid 1699) 1791s
run: redis_lb: (pid 9841) 19s; run: log: (pid 1682) 1791s

Step2: Install Chef DK on Workstation

Here we are going to configure the Linux workstation to act as Development platform for generating and authoring your cookbooks and recipes.

[root@desktop1 ~]# yum -y install chefdk-2.1.11-1.el7.x86_64.rpm

[root@desktop1 ~]# chef –version
Chef Development Kit Version: 2.1.11
chef-client version: 13.2.20
delivery version: master (73ebb72a6c42b3d2ff5370c476be800fee7e5427)
berks version: 6.3.0
kitchen version: 1.17.0
inspec version: 1.33.1

Step3: Configure your workstation to communicate with the Chef Server

For this step you need to download your chefadmin user private RSA key and knife.rb configuration file from Chef management console and copy into your workstation under .chef directory. This will enable knife to authenticate and communicate with the Remote Chef server

[root@desktop1 .chef]# pwd
/root/chefspace/.chef

[root@desktop1 .chef]# ls -ltr
total 8
-rw-r–r–. 1 root root  441 Aug 28 06:32 knife.rb
-rw-r–r–. 1 root root 1679 Aug 28 06:32 chefadmin.pem

You will need to fetch the Chef server certificate and store it in the local trust store for ssl validation of the Chef server

[root@desktop1 .chef]# knife ssl fetch

[root@desktop1 .chef]# knife ssl check
Connecting to host rhel-101.rhpds.opentlc.com:443
Successfully verified certificates from `rhel-101.rhpds.opentlc.com’

Step4: Upload your sample cookbook onto Chef server

Here, just create a simple helloworld cookbook under your chef workspace and upload the cookbook to the Chef server

[root@desktop1 cookbooks]# knife cookbook upload helloworld
Uploading helloworld   [0.1.0]
Uploaded 1 cookbook.

Step5: Create an EC2 t2.micro instance on AWS

We have our Chef server setup with a cookbook uploaded into it and workstation setup with Chef DK installed and communicating with Chef server. Now, launch t2.micro ec2 instance which we would be managing using Chef server. You can follow the procedure provided in below AWS article for launching an ec2 instance

https://learn.chef.io/modules/learn-the-basics/rhel/aws/set-up-a-machine-to-manage#/step2

Make sure that you have allowed SSH, HTTP and HTTPS ports on your ec2 instance and you are able to SSH into instance and have working internet connectivity.

The important things that you need to capture from the launched EC2 instance are its
Public IP address or Public DNS
And your Private RSA key which you can download while launching the EC2 instance in wizard mode.

Step6: Bootstrap using key based authentication

[root@desktop1 cookbooks]# knife bootstrap ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com –ssh-user ec2-user –sudo –identity-file /root/mychefadminec2.pem –node-name node1-rhel7 –run-list ‘recipe[helloworld]’
Creating new client for node1-rhel7
Creating new node for node1-rhel7
Connecting to ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com —–> Installing Chef Omnibus (-v 13)
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com downloading https://omnitruck-direct.chef.io/chef/install.sh
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com   to file /tmp/install.sh.1339/install.sh
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com trying curl…
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com el 7 x86_64
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Getting information for chef stable 13 for el…
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com downloading https://omnitruck-direct.chef.io/stable/chef/metadata?v=13&p=el&pv=7&m=x86_64
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com   to file /tmp/install.sh.1345/metadata.txt
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com trying curl…
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com sha1        d3d26412b6304c92f72749d00e62e0191ceada05
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com sha256      fe051b504856a74ccce1fd23ff92c296506cb8292a3933c71069ae915e7a4a00
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com url https://packages.chef.io/files/stable/chef/13.3.42/el/7/chef-13.3.42-1.el7.x86_64.rpm
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com version     13.3.42
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com downloaded metadata file looks valid…
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com downloading https://packages.chef.io/files/stable/chef/13.3.42/el/7/chef-13.3.42-1.el7.x86_64.rpm
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com   to file /tmp/install.sh.1345/chef-13.3.42-1.el7.x86_64.rpm
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com trying curl…
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Comparing checksum with sha256sum…
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Installing chef 13
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com installing with rpm…
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com warning: /tmp/install.sh.1345/chef-13.3.42-1.el7.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Preparing…                          ################################# [100%]
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Updating / installing…
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com    1:chef-13.3.42-1.el7               ################################# [100%]
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Thank you for installing Chef!
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Starting the first Chef Client run…
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Starting Chef Client, version 13.3.42
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com resolving cookbooks for run list: [“helloworld”]
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Synchronizing Cookbooks:
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com   – helloworld (0.1.0)
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Installing Cookbook Gems:
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Compiling Cookbooks…
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Converging 1 resources
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Recipe: helloworld::default
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com   * file[/tmp/hello] action create
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com     – create new file /tmp/hello
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com     – update content in file /tmp/hello from none to a591a6
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com     — /tmp/hello  2017-08-28 11:18:36.796112684 +0000
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com     +++ /tmp/.chef-hello20170828-1420-96ju1t        2017-08-28 11:18:36.796112684 +0000
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com     @@ -1 +1,2 @@
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com     +Hello World
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com     – restore selinux security context
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Running handlers:
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Running handlers complete
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com Chef Client finished, 1/1 resources updated in 03 seconds

Step7: Validate the bootstrapped node

[root@desktop1 cookbooks]# ssh ec2-user@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com -i /root/mychefadminec2.pem
Last login: Mon Aug 28 11:18:21 2017 from 104.xxx.xxx.xxx

[ec2-user@ip-172-31-22-149 ~]$ cat /tmp/hello
Hello World

Hope you enjoyed reading this article. Thank you.