Baking AMI Image in Spinnaker CD Pipeline

To create an EC2 instance, or maintain high-availability of the application with auto-scaling-group requires a base image in AWS. This base image should contain the application that you want to install. 

In this article, we will be covering the procedure to bake an AMI image in the Spinnaker CD pipeline.

Pre-Requisites

For setting up a successful baking stage in Spinnaker, the requirements are as follows:

  • A working Spinnaker instance.
  • Access to the AWS console with a valid account.
  • Working knowledge of AWS services like IAM and EC2 including AMI, EC2 instance, AutoScaling Group.
  • Working knowledge of the Linux environment.

Spinnaker Pipeline requires the following:

  • A Jenkins job that publishes debian packages (for Ubuntu image build), so Spinnaker can read the package name.
  • A debian repository to install application using the .deb packages.

Understanding the Baking Process

The process of baking an AMI image refers to creating an immutable base Image that becomes the basis for creating application EC2 instances or server-groups. This process of installing an application on the existing OS, and generating an AMI from the current state is known as Baking AMI.

For Ubuntu operating systems, you may use the `apt` command-line program to install from a debian repository. In Spinnaker, you need to mention the repository from where to fetch the application, and the installation process is abstracted implicitly.

Spinnaker’s Rosco service is responsible for governing the baking process. So, you need to provide the package repository location and the IAM credentials to connect with your AWS account. Under the hood, the Rosco is using `packer` to perform AMI baking.

Typically the baking process takes place after a successful CI build process which would generate a .deb package and publish to a repository. Hence Baking AMI process depends on a CI system like Jenkins.

Creating IAM User with Baking permissions

In order for Spinnaker to connect with AWS, it should have a proper credential. This credential is provided by the AWS admin. 

The AWS admin sets up IAM credentials as below:

  1. Go to IAM in AWS Console. Users > Add User > Name ‘SpinBakeryUser’. Select ‘Programmatic access’.
  2. Next: Tags > Have optional tags as required
  3. Review and Create. Make a note of the ACCESS_KEY and SECRET_KEY
  4. Next, Add a permission policy with a dedicated policy ‘AMIBakeryPolicy’ or with an Inline Policy to the user. The policy JSON as as below.
{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Action": [
                 "ec2:*"
           ],
           "Resource": [
                   "*"
           ],
           "Effect": "Allow"
       }
   ]
}

Configuring Rosco Service to access Debian Packages

Remember, Custom profiles are configuration files that override the default configuration files for each Spinnaker Service. These files are created in Halyard path: ~/.hal/default/profiles/

So, to let Rosco know from where to read the Debian packages, you need to update the Rosco profile with a custom rosco-local.yml file. The content as mentioned below:

debianRepository: http://demojenkins.net:8181/ trusty main

It is important that Halyard and Rosco are able to connect with the repository. You may check the connectivity from Halyard by using the following command :

nc -zvw5 demojenkins.net:8181

Applying the Configurations via Halyard

Once an IAM user is created with sufficient EC2 permissions and Rosco is configured with a debian repository, you can safely enable the AWS bakery flags. Run the following commands in Halyard:

export ACCESS_KEY_ID=xxxxxxxxxx
hal config provider aws bakery edit --aws-access-key ${ACCESS_KEY_ID} --aws-secret-key #You will be prompted for secret
# Optionally, you can specify a default subnet to bake
# hal config provider aws bakery edit --aws-subnet subnet-abc012345
hal deploy apply

Note: In case you did not mention the target subnet in here, you can still configure the VPC and Subnet in the Pipeline Bake stage.

Bake Stage in Pipeline

The Bake stage depends on a successful build on Jenkins. Jenkins should have a post-build Archive step for Spinnaker to read the packages from it. You should have integrated Jenkins beforehand using Halyard. 

Pipeline Configuration

In the Pipeline configuration stage, under Trigger Type, choose ‘Jenkins’ and the Controller to ‘<Jenkins master-name>’. The Job  name should be the one that generates the debian package.

Bake Stage 

Once the pipeline is configured, follow the steps given below:

  1. In the Pipeline, add a stage after Configuration stage. Let the type be ‘Bake’. 
  2. In the ‘Bake Configuration’ select a region to bake the AMI and type the package name without the version information. 
  3. Choose a Base OS from the available list.

Here, the Bake stage configuration screen for your reference.

  1. After setting up the Bake Configuration, go back to the Pipeline Execution page and trigger a manual build. Wait for the pipeline execution to be completed. 
  2. Once it is done, you can review the Baking log to identify the AMI id the pipeline has created.

Common Errors and Fixes

1. Error: Pipeline exits with the log : 

packer_5f4ee5e0-c7d8-6524-7f28-bcfc6a516c69
Build ‘amazon-ebs’ errored: Error creating temporary keypair: retry count exhausted. Last err: UnauthorizedOperation: You are not authorized to perform this operation. Encoded authorization failure message: SKHQSzSTAfx- *********************_-0GSM0gCHuTt
status code: 403, request id: 713d9451-6674-47b9-9ed4-99f986a912f9

  • Cause: The bakery user does not have sufficient permission to create Keypair and EC2 instances.
  • Fix:  Go to the AWS console, and edit User’s policy with “ec2:*” to Actions.
2. Error: Fail to fetch: 

W: Failed to fetch http://demojenkins:8181/dists/trusty/InRelease

    amazon-ebs:
   amazon-ebs: W: Failed to fetch
http://demojenkins:8181/dists/trusty/Release.gpg  Could not resolve 'demojenkins'

   amazon-ebs:
   amazon-ebs: W: Some index files failed to download. They have been ignored, or old ones used instead.
  • Cause: The debian repository server is not reachable.
  • Fix:  Check the connectivity using `ping <server>`, or else try using the server’s IP address. Also make sure the debian repository service (Aptly is running) on the remote server.

Summary

We have explained the procedure to perform a successful AMI baking through Spinnaker CD pipeline by creating an IAM user with sufficient permissions, configuring Halyard with debian-repository and enabling AWS bakery configuration. Now that you know the process go ahead and give it a try yourself!

Leave a Comment

Your email address will not be published.

You may like