Background

I have, as yet, been unable to determine how AWS generates its list of AMI IDs in the “Launch Instance Quick Start” as seen here, since these Image IDs are not always the most recent and are not consistently aged across OS versions. Because of this, I wanted to write myself a command line tool that could approximate a standard, repeatable process for selecting an AMI ID that fit my interpretation of the various OS vendors inclinations for AMI ID selection.

For instance, RHEL has this page that indicates the single number preceding ‘-Hourly2’ is the different kernel versions. They tell us “you should probably be using the latest kernel version where possible.” Canonical publishes the Amazon EC2 AMI Locator for the different versions of Ubuntu. This appears to return the most recent image ID. For Amazon Linux 2, I relied on similar logic to Ubuntu.

It is important to remember that AMI IDs are REGION specific. If you generate your credentials using the aws_saml_auth script, you can specify the region and this script will be able to call using the temporary credentials provided they are set as the default credential profile.

The Script

Breakdown

: ${1?"Usage: $0 RHEL||Amazon||Ubuntu"} performs parameter validation to verify that we have passed an argument to the script. This does not validate whether or not the text is correct - just that there is A value. If no argument is passed, we display a usage “error” message: Usage: $0 RHEL||Amazon||Ubuntu.

Next, we move into the validation of the first argument (additional arguments have no effect). We look for the text RHEL, Amazon, or Ubuntu, otherwise we send a message to the user that they need to choose one of those options.

Within each proper OS option, we run the aws ec2 describe-images command. Each has a filter based on a standard version that we have validated and are using now for production systems (RHEL7.4, Amazon2 2017.12, Ubuntu Xenial 16.04). The filter usually fills in for the years in the name, but doubles for the kernel revision for RHEL. This is followed by a query using JMESPath, which differs slightly between RHEL and the others. Finally, we tell the CLI to output text, not JSON.

For RHEL, we have the most complicated command. We start off the same, with the owner and filter defined, follwed by the JMESPath query, looping through images that match, selecting their Image ID, Name, CreationDate, and OwnerID. Those details aren’t used, but are useful to have if we want to see more info about this image. The text output looks like this:

ami-26ebbc5c    RHEL-7.4_HVM-20180103-x86_64-2-Hourly2-GP2      2018-01-04T01:55:42.000Z        309956199498
ami-76a3970c    RHEL-7.4_HVM-20180122-x86_64-1-Hourly2-GP2      2018-01-22T17:34:08.000Z        309956199498

We use sed with capture groups to move the ami ID to the end of the line: sed 's/\(\S*\)\(.*\)/\2 \1/'. This command groups all non-whitespace prior to the first space (i.e., amiID) into group 1 and everything else into group 2. This is accomplished by use of the escaped parantheses (\(\)) and is not unique to sed (for more on regex capture groups, see the Introducing Regular Expressions by Michael Fitzgerald, O’Reilly Media, Inc., 2012). Next, we use awk to remove all text before the AMI architecture (awk -F'x86_64-' '{print $2}'). This leaves us with the kernel revision specified in the aforementioned redhat solution page as the first character on the line. We are then able to sort to put the highest number last, grabbing the last line with tail and using awk to extract the AMI ID alone: sort | tail -n 1 | awk -F'ami-' '{print "ami-"$2}'.

For Amazon or Ubuntu, the command is much easier. We filter based on the name and use a query to retrieve the most recent image based on creation date: --query 'sort_by(Images, &CreationDate)[-1].[ImageId]'. JMESPath will return just the instance ID in htis case.

Usage

Below are the three examples and their output in us-east-1 as of 20180320:

# Set credentials
python aws_saml_auth.py

# Get RHEL ID
bash get_recent_ami.sh RHEL

# Get Amazon ID
bash get_recent_ami.sh Amazon

# Get Ubuntu ID
bash get_recent_ami.sh Ubuntu