Part III: Practical Examples of DevOps Unit Testing

Example unit test for a Terraform project that creates a simple S3 bucket in AWS to address public cloud storage.

In my last two articles, I’ve talked conceptually and theoretically about the need for DevOps testers.

Part I: Does DevOps Need Dedicated Testers?
Part II: 2019 Cloud Breaches Prove DevOps Needs Dedicated Testers

In this article, I will provide practical examples of unit testing.

Since public cloud storage seems to be a common problem, I will begin with an example unit test for a terraform project which creates a simple S3 bucket.

First, we need to install localstack, so we can test AWS locally.

pip install localstack
export SERVICES=s3
export DEFAULT_REGION='us-east-1'
localstack start

In a new console/terminal and new directory, create a simple terraform project. The provider.tf file should point to the localstack ports.

provider "aws" {
	region = "us-east-1"
	skip_credentials_validation = true
	skip_metadata_api_check = true
	s3_force_path_style = true
	skip_requesting_account_id = true
	skip_get_ec2_platforms = true
	access_key = "mock_access_key"
	secret_key = "mock_secret_key"
	endpoints {
    	s3 = "http://localhost:4572"
	}
}

resource "aws_s3_bucket" "b" {
  bucket = "test"
  acl    = "private"

  tags = {
	Name    	= "My bucket"
	Environment = "Dev"
  }
}

Deploy the terraform project.

terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

aws_s3_bucket.b: Refreshing state... [id=test]

------------------------------------------------------------------------


An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_s3_bucket.b will be created
  + resource "aws_s3_bucket" "b" {
  	+ acceleration_status     	= (known after apply)
  	+ acl                     	= "private"
  	+ arn                     	= (known after apply)
  	+ bucket   	               = "test"
  	+ bucket_domain_name      	= (known after apply)
  	+ bucket_regional_domain_name = (known after apply)
  	+ force_destroy           	= false
  	+ hosted_zone_id          	= (known after apply)
  	+ id                          = (known after apply)
  	+ region                  	= (known after apply)
  	+ request_payer           	= (known after apply)
  	+ tags                    	= {
      	+ "Environment" = "Dev"
      	+ "Name"	    = "My bucket"
    	}
  	+ website_domain          	= (known after apply)
  	+ website_endpoint        	= (known after apply)

  	+ versioning {
      	+ enabled	= (known after apply)
      	+ mfa_delete = (known after apply)
    	}
	}

Plan: 1 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terrafor can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

$ terraform apply
aws_s3_bucket.b: Refreshing state... [id=test]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_s3_bucket.b will be created
  + resource "aws_s3_bucket" "b" {
  	+ acceleration_status     	= (known after apply)
  	+ acl                     	= "private"
  	+ arn         	            = (known after apply)
  	+ bucket                  	= "test"
  	+ bucket_domain_name      	= (known after apply)
  	+ bucket_regional_domain_name = (known after apply)
  	+ force_destroy           	= false
  	+ hosted_zone_id          	= (known after apply)
  	+ id                      	= (known after apply)
  	+ region                  	= (known after apply)
  	+ request_payer           	= (known after apply)
  	+ tags                    	= {
      	+ "Environment" = "Dev"
      	+ "Name"    	= "My bucket"
    	}
  	+ website_domain          	= (known after apply)
  	+ website_endpoint        	= (known after apply)

  	+ versioning {
      	+ enabled	= (known after apply)
      	+ mfa_delete = (known after apply)
    	}
	}

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions in workspace "kitchen-terraform-base-aws"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_s3_bucket.b: Creating...
aws_s3_bucket.b: Creation complete after 0s [id=test]

Create a test.py file with the following code to test the deployment of the S3 bucket.

import boto3


def test_s3_bucket_creation():
	s3 = boto3.client(
    	's3',
    	endpoint_url='http://localhost:4572',
    	region_name='us-east-1'
	)
	# Call S3 to list current buckets
	response = s3.list_buckets()

	# Get a list of all bucket names from the response
	buckets = [bucket['Name'] for bucket in response['Buckets']]

	assert len(buckets) == 1

Test that the bucket was created.

$ pytest test.py
=============================================================== test session starts ===============================================================
platform darwin -- Python 3.6.0, pytest-5.2.2, py-1.8.0, pluggy-0.13.0
rootdir: /private/tmp/myterraform/tests/test/fixtures
plugins: localstack-0.4.1
collected 1 item

test.py .

Now, let’s destroy the S3 bucket.

$ terraform destroy
aws_s3_bucket.b: Refreshing state... [id=test]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # aws_s3_bucket.b will be destroyed
  - resource "aws_s3_bucket" "b" {
  	- acl                     	= "private" -> null
  	- arn                     	= "arn:aws:s3:::test" -> null
  	- bucket                  	= "test" -> null
  	- bucket_domain_name      	= "test.s3.amazonaws.com" -> null
  	- bucket_regional_domain_name = "test.s3.amazonaws.com" -> null
  	- force_destroy           	= false -> null
  	- hosted_zone_id          	= "Z3AQBSTGFYJSTF" -> null
  	- id                      	= "test" -> null
  	- region                  	= "us-east-1" -> null
  	- tags                    	= {
      	- "Environment" = "Dev"
      	- "Name"    	= "My bucket"
    	} -> null

  	- object_lock_configuration {
    	}

  	- replication_configuration {
    	}

  	- server_side_encryption_configuration {
    	}

  	- versioning {
      	- enabled	= false -> null
      	- mfa_delete = false -> null
    	}
	}

Plan: 0 to add, 0 to change, 1 to destroy.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

aws_s3_bucket.b: Destroying... [id=test]
aws_s3_bucket.b: Destruction complete after 0s

Destroy complete! Resources: 1 destroyed.

Next, we will install the terraform-compliance python module.

pip install terraform-compliance

Next, we will set up the directory for our test.

mk features
cd features

Next, make a file named s3.features inside the features directory with the following content.

Feature: test

 	In order to make sure the s3 bucket is secure:

 	Scenario: No public read
     	Given I have AWS S3 Bucket defined
     	When it contains acl
     	Then its value must not match the "public-read" regex

Now, we will return to the root directory for the project and run a terraform plan to get the plans output in json format.

terraform plan -out=myout
terraform show -json myout > myout.json

Lastly, we will test the terraform project against the feature file to see if the project is compliant.

$ terraform-compliance -p /tmp/junk/myout.json -f /tmp/junk/features
terraform-compliance v1.1.7 initiated

🚩 Features : /tmp/junk/features
🚩 Plan File : /tmp/junk/myout.json

🚩 Running tests. 🎉

Feature: test  # /tmp/junk/features/s3.feature
	In order to make sure the s3 bucket is secure:

	Scenario: No public read
    	Given I have AWS S3 Bucket defined
    	When it contains acl
    	Then its value must not match the "public-read" regex

1 features (1 passed)
1 scenarios (1 passed)
3 steps (3 passed)

As you will notice from the results, all tests passed because the S3 bucket deployed is private.

While these are just basic examples, they are intended to demonstrate the concept of unit testing infrastructure-as-code, and testing for various rules.

Read the Entire DevOps Testing Series