CI/CD with Stackery & Travis CI

This guide is written for those building their own CI/CD solution for serverless. If you're starting from zero or want a ready-to-use solution from Stackery, feel free to skip right to our Verification Pipeline, which offers a solution for your functional tests, audit checks, and more.

Continuous integration/delivery is, fundamentally, concerned with modeling your software off a pipeline. If you’re reading this, you are likely aware of the plethora of tools designed to tackle this challenge. While we make Stackery with the mission of freeing you from infrastructure management so you can focus your time on application development, we also strive to integrate well with the tools you've already built a workflow around, like Travis CI.

The goal of this guide is to demonstrate how to automatically deploy a serverless application when commits are pushed to specific branches in the project's repository. By the end of this guide you'll have:

• One development environment to build, test, and manually deploy early versions of your serverless applications
• Two environments to automatically deploy staging and production-ready versions of those applications
• A customizable CI/CD workflow that integrates the Stackery CLI with Travis CI builds.

The source code used in this guide can be found in the Stackery + Travis CI repository

What you'll need:

• GitHub account (will be synced with Travis CI)
• Stackery Account with completed setup steps (Linked GitHub + AWS)
• Travis CI account
  • First 100 builds free for private repositories (orgs)
  • Always free for public repositories (open source)
• AWS Account to deploy

Stackery Setup

Add the environments

1. Log in to Stackery, head to the Environments page, and select Add an Environment

2. For this guide, create three separate environments dev, stg, and prod
3. Each environment will exist in that same AWS account and region, us-east-1

Add a stack

1. Next up, navigate to the Stacks page, then select Add a Stack

2. For the Git Hosting Provider select GitHub
3. Name it stackery-travis-demo, and create a new, blank repo in the Stackery GitHub

4. Once the stack is created, select the link under the stack name to go to the stack's GitHub repo
5. Clone the repo in a terminal window and cd into it, we'll come back to it shortly

Working with Travis CI

Setting up Travis CI

1. Head over to Travis CI to get started
2. If you already have an account go ahead and skip this section. If not, select 'Sign up with GitHub'
3. Setup steps are easy to follow and will guide you through the process of syncing up your GitHub repos with Travis CI

Gather deployment credentials

Two sets of credentials are needed to deploy a stack using Stackery:

1. AWS credentials (see the AWS docs for a description of access keys and how to create them)
2. Stackery credentials

To create Stackery credentials for your Travis CI builds login to Stackery then navigate to Settings > CLI Tokens. Create a new token:

Now copy the generated key and secret for use in the next step.

Storing secrets in Travis CI

1. With GitHub repos synced and populated on the Travis CI homepage, select the repo stackery-travis-demo
2. The .travis.yml and deploy.sh files don't exist in this repo yet, so we should be greeted with the following:

3. Before adding those files to our repo, we'll define the secrets that will be used in deploy.sh
4. Select More options on the top right, then Settings
5. On the Settings page, scroll down to Environment Variables and add the following using your AWS and Stackery credentials:

Travis CI files

1. Back in the terminal window, in stackery-travis-demo, add a file named .travis.yaml
2. Next, create a directory called scripts, then add a file into it named deploy.sh
3. stackery-travis-demo should now look like this:

stackery-travis-demo
  |scripts
    |- deploy.sh
  |.travis.yml
  |template.yaml

Configuring .travis.yml

Open stackery-travis-demo in a code editor paste in the following code snippet into .travis.yml

language: bash
branches:
  only:
    - stg
    - master
deploy:
  provider: script
  script: bash scripts/deploy.sh

We configure Travis CI to run using bash as the build language.

The branches property and its only sub property serve as a conditional for the build. It tells Travis to care only about changes that have been pushed to the stg or master branches in the repository. This is so Travis doesn't run this build and the Stackery deploy script every time we commit and push to our dev branch.

Travis will also recognize when a pull request has been opened on either stg or prod and run a pull request build that skips the deployment.

Under deploy we have a provider property that informs Travis we'll be feeding in a custom script to run on this build. The script property runs a bash command that searches this repo for scripts/deploy.sh and runs the script inside. The script includes the Stackery CLI commands needed to deploy our stack to CloudFormation, which we cover next.

Customizing Travis with deploy.sh

Paste in the following code snippet into scripts/deploy.sh

# Linux OS curl to download + install Stackery
curl -Ls --compressed https://ga.cli.stackery.io/linux/stackery > /tmp/stackery
sudo -i chmod a+x /tmp/stackery

stackery=/tmp/stackery
branch=$TRAVIS_BRANCH
repo=$TRAVIS_REPO_SLUG
repo=${repo#*/} # Remove 'github-owner/' from the string to retrieve 'repo-name'

# Set '$env' to 'prod' or branch name
[[ "$branch" == "master" ]] && env="prod" || env="$branch"

# Deploy to Stackery using AWS access and secret keys
echo "Deploying stack $repo to environment $env using git branch $branch"
$stackery deploy -n $repo -e $env -r $branch --non-interactive

If you've installed the Stackery CLI before, you'll recognize the first two lines. This curl command allows Travis to download and install Stackery into the virtual machine's tmp folder and modify it to be an executable file.

Lines 4-7 are variable definitions to improve the readability of the Stackery CLI commands that follow. Variables that begin with $TRAVIS are the default environment variables provided by Travis CI. The secret values that we populated this repo's settings with earlier can all be accessed by appending $ to the key name, like $STACKERY_EMAIL.

The next condition statement assigns the value prod to our $env variable when the branch being pushed to, is master. This is because we set up our environments to have a prod environment instead of naming it 'master'. Since this build only runs for the stg and master branches, the only other value $env could hold is stg.

The next CLI command is the most important: stackery deploy. This command deploys the stack stored in our repository to the AWS environment that matches up with the repository branch we've pushed to. It uses our Stackery credentials stored in Travis environment variables to prepare a change set, and then uses our AWS credentials, also stored in Travis, to access CloudFormation and deploy our stack. The --non-interactive argument instructs the CLI to immediately fail if it's missing any information instead of trying to prompt for information as though the command were run by a human user.

Initial Travis CI build

1. Back in the terminal, in stackery-travis-demo, commit and push these new changes to master
2. On the Travis CI site, notice the first build is running

3. Click on the job number on the right to open up the current build job, it will be in run for a few minutes

4. Once the build passes, open up the Job Log below

5. If we open up Deploying application, we'll see the messages logged in our bash script followed by AWS account info and the Stackery CLI deploying an empty stack to CloudFormation

6. On the Stackery homepage, we can verify the stack has been deployed to the correct prod environment under Recent Activity

Now that our master branch has been pushed to our GitHub repo, any branches derived from master will have the required .travis.yml and deploy.sh files for them to trigger a Travis CI build. A .travis.yml file needs to be present on all active branches in the repo

Bringing it all together

This section illustrates the following example workflow:

• Develop the serverless application with Stackery in the dev branch
• In GitHub, submit and approve a pull request to merge changes in dev into stg branch
• Once merged in GitHub, Travis CI runs our custom script to deploy an updated stack to the stg environment in AWS

Stackery

1. In the stack stackery-travis-demo, we'll create two branches dev and stg each branching from master. When we were configuring our Travis files, we mentioned that we want these two branches to match the Stackery Environments they'll be deploying to.

   After creating the dev branch, select master from the drop down option again, then Create branch to create the stg branch. This is to ensure both branches stem from master and to avoid possible merge conflicts on future pull requests.

2. In the dev branch, add a Function resource (don't worry about configuring it)
3. Select Commit and confirm.
4. Commiting in Stackery will push these changes to the dev branch.

This push will not trigger a Travis CI build since we specified in .travis.yml to only run when we push to the stg or master branches.

GitHub

1. In the stackery-travis-demo repo, submit a pull request to merge the changes in dev into the stg branch

2. This pull request will start a new build in the Travis dashboard, but will not run the deploy.sh script. In this guide, we're more interested in the Travis build that occurs as a result of the merge

   The Job Log of a pull request Travis build has the following message at the very bottom:

3. Other developers will typically review the pull request, but let's assume this a perfect PR and merge it in.

Travis CI

1. Soon after the pull request is merged, notice a new build running for the stg branch in our Travis CI dashboard
2. Once it passes, open up Deploying application in the Job Log to verify the stack has been updated successfully

You can also verify in the AWS console, that Function resource has been created in the stg environment

Repeat this process when the stack in the stg environment completes its required tasks or tests and is ready to be deployed to the prod environment.

Developing and deploying your stack is now a part of a CI/CD process!

Again, there are preexisting tools to help you with the middle of the continuous integration and delivery process. Stackery comes in to help at both the beginning and end of your existing CI/CD procedure: initially, when you are creating and defining your infrastructure and finally, during the namespacing of environments across multiple AWS accounts.

With Stackery’s seamless integration into your existing toolkit, CI/CD is truly continuous— throughout the entire lifecycle of development.