Serverless CI with Audits & Tests

Overview

Stackery now offers a set of continuous integration functionality to help you and your team verify, review, and iterate on changes before they go live. We call this the verification pipeline which includes:

• Dependency vulnerability checks catch known upstream security vulnerabilities in packages used by your functions
• Functional tests are executed against stack sandboxes, eliminating the need for you to maintain stacks for tests
• Preview deployments are updated on each change and destroyed when the PR is closed, facilitating team-based review processes

Each of these are executed as parallel steps when a pull-request is opened against a participating stack repository. The preview stack is refreshed when commits are pushed to the branch associated with the pull-request, and is un-deployed when the PR is merged or closed.

How Does It Work?

Stackery integrates with your Git provider to help you manage the source code for your stacks. When you enable Stackery's Verification Pipeline Stackery will subscribe to pull request (PR) events from your Git provider to be notified when a PR is opened, updated, merged, and closed.

When a PR is opened or updated, Stackery will start parallel processes for dependency vulnerability checks, functional tests, and deploying or updating a preview deployment. Each of these processes are performed within an AWS CodeBuild job in your own AWS account. This ensures code confidentiality during audit and build phases and provides you with complete visibility into how each step is performed.

Dependency Vulnerability Checks

This step checks out the source code for the PR within an AWS CodeBuild job, locates each Function in the project, then runs the dependency vulnerability check appropriate for the function's runtime. For example, npm audit is performed for Node.js runtimes. This step will report success back to your Git provider if all functions pass the audit checks.

Functional Tests

This step deploys the changes in the PR to a sandboxed test deployment. An AWS CodeBuild job is then executed which checks out the source code for the PR, locates each Function in the project, and runs tests for the function. Appropriate tests are run based on the runtime of each function. For example, npm test is executed for each Node.js function.

Information about deployed resources is made available in stack-info.json, which contains data equivalent to running stackery describe --json.

The sandboxed test deployment is deprovisioned once all tests have completed. This step will report success back to your Git provider if all function tests pass.

Preview Deployments

This step creates a sandboxed preview deployment that exists for the life of the PR. The preview deployment is updated each time changes are pushed to the PR. The preview deployment is deprovisioned when the PR is closed. Clicking the preview deployment link takes you to the deploy view of your PR in the Stackery Dashboard.

Runtime Support

The following table provides the supported runtimes for dependency vulnerability checks and functional tests as well as the underlying command used in each step.

Runtime	Dependency Vulnerability Check	Functional Tests
Node.js	npm audit	npm test
Python	safety check	pytest
.NET	N/A	dotnet test

Supported Git Providers

Verification Pipelines can be enabled on stacks backed by the following git providers:

• GitHub
• GitLab
• Bitbucket

Setup

Enabling/Disabling

Each stack has a Verification Pipeline that can be enabled/disabled. To enable the Verification Pipeline, enable at least one of the pipeline steps mentioned below. Disable all pipeline steps in order to disable the Verification Pipeline.

1. Run the following command to get the latest version of the Stackery CLI:

stackery update

2. Provision the stackery-deployer role in your AWS account. This will grant Stackery permission to deploy to ephemeral test environments on your behalf:

stackery aws update-role --deployments enable

Stackery Deployer Role

To facilitate the management of preview stacks for use in verification pipelines, Stackery requires additional permissions to deploy and provision resources into your AWS account.

The stackery-deployer role uses the “AdministratorAccess” policy that exists in the AWS account linked to Stackery.

As an alternative to granting “AdministratorAccess”, you can provide the ARN of an IAM policy for the stackery-deployer role to use. However, it is your responsibility to manage this IAM policy such that the stackery-deployer role has permissions to create, read, update and destroy each resource in the stack.

stackery aws update-role --deployments enable --deployer-policy-arn <your-custom-policy-arn>

Existing stack deployment functionality remains the same. The stackery-deployer role is only used for Stackery verification pipeline and automated deployment functionality.

Enabling the pipeline

1. In Stackery, select the stack you wish to enable the Verification Pipeline on, then navigate to the stack Settings page

2. Enable one or more of the Verification Pipeline steps

3. Choose an environment to use for Verification Pipeline deployment parameters and secrets

4. Select Save

Initiating the pipeline with a pull or merge request

1. After completing work on a feature branch, navigate to the stack repository and create a new pull request

2. Verification Pipeline steps will occur within moments of creating a pull

Re-Authorizing Bitbucket

Some Bitbucket users will be required to re-authorize Bitbucket in order to enable the webhook used to trigger the Verification Pipeline on pull requests. Earlier authorization of Bitbucket (i.e. on stack creation) does not contain the necessary permissions for the Verification Pipeline.

Monitoring the pipeline

GitHub Pull Requests

When a step is in progress or completed, selecting Details will redirect to the associated AWS CodeBuild job in your AWS console.

GitLab Merge Requests

When a step is in progress or completed, selecting a pipeline step will redirect to the associated StackeryPipelineRunner CodeBuild job in your AWS console.

Bitbucket Pull Requests

When a step is in progress or completed, selecting a pipeline step will redirect to the associated StackeryPipelineRunner CodeBuild job in your AWS console.

Deploying Serverless Framework stacks

Serverless Framework stacks can also be deployed through the Verification Pipeline. The deployment process must be augmented to support Serverless Framework projects:

1. Create a file in the project root directory (or the same level dir as the serverless.yml file) and name it stackery.deployspec.yaml

2. Add the following YAML to stackery.deployspec.yaml:

stages:
 setup:
   - stackery deploy-stage setup
 build:
   - stackery deploy-stage convert-edge-functions
   - stackery deploy-stage build
 addIntegrations:
   - stackery deploy-stage add-integrations
   - stackery deploy-stage add-environment-context
 package:
   - stackery deploy-stage package
   - stackery deploy-stage restore-edge-functions
 submitChangeSet:
   - stackery deploy-stage submit-change-set
 executeChangeSet:
   - stackery deploy-stage execute-change-set

This file opts serverless framework projects into the new deployment stages necessary for Verification Pipelines.

3. Run the stackery deploy command to verify successful stack deployment

stackery deploy

Your stack is ready for the Verification Pipeline once a deployment completes successfully.