Image Processing

Use Case

If your users can upload pictures, one feature you may want is to automatically create smaller versions of the pictures. These smaller versions will be better suited for serving to mobile users or to use as thumbnails. In this guide, we’ll show how to accomplish this with Stackery.

In this guide, we will be using 2 node types:

  • Function nodes to do the image processing and handle/filter upload notifications
  • ObjectStore nodes to store the images from the user

Prerequisites

How To

The overall architeture of this stack to this will look like this: Image Processing Stack

A complete example stack repo is available at https://github.com/stackery/image-processing-example.

Follow these steps to build out the architecture.

  1. Create a new Stackery stack.
    • Select a git provider (In order to follow along, it will be easiest if you choose GitHub. If you want to use AWS, you’ll need to understand how to clone from AWS CodeCommit).
    • Set the Stack Name to ‘imageProcessing’ or another unique name of your choice.
    • Delete the 4 default nodes.
  2. Add an ObjectStore node .
    • This is where users will upload pictures.
    • Set the Name to Uploaded Images.
  3. Add a Function node .
    • This function will modify the image and move it to the final ObjectStore.
    • Set the Name to imageProcessor.
    • Connect the input to the notifier functions’s output.
    • Set the Outputs to 1.
    • Set the Timeout to 120 seconds.
  4. Add an ObjectStore node.
    • Set the Name to Processed Images.
    • Connect the input to the imageProcessor function output port
  5. Click the Save button to save the stack.

In the infrastructure provisioing and deployment process, Stackery will correctly set all IAM permissions for the interactions between the various nodes.

Now let’s flesh out the imageProcessor function code. We will need to clone the repo and make some changes locally.

  1. Clone your git repo
  2. In your editor of choice open <repo>/Stackery/functions/imageProcessor/index.js
    • Overwrite the existing code with the following:
      const AWS = require('aws-sdk');
      const gm = require('gm').subClass({imageMagick: true});
      const s3 = new AWS.S3();
      
      module.exports = function handler (event, context, callback) {
        console.log(event);
      
        let record = event.Records[0];
        if (record.eventName !== 'ObjectCreated:Put') {
          return;
        }
        let sourceBucket = record.s3.bucket.name;
        let objectKey = record.s3.object.key;
      
        // Only operate on image files
        // For simplicity in this guide, we'll only operate on '.jpg' files
        let result = objectKey.match(/(.*)\.jpg$/);
        if (!result) {
          return;
        }
        let imageBuffer;
        let params = {
          Key: objectKey,
          Bucket: sourceBucket
        };
      
        // Retrieve image from ObjectStore "Uploaded Images"
        s3.getObject(params).promise()
          .then((data) => {
            return new Promise((resolve, reject) => {
              imageBuffer = data.Body;
      
              // imageMagick create a 200x200 thumbnail
              gm(imageBuffer)
                .resize(200, 200)
                .stream((err, stdout, stderr) => {
                  let chunks = [];
      
                  stdout.on('data', (chunk) => {
                    chunks.push(chunk);
                  });
      
                  stdout.on('end', () => {
                    resolve(Buffer.concat(chunks));
                  });
      
                  if (err) {
                    console.log(`Error resizing image: ${err}`);
                    reject(err);
                  }
      
                  stderr.on('data', function (data) {
                    console.log(`Error resizing image: ${data}`);
                    reject(new Error('Error resizing image'));
                  });
                });
            });
          })
          .then((outputBuffer) => {
            // Store generated thumbnail to Object Store "Processed Images"
            // We look up the S3 bucket using the STACKERY_PORTS env var
            const ports = JSON.parse(process.env.STACKERY_PORTS);
            let params = {
              Body: outputBuffer.toString('binary'),
              Key: `200x200-${objectKey}`,
              Bucket: ports[0][0].bucket
            };
            return s3.putObject(params).promise();
          })
          .then((response) => {
            // Done!
            callback(null, {});
          })
          .catch((error) => {
            callback(error);
          });
      };
      
      • Save and close the file
  3. Create a new file <repo>/Stackery/functions/imageProcessor/package.json
    • Copy the following code into the file:
        {
          "name": "ImageProcessor",
          "version": "",
          "dependencies": {
            "aws-sdk": "^2.184.0",
            "bluebird": "^3.5.0",
            "gm": "^1.23.0"
          }
        }
      
    • Save and close the file
  4. Commit your changes
  5. Push your changes back up to the git repo.

Now all that is left is to deploy the stack!

Deploy via Stackery CLI

Test it out

  1. Click on the deployment in the Current Deployment section.
  2. Select the Uploaded Images ObjectStore node.
  3. Copy the S3 Bucket Name.
  4. Open https://s3.console.aws.amazon.com/s3/buckets/<S3 Bucket Name> in a new tab.
  5. Upload an image to the S3 Bucket.
  6. Go back to the Stackery Dashboard.
  7. Select the Processed Images ObjectStore node.
  8. Copy the S3 Bucket Name.
  9. Open https://s3.console.aws.amazon.com/s3/buckets/<S3 Bucket Name> in a new tab.
    • You should see the original image and a thumbnail suffixed with ‘x200’

And just like that, you have a simple, serverless-based, automatic thumbnail generation!

Try Stackery For Free

Gain control and visibility of your serverless operations from architecture design to application deployment and infrastructure monitoring.