Stream

Stream nodes receive messages and insert them into a high-availability, high-throughput queue. Every node connected to the output port will receive copies of each message in the stream.

The stream will retry outputting a message to a node if the node fails to process the message. The stream will continue to output the message until either the node processes the message successfully or the message expires after 24 hours.

Message Formats

Events are sent to Function nodes connected to the output of Topic nodes. Messages are batched together into events to ensure high throughput processing. Here is an overview of commonly used event properties:

Property Type Description
Records Array Array of event records.
Records[*].kinesis.partitionKey string Message partition key. Example: partitionKey-3
Records[*].kinesis.data string Message data in Base64 format


See https://docs.aws.amazon.com/lambda/latest/dg/eventsources.html#eventsources-kinesis-streams for further details on the event format.

Settings

Name

A label used for the node in the canvas.

Shard Count

The number of shards in the stream.

Messages are inserted into a single random shard. Each shard can send one copy of a message at a time to each node connected to the output port. Each message is output by each shard in the order it was received.

Stackery Port Data

When connecting a wire between the output of a Function node to the input of an Object Store node, the STACKERY_PORTS environment variable for the Function node will contain the following information in JSON form:

Property Type Description
type string The type of the connected resource. Always set to stream.
id string The Stackery Stream node’s ID. Example: abcd1234.
name string The Stackery Stream node’s name. Example: Records.
streamName string The name of the AWS Kinesis data stream. Example: stackery-98237489374-Streamabcd1234
streamArn string The AWS ARN of the AWS Kinesis data stream. Example: arn:aws:kinesis:us-west-2:123456789012:stream/stackery-98237489374-Streamabcd1234


See Function Output Port Connections for more details on the STACKERY_PORTS environment variable.

Referenceable Properties

The following properties may be referenced by Function, Docker Service, and Docker Task nodes:

Property Value Example
id Cloud Service Identifier arn:aws:kinesis:us-east-1:0123456789012:stream/stackery-298374238-Stream8fd73k29s

Try Stackery For Free

Gain control and visibility of your serverless operations from architecture design to application deployment and infrastructure monitoring.