A Stream represents an Amazon Kinesis Stream. A Kinesis Stream can be used to collect, process, and analyze large amounts of video and data streams in real time. The stream consists of shards, which hold data records and organize data to be consumed.
Key Features of Amazon Kinesis Streams include:
- Scalable, fault-tolerant consumption of continuous data
- Quickly aggregate data into a data warehouse or map-reduce cluster
- Automatically encrypts sensitive data as it enters a stream
On one end of a Stream you have a producer service submitting data and on the other end you have a consuming service ingesting that data. A web service submitting log data is an example of a producing resource. A data warehouse set up to perform analysis on this large amount of analytics is an example of a consuming resource.
Event subscription wires (solid line) connect two or more resources and are required to accomplish certain tasks or improve performance within an application's flow.
The following resources can be subscribed to a Streams:
Service discovery wires (dashed line) provide compute resources (Function, Edge Function, Docker Task) with the permissions and environment variables required to perform actions using cloud resources within the stack. This resource is on the receiving end of a service discovery wire originating from compute resources.
The following compute resources can use a service discovery wire to access a Stream resource:
Human readable name for this resource that is displayed on the Stackery Dashboard and Stackery CLI output.
The number of shards in the stream.
Messages are inserted into a single random shard. Each shard can send one copy of a message at a time to each resource connected to the output port. Each message is output by each shard in the order it was received.
Use Existing Kinesis Stream
When enabled, this feature provides you with a field to specify the Amazon Resource Name (ARN) of an existing Kinesis Stream to reference in your application.
You may reference an environment parameter in order to conditionally reference existing infrastructure based on environment.
When connected by a service discovery wire (dashed wire), a Function or Docker Task will add the following IAM policy to its role and gain permission to access this resource.
Grants a Function or Docker Task permission to create, read, update, and delete video and dat shards from your Stream in the stack, or an existing Kinesis Stream outside of the current stack.
Grants a Function or Docker Task permission to read from video and data shards from your Stream in the stack, or an existing Kinesis Stream outside of the current stack.
When connected by a service discovery wire (dashed wire), a Function or Docker Task will automatically populate and reference the following environment variables in order to interact with this resource.
The Logical ID of the Stream resource.
The Amazon Resource Name of the Kinesis Data Stream.
Related AWS Documentation
AWS Documentation: AWS::Kinesis::Stream