> ## Documentation Index
> Fetch the complete documentation index at: https://sequinstream.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# AWS Kinesis sink

> Reference for configuring and using the AWS Kinesis sink with Sequin.

The Kinesis sink sends messages to an Amazon Kinesis data stream.

Kinesis is a durable, distributed, sharded queue that stores messages for up to 1 year, with ordering guaranteed within each shard based on partition keys.

Unlike SQS or SNS, Kinesis maintains message ordering and enables multiple consumers to process the same data stream independently.

<Tip>
  This is the reference for the Kinesis sink. See the [quickstart](/quickstart/kinesis) for a step-by-step walkthrough or the [how-to guide](/how-to/stream-postgres-to-kinesis) for an explanation of how to use the Kinesis sink.
</Tip>

## Configuration

* **Stream ARN**

  The ARN of your Kinesis data stream. Must be in the format `arn:aws:kinesis:<region>:<account-id>:stream/<stream-name>`.

<Snippet file="aws-access-key-config.mdx" />

## IAM permissions

Sequin requires the following AWS permissions to send messages to Kinesis:

```json theme={null}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["kinesis:PutRecords", "kinesis:DescribeStream"],
      "Resource": "<your-stream-arn>"
    }
  ]
}
```

Replace `<your-stream-arn>` with your actual stream ARN (e.g., `arn:aws:kinesis:us-east-1:123456789012:stream/my-stream`).

<Snippet file="aws-iam-settings.mdx" />

## Configuration examples

### Using access keys

```yaml theme={null}
sinks:
  - name: "kinesis-sink"
    batch_size: 10
    destination:
      type: "kinesis"
      stream_arn: "arn:aws:kinesis:us-east-1:123456789012:stream/my-stream"
      access_key_id: "AKIA..."
      secret_access_key: "your-secret-key"
```

### Using task role (recommended for AWS environments)

```yaml theme={null}
sinks:
  - name: "kinesis-sink"
    batch_size: 10
    destination:
      type: "kinesis"
      stream_arn: "arn:aws:kinesis:us-east-1:123456789012:stream/my-stream"
      use_task_role: true
```

<Note>
  When using `use_task_role: true`, ensure your ECS task role, EC2 instance profile, or EKS service account has the required Kinesis permissions.
</Note>

## Message format

Sequin sends messages to Kinesis as base64-encoded JSON. You can find the shape of the messages in the [messages reference](/reference/messages).

## Retry behavior

If Sequin is unable to deliver a message to Kinesis, it will retry the message indefinitely. Sequin will exponentially back off the retry interval, with a maximum backoff of roughly 3 minutes.

## Max record size

Kinesis has a [maximum record size of 1 MB](https://docs.aws.amazon.com/streams/latest/dev/service-sizes-and-limits.html). If Kinesis rejects a message due to size limitations, Sequin will retry it indefinitely.

This means that the failing message will be saved until you take appropriate action and/or manually discard the message in the Sequin console.

For example, if one particular column caused the issue, you could define a [transform function](/reference/transforms) which removed that column from the message payload.

Then, after attaching that transform to your Kinesis sink, the next retry of the oversize message (automatic or manual) should succeed.

<Info>
  If you want to see logging/alerting for this situation, please [upvote the corresponding issue](https://github.com/sequinstream/sequin/issues/596).
</Info>

## Ordering

When you configure a Kinesis sink, you can specify message grouping behavior. The grouping value is used as the partition key for records put into Kinesis. Records with the same key are ordered within a shard.

The default group for a message is the source row's primary key(s). You can override this by specifying one or more columns to use for grouping.

<Note>
  Kinesis partition keys have a maximum size of 256 bytes. If the partition key exceeds this limit, Sequin automatically hashes it to ensure compatibility.
</Note>

Sequin will order the delivery of messages with the same group according to their commit timestamp.

## Routing

You can use [routing functions](/reference/routing) to dynamically route messages to different Kinesis streams based on the message content.

Your routing function should return a map with the `stream_arn` field:

```elixir theme={null}
def route(action, record, changes, metadata) do
  # Route to different streams based on record type
  stream_name = case record["type"] do
    "user" -> "user-events"
    "order" -> "order-events"
    _ -> "general-events"
  end

  %{
    stream_arn: "arn:aws:kinesis:us-east-1:123456789012:stream/#{stream_name}"
  }
end
```

This allows you to dynamically select which Kinesis stream receives each message based on your routing logic.

## Debugging

You can view the status of your Kinesis sink in the Sequin web console.

On the "Messages" tab, you can see which messages are in-flight to Kinesis, which messages Sequin is unable to deliver, and recently delivered messages.

Messages that Sequin is unable to deliver will have a "Deliver count" greater than `1`. You can click on a message to see more details, including the last error response received from Kinesis.