> ## Documentation Index
> Fetch the complete documentation index at: https://sequinstream.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# How to stream Postgres to Typesense
> Build real-time search indexes with Postgres change data capture (CDC) and Typesense. Learn to keep your search collections in sync with your database.
This guide shows you how to set up Postgres change data capture (CDC) and stream changes to Typesense using Sequin.
With Postgres data streaming to Typesense, you can build real-time search experiences, maintain up-to-date search indexes, and provide fast, typo-tolerant search for your users.
By the end of this how-to, you'll have a configured Typesense cluster that stays in sync with your database.
This is the how-to guide for streaming Postgres to Typesense. See the [quickstart](/quickstart/typesense) for a step-by-step walkthrough or the [reference](/reference/sinks/typesense) for details on all configuration options.
## Prerequisites
If you're self-hosting Sequin, you'll need:
1. [Sequin installed](/running-sequin)
2. [A database connected](/connect-postgres)
## Basic setup
### Prepare your Typesense cluster
Sequin converts Postgres changes and rows into JSON documents and sends them to your Typesense cluster.
You'll need to have a Typesense cluster running and accessible to Sequin. You can run Typesense locally for development or use a hosted instance.
If you're developing locally, you can connect Sequin to a local Typesense instance.
### Mapping Postgres tables to Typesense collections
From the [Typesense docs](https://typesense.org/docs/guide/organizing-collections.html):
> In general, we recommend that you create one collection per type of document / record you have.
Similarly, Sequin recommends that you sink one Postgres table to one Typesense collection using a [routing function](/reference/routing).
When significantly scaling Typesense, you may want to [shard collections](https://typesense.org/docs/guide/organizing-collections.html#sharding-collections). For instance, you may shard your `users` collection by region into `users_us`, `users_eu`, `users_asia`, etc.
## Create a Typesense sink
Navigate to the "Sinks" tab, click "Create Sink", and select "Typesense Sink".
### Configure the source
Under "Source", select the table(s) or schema(s) you want to stream data from.
Indicate whether you want to receive `insert`, `update`, and/or `delete` actions. We recommend selecting all three, which will ensure that your Typesense collection is kept in sync with your Postgres table.
You can for instance unselect `deletes` if you want to exclude deleted rows from your Typesense collection. This means that records deleted from Postgres will not be removed from your Typesense collection and will still be searchable.
You can also specify [filter functions](/reference/filters) to narrow down the documents you want to index. For example, if you only want to index `products` that are currently `in_stock`, you can add a filter on `in_stock = true`. This is also useful for sharding or multi-tenancy.
Under "Message grouping", we strongly recommend enabling message grouping to ensure that changes are processed in the correct order.
While your Typesense collection is processing a message, if a new change or version related to the same row is captured, Sequin will hold the message back until the first message is processed. This is almost always the desired behavior.
If your sink includes **a single table**, you can optionally enrich your documents with additional data from your database using an [enrichment function](/reference/enrichment). This can add helpful metadata and context to your documents to increase the relevance of your search results.
For example, if you want to enrich your `products` collection with the `category` field from your `categories` table, you can add a function enrichment that returns the `category` field for each document:
```sql theme={null}
SELECT
p.id, -- you must select all primary keys for Sequin to associate the enrichment with the message
pc.name as category_name,
pc.additional_info-- example of an enrichment
FROM
product p
JOIN
product_category pc on p.category_id = pc.id
WHERE
p.id = ANY($1)
```
Typesense collections expect documents to have a specific schema, including:
1. An `id` field, which must be a string.
2. The additional fields that are searchable.
You can map your Postgres table columns to Typesense fields using a [functional transform](/reference/transforms#function-transform).
We recommend mapping your table to a map literal, such as:
```elixir theme={null}
def transform(record) do
%{
id: record["id"],
name: record["name"],
description: record["description"],
price: record["price"],
in_stock: record["in_stock"]
}
end
```
This ensures that the schema of your Typesense collection is well defined.
If instead you want any changes in your underlying table schema to be immediately reflected in your Typesense collection, you can return the `record` directly, perhaps only modifying the `id` field to ensure it exists and is a string.
```elixir theme={null}
# Return the record directly
def transform(record) do
record
end
```
```elixir theme={null}
# Set the id field to the product_id field
def transform(record) do
Map.put(record, "id", record["product_id"])
end
```
You can optionally indicate if you want to seed your Typesense collection with a [backfill](/reference/backfills) of all or a portion of the table's existing data.
Backfills are useful if you want Sequin to populate your Typesense collection with existing data from your Postgres table. If you choose not to backfill on creation, you can backfill at any time.
### Configure delivery
Under "Typesense configuration", you'll enter the URL for your Typesense cluster and provide an API key.
Additionally, under "Advanced Typesense settings", you can specify a request timeout (in seconds) for Typesense requests. You may want to increase the timeout if you have a slow network connection or are indexing a large number of documents per batch.
In the "Routing" section, you can specify how Sequin should route documents to your Typesense collection.
If you are sinking **a single table**, you don't need to apply any routing and simply enter the collection Sequin should deliver documents to.
If you are sinking **multiple tables**, you can use a [routing function](http://localhost:3000/reference/routing#typesense-sink) to route documents to the correct collection.
Additionally, you can customize actions in Typesense routing functions. For example, you can specify that a soft delete in Postgres should be reflected as a hard delete in Typesense.
In the "Sink settings" section, you can specify advanced configurations like batch size and timestamp format.
In most cases, you can leave the default values - which for Typesense is optimized to the default batch size of [40 documents](https://typesense.org/docs/27.0/api/documents.html#index-multiple-documents). However, if you've configured your Typesense cluster to process more documents per batch, you can increase the batch size here.
Give your sink a name, then click "Create Typesense Sink".
## Verify & debug
To verify that your Typesense sink is working:
1. Make some changes in your source table.
2. Verify that the count of messages for your sink increases in the Sequin web console.
3. Verify receipt of documents in your Typesense collection by searching for them.
If documents don't seem to be flowing:
1. Click the "Messages" tab to view the state of messages for your sink.
2. Click any failed message.
3. Check the delivery logs for error details, including the response from Typesense.
## Next steps
Assuming you've followed the steps above for your local environment, "[How to deploy to production](/how-to/deploy-to-production)" will show you how to deploy your implementation to production.