Neon’s support for Postges’ logical replication features opens up a variety of interesting use cases for real-time streaming architectures based on change data capture. We previously demonstrated how to use Debezium to fan out changes from Postgres by using Redis as a message broker.
Today, we’ll explore how you can leverage the Apache Kafka and Kafka Connect ecosystem to capture and process changes from your Neon Postgres database. Specifically, you’ll learn how to stream changes from Postgres to Apache Kafka and process those changes using ksqlDB to create a materialized view that updates in response to database changes.