Debezium

Revision	Date	Description
`1.0`	24.07.2024	Init Changelog

Introduction

Debezium is an open source distributed platform for change data capture. Start it up, point it at your databases, and your apps can start responding to all of the inserts, updates, and deletes that other apps commit to your databases. Debezium is durable and fast, so your apps can respond quickly and never miss an event, even when things go wrong.

Kubernetes Deployment

deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: debezium
    cluster: debezium
    super: metric
    batch: no
    online: yes
  name: debezium
spec:
  replicas: 0
  selector:
    matchLabels:
      app: debezium
      cluster: debezium
  template:
    metadata:
      labels:
        app: debezium
        cluster: debezium
    spec:
      automountServiceAccountToken: true
      containers:
        - env:
            - name: MY_POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: MY_POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: MY_POD_IP
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: status.podIP
            - name: MY_CLUSTER_LABEL
              value: cluster=debezium
          image: debezium
          imagePullPolicy: IfNotPresent
          name: debezium
          ports:
            - containerPort: 8083
              name: http
              protocol: TCP
          resources:
            limits:
              cpu: 3000m
              memory: 3072Mi
            requests:
              cpu: 1500m
              memory: 1536Mi
      imagePullSecrets:
        - name: helm

service.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: debezium
spec:
  ports:
    - name: http
      port: 8083
      protocol: TCP
      targetPort: 8083
  selector:
    app: debezium

ingress.yaml

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: debezium
  labels:
    app: debezium
spec:
  rules:
    - host: debezium.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: debezium
                port:
                  number: 8083

Configuration

Example config

{
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"tasks.max": "1",
"database.hostname": "paybm2-test.postgresql.dc0",
"database.port": "5432",
"database.user": "paybm_debezium",
"database.password": "dupa.8",
"database.dbname" : "paybm",
"database.server.name": "Debezium",
"provide.transaction.metadata": "true",
"transaction.topic":"Debezium-Transactions",
"transforms": "route",
"transforms.route.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.route.regex": "([^.]+)\\.([^.]+)\\.([^.]+)",
"transforms.route.replacement": "$1-DataSources",
"plugin.name": "wal2json",
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
"key.converter.schemas.enable": "false",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "false",
"table.include.list": "public.payway,public.paywayconf"
}

Applying configuration

To apply configuration into Debezium, you need to send POST request on Debezium Service URL:

http://debezium.example.com/connectors/paybm-connector/config

Restart Debezium

Run kubectl command below to restart Debezium Service:

kubectl rollout restart deployment/NAME

Remember to change NAME with name of Debezium Deployment.

Last modified: 17 February 2025