Kafka Connect moves data between Kafka and other systems with configuration instead of code. Connector type Examples Source (into Kafka) Debezium CDC (Postgres, MySQL, SQL Server), JDBC source, S3, MQTT Sink (out of Kafka) JDBC sink, Elasticsearch/OpenSearch, S3/GCS, Snowflake, BigQuery Key features Runs as a scalable worker cluster (distributed mode) managed over a REST API (:8083). Single Message Transforms (SMTs) for light changes: rename fields, route topics, mask data. Converters pick the format (JSON, Avro, Protobuf). Error handling: errors.tolerance=all + a dead letter queue topic for bad records. Debezium CDC is the standard way to turn database changes into events (and to implement the outbox pattern) without dual writes.
Key Takeaways
Connect integrates external systems by configuration, not custom code.
Debezium CDC streams database changes without dual writes.
Use SMTs for light transforms and Streams/Flink for real processing.
Route bad records to a dead letter queue instead of stopping the connector.