Operations | Monitoring | ITSM | DevOps | Cloud

TL;DR Aiven contributed GCP support and private networking to the open source OpenSearch Migration Assistant, previously an AWS-only tool. Delivered via five Terraform-based PRs, the changes add a GKE deployment path and let regulated industries migrate without using the public internet - giving Elasticsearch, OpenSearch, and Solr users on GCP a supported route to Aiven for OpenSearch.

If you’re using Apache Kafka for data streaming, then you’re likely to want to take data from one topic and write an amended version of that data to another topic, for instance: for filtering (give me the items that have been delivered), enrichment (add some more information to messages, from another data source) or anomaly detection (show me potential signs of trouble). The premier solution in this space is the Apache Kafka Streams library.

Let me start with something most Kafka contributors think but rarely say out loud: nobody understands all of Kafka. I'm not a core committer and have only contributed a few times, but those contributions I have made have been in part thanks to using coding assistants. There are some issues with this approach though, the Apache Kafka project is huge. It's split into many parts: the core, the server, the client libraries, the streams engine, the storage layer, the consensus code, and more.