Why you would want to use Amazon MSK

Amazon MSK is one of the best way to deploy Apache Kafka in your AWS VPC securely and quickly. The main advantages that you will get are

Managed service: you don’t have to bring together an entire engineering team together to setup Apache Kafka. You can start building your applications in less than 15 minutes

you don’t have to bring together an entire engineering team together to setup Apache Kafka. You can start building your applications in less than 15 minutes Network security: Apache Kafka on Amazon MSK is deployed within your VPC, meaning that Apache Kafka network packets never goes out on the internet. This is a big difference from public managed solutions such as Confluent Cloud.

Apache Kafka on Amazon MSK is deployed within your VPC, meaning that Apache Kafka network packets never goes out on the internet. This is a big difference from public managed solutions such as Confluent Cloud. Kafka security: MSK supports SSL based security and SASL/SCRAM. I’ve setup Kafka security before, and I can tell you it’s error prone and hard. You can directly use a secure Kafka cluster on MSK.

you do not pay for Apache Kafka replication traffic going across your AZ

Cost savings: one HUGE advantage of using Amazon MSK is that you do not pay for Kafka replication traffic going across your AZ. If you are going to run Apache Kafka on EC2 machines yourself, and set a replication factor of 3, the network bill can become pretty significant if you have huge data volumes. There’s a handy spreadsheet here to compute your potential cost savings when using MSK

one HUGE advantage of using Amazon MSK is that If you are going to run Apache Kafka on EC2 machines yourself, and set a replication factor of 3, the network bill can become pretty significant if you have huge data volumes. There’s a handy spreadsheet here to compute your potential cost savings when using MSK Managed upgrades: one simple API to upgrade your Kafka cluster with no downtime.

The rest is a full Apache Kafka experience. You can still customize settings if you’re an advanced user, it is using the standard Apache Kafka distribution and therefore all your Kafka Streams, Kafka Connect, or any Kafka applications will still work.

Amazon did create some nice AWS service integrations with MSK: