NOTE: In mid-2019 we re-launched and intensified our efforts, deprecating and archiving the old code base of the AWS Service Operator and changing to a community-driven approach. We’re currently in the design phase and invite you to comment on the design issues and become a contributor to the new project, see details at the new GitHub repo aws/aws-service-operator-k8s. (update by Michael Hausenblas, 02/2020).

The AWS Service Operator is an open source project in developer preview which allows to you manage your AWS resources directly from Kubernetes using the standard Kubernetes CLI, kubectl . It does so by modeling AWS Services as Custom Resource Definitions (CRDs) in Kubernetes and applying those definitions to your cluster. This means that a developer can model their entire application architecture from container to ingress to AWS services, backing it from a single YAML manifest. We anticipate that the AWS Service Operator will help reduce the time it takes to create new applications, and assist in keeping applications in the desired state.

Have you ever tried to integrate Amazon DynamoDB with an application running in Kubernetes? How about deploying an S3 Bucket for your application to use? If you have, you will know this usually requires you to use some tool such as AWS CloudFormation or Hashicorp Terraform. Then you’ll need to create a way to deploy those resources, either as one-off components or in a pipeline using the appropriate toolkit. This causes disjointed experiences between Kubernetes and AWS, requiring you as an operator to manage and maintain this lifecycle. What if instead you could do this all using Kubernetes’ built-in control loop, storing a desired state within the API server for both the Kubernetes components and the AWS services needed? This is where the AWS Service Operator comes into play.

What is an Operator?

Kubernetes is built on top of what is called the controller pattern. This pattern allows applications and tools to listen to a central state manager (etcd), and take action when something happens. Examples of such applications include cloud-controller-manager , controller-manager , etc. The controller pattern allows us to create decoupled experiences and not have to worry about how other components are integrated. An operator is a purpose-built application that will manage a specific type of component using this same pattern. To learn about other operators, check out Awesome Operators.

AWS Service Operator

The AWS Service Operator as of today exposes a way to manage DynamoDB Tables, S3 Buckets, Amazon Elastic Container Registry (Amazon ECR) Repositories, SNS Topics, SQS Queues, and SNS Subscriptions, with many more integrations coming soon.

Prerequisites

Kubernetes Cluster – try using Amazon EKS or kops to set one up kubectl awscli

Getting Started

Before we can get the AWS Service Operator deployed, we need to set up a way to manage AWS IAM credentials to Kubernetes pods. For demo purposes, we’re going to grant those admin permissions to our Kubernetes worker nodes.

If you are using an Amazon Elastic Container Service for Kubernetes (EKS) cluster, you most likely provisioned this with CloudFormation. The command shown below will use the CloudFormation stacks to try to update the proper Instance Policy with the correct permissions.

aws iam attach-role-policy \ --policy-arn arn:aws:iam::aws:policy/AdministratorAccess \ --role-name $(aws cloudformation describe-stacks --stack-name ${STACK_NAME} | jq -r ".Stacks[0].Outputs[0].OutputValue" | sed -e 's/.*\///g')

Make sure to replace ${STACK_NAME} with the nodegroup stack name from the CloudFormation console.

You can now download the latest aws-service-operator.yaml Kubernetes manifest file and change a couple of values.

wget https://raw.githubusercontent.com/awslabs/aws-service-operator/master/configs/aws-service-operator.yaml

With the file downloaded locally, you need to make a few edits to include your environment information. On line #96-98 you will see <CLUSTER_NAME> , <REGION> , and <ACCOUNT_ID> . Change these values to values appropriate for your environment.

- --cluster-name=aws-service-operator-demos - --region=us-west-2 - --account-id=000000000000

You can get your Account ID by using the aws sts get-caller-identity AWS CLI call. In the output you should see your Account ID.

After you have updated this file, apply it to your cluster.

$ kubectl apply -f aws-service-operator.yaml namespace/aws-service-operator created clusterrole.rbac.authorization.k8s.io/aws-service-operator created serviceaccount/aws-service-operator created clusterrolebinding.rbac.authorization.k8s.io/aws-service-operator created deployment.apps/aws-service-operator created

This creates a new namespace aws-service-operator and deploys a Cluster Role, Service Account, and Role Binding, along with the deployment which manages the AWS resources.

Now you can check to see if the operator was deployed correctly by running

$ kubectl get customresourcedefinitions NAME CREATED AT cloudformationtemplates.operator.aws 2018-09-27T21:30:10Z dynamodbs.operator.aws 2018-09-27T21:30:10Z ecrrepositories.operator.aws 2018-09-27T21:30:10Z s3buckets.operator.aws 2018-09-27T21:30:10Z snssubscriptions.operator.aws 2018-09-27T21:30:10Z snstopics.operator.aws 2018-09-27T21:30:10Z sqsqueues.operator.aws 2018-09-27T21:30:10Z

If you see all or more of these CRDs applied to your cluster, you are ready to start using those resources.

Deploy AWS Resources

For this example, we’re going to use the AWS Service Operator to create a DynamoDB table and deploy an application that uses the table after it’s been created.

Here’s an example of an Amazon DynamoDB Table manifest file:

apiVersion: service-operator.aws/v1alpha1 kind: DynamoDB metadata: name: dynamo-table spec: hashAttribute: name: name type: S rangeAttribute: name: created_at type: S readCapacityUnits: 5 writeCapacityUnits: 5

As you can see, it’s using the service-operator.aws/v1alpha1 API Version, which is our custom API Version for the AWS Service Operator.

The entire manifest we’re going to deploy will include a Deployment, Service, and DynamoDB table. Now let’s apply this manifest to our cluster to see everything work.

kubectl apply -f https://gist.githubusercontent.com/christopherhein/1cb7da812a197ac3cc547ed2495faf9d/raw/4909e8ad6b35843e1696735e9f62301e9bde7ff9/dynamodb-app.yaml

This will apply this manifest to your cluster and create a new DynamoDB table that your applications can use. Next, list your DynamoDB tables that are managed by the AWS Service Operator. This command uses the -w (watch) flag, which will update inline when there are changes.

$ kubectl get dynamodb -o yaml -w additionalResources: configMaps: null apiVersion: service-operator.aws/v1alpha1 kind: DynamoDB metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"service-operator.aws/v1alpha1","kind":"DynamoDB","metadata":{"annotations":{},"name":"dynamo-table","namespace":"default"},"spec":{"hashAttribute":{"name":"name","type":"S"},"rangeAttribute":{"name":"created_at","type":"S"},"readCapacityUnits":5,"writeCapacityUnits":5}} clusterName: "" creationTimestamp: 2018-10-04T21:39:39Z generation: 1 name: dynamo-table namespace: default resourceVersion: "145514" selfLink: /apis/service-operator.aws/v1alpha1/namespaces/default/dynamodbs/dynamo-table uid: ff66c9a0-c81d-11e8-ab31-02d2702eca80 output: tableARN: "" tableName: "" spec: cloudFormationTemplateName: "" cloudFormationTemplateNamespace: "" hashAttribute: name: name type: S rangeAttribute: name: created_at type: S readCapacityUnits: 5 rollbackCount: 0 writeCapacityUnits: 5 status: resourceStatus: CREATE_IN_PROGRESS resourceStatusReason: "" stackID: arn:aws:cloudformation:us-west-2:XXXXXXXXXXXX:stack/aws-service-operator-dynamodb-dynamo-table-default/bf4e7670-c81c-11e8-9ca9-0a473bf201de

In this output you will see the status key which represents the status of the resource being created, as well as an output key which will update when the resource is provisioned. The final output of this command should be:

additionalResources: configMaps: - dynamo-table apiVersion: service-operator.aws/v1alpha1 kind: DynamoDB metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"service-operator.aws/v1alpha1","kind":"DynamoDB","metadata":{"annotations":{},"name":"dynamo-table","namespace":"default"},"spec":{"hashAttribute":{"name":"name","type":"S"},"rangeAttribute":{"name":"created_at","type":"S"},"readCapacityUnits":5,"writeCapacityUnits":5}} clusterName: "" creationTimestamp: 2018-10-04T21:39:39Z generation: 1 name: dynamo-table namespace: default resourceVersion: "145600" selfLink: /apis/service-operator.aws/v1alpha1/namespaces/default/dynamodbs/dynamo-table uid: ff66c9a0-c81d-11e8-ab31-02d2702eca80 output: tableARN: arn:aws:dynamodb:us-west-2:XXXXXXXXXXXX:table/dynamo-table tableName: dynamo-table spec: cloudFormationTemplateName: "" cloudFormationTemplateNamespace: "" hashAttribute: name: name type: S rangeAttribute: name: created_at type: S readCapacityUnits: 5 rollbackCount: 0 writeCapacityUnits: 5 status: resourceStatus: CREATE_COMPLETE resourceStatusReason: "" stackID: arn:aws:cloudformation:us-west-2:XXXXXXXXXXXX:stack/aws-service-operator-dynamodb-dynamo-table-default/bf4e7670-c81c-11e8-9ca9-0a473bf201de

We can now get the Service’s load balancer endpoint by using kubectl get service -o wide which will allow us to load the application seeing the data that persists in DynamoDB.

$ kubectl get service -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR frontend LoadBalancer 10.100.53.114 aff6aa155c81d11e8ab3102d2702eca8-294585320.us-west-2.elb.amazonaws.com 80:32200/TCP 2m app=frontend

Get Involved

Learn more about the AWS Service Operator project on GitHub, and get involved! Pull requests and issues are welcome. We’d love to hear what services you think should be implemented to make them easier to consume from within your Kubernetes cluster. Please reach out and let us know what you think.

Read more from Chris.