In a growing world it’s become the norm for teams to be able to own their services, in their entirety. Everything from initial planning to development and finally deployment. Gone are the days of throwing the code over the wall to an Ops team you never met or only had arguments with, welcome to the future.

At Turnitin we’re beginning to strongly leverage the power of Kubernetes for our deployments. We have a large breadth of teams working on anything from integrations with LMSs (Moodle, Blackboard, Canvas, etc.), to the powerful services we use for originality matching, all of this has to be housed somewhere and housed in such a fashion that the teams responsible for the services are able to manage their deployments.

Enter namespaces

We’re going to play with minikube in order to leverage the power of namespaces, let’s take a look at what’s provided by default.

➜ ~ kubectl get namespaces

NAME STATUS AGE

default Active 69d

kube-public Active 69d

kube-system Active 69d

This is great, but our teams are going to need a namespace of their own, we can create namespaces using kubectl ! Check out this gist for a description of team-a’s namespace. Let’s use that to set up an area for the team.

It contains a little bit of information regarding the namespace we’re going to create, most importantly the name!

➜ ~ kubectl create -f team-a-namespace.json

namespace “team-a” created ➜ ~ kubectl get namespaces

NAME STATUS AGE

default Active 69d

kube-public Active 69d

kube-system Active 69d

team-a Active 9s

There’s a variety of super great policies that can be defined when creating namespaces, one that you might be interested to look at involves resource quotas.

Handling deployments

A namespace for the team is no good if they can’t put anything there. We don’t want to give every team super access to the cluster so we should create a service account that has permissions to manage their namespace for them.

Kubernetes has a variety of authorization mechanisms but the one we’re going to be looking at is RBAC, specifically Role's and RoleBinding’s.

Creating the ServiceAccount

Creating a service account is a simple process, in order to do that we need to define the name of the account, and also define the namespace in which it will live, the gist is here, but it’s small enough that we can take a look.

apiVersion: v1

kind: ServiceAccount

metadata:

name: team-a-service-account

namespace: team-a

Super easy, let’s have kubectl create that for us.

➜ ~ kubectl create -f team-a-service-account.yaml

serviceaccount “team-a-service-account” created

Creating the Role

We have a service account, but it can’t do anything yet, we need to give it a role. The documentation is super great, so I suggest you take a read, for the sake of this team a service account that permissions create and update deployments and services will fit the teams needs.

The definition for such a role is available on gist here and looks like this

kind: Role

apiVersion: rbac.authorization.k8s.io/v1beta1

metadata:

namespace: team-a

name: team-a-deploy-role

rules:

- apiGroups: [“extensions”, “apps”]

resources: [“deployments”]

verbs: [“get”, “create”, “update”, “patch”]

- apiGroups: [“”]

resources: [“services”]

verbs: [“get”, “create”, “update”, “patch”]

Again not much to it, we have defined a couple of explicit rules for the role that perfectly fit what our team might need. We could add more as desired or require, but this fits our current teams needs, importantly we’ve also called out the namespace in which this role is applicable. Let’s apply it with kubectl

➜ ~ kubectl create -f team-a-deploy-role.yaml

role “team-a-deploy-role” created

Creating the role RoleBinding

We’re almost there, now we just need to bind the role to the service account we created, again the documentation is a great resource if you’re struggling to understand any of the concepts. Let’s take a look.

kind: RoleBinding

apiVersion: rbac.authorization.k8s.io/v1beta1

metadata:

name: team-a-binding

namespace: team-a

subjects:

- kind: ServiceAccount

name: team-a-service-account

namespace: team-a

roleRef:

kind: Role

name: team-a-deploy-role

apiGroup: rbac.authorization.k8s.io

So what’s happening here? A new RoleBinding is being created with the name team-a-binding, we’re binding it to subject team-a-service-account and referencing the role we created team-a-deploy-role, all of this is happening inside of the team-a namespace. Let’s have kubectl run this for us.

➜ ~ kubectl create -f team-a-role-binding.yaml

rolebinding “team-a-binding” created

Share the credentials

Great, so we have a a service account that is capable of doing the deployments the team requested but now we actually need to share the credentials with team-a, how do we do that?

After we created the service account Kubernetes created a bunch of stuff for us, including a secret that contains the accounts CA and token. Let’s have a look at the resource.

➜ ~ kubectl get sa team-a-service-account — namespace team-a -o json

{

“apiVersion”: “v1”,

“kind”: “ServiceAccount”,

“metadata”: {

“creationTimestamp”: “2018–05–20T15:59:38Z”,

“name”: “team-a-service-account”,

“namespace”: “team-a”,

“resourceVersion”: “18409”,

“selfLink”: “/api/v1/namespaces/team-a/serviceaccounts/team-a-service-account”,

“uid”: “cc52ba75–5c46–11e8–9562–080027247075”

},

“secrets”: [

{

“name”: “team-a-service-account-token-lf4k7”

}

]

}

We can see a secret resource has been created for us named team-a-service-account-token-lf4k7, we’re going to use that to grab some information that we’re going to share with the team.

Some of the next steps are going to include a great tool called jq , if you haven’t played with it before I’d advise installing it and taking a look at the documentation, it’s a great tool for working with JSON!

Grabbing the Certificate Authority

The nature of how we’ve created the service account means that we authenticate against the cluster with a CA and a token, we can grab the CA and throw it into a certificate file using kubectl !

➜ ~ kubectl get secret team-a-service-account-token-lf4k7 — namespace team-a -o json | jq -r ‘.data[“ca.crt”]’ > team-a.crt ➜ ~ cat team-a.crt

ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJ….

Grabbing the User Token

In addition to the CA, we need the bearer token that’s used for authentication against the Kubernetes API, we can grab it in a very similar fashion.

➜ ~ kubectl get secret team-a-service-account-token-lf4k7 — namespace team-a -o json | jq -r ‘.data[“token”]’ > team-a.token ➜ ~ cat team-a.token

ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJ….

Talking with Kubernetes

As the Kubernetes administrator our job is now done, we can pass the certificate and token on to the team that requires them and wipe our hands with it, but as the team, how do we now talk to the cluster?

Well first thing first, the certificate we grabbed using kubectl is base64 encoded, we need to decide that before we move on.

➜ ~ base64 — decode -i team-a.crt > team-a-decoded.crt

Great, now let’s use kubectl to setup our local configuration so that we can talk to the cluster.



Cluster “minikube” set. ➜ ~ kubectl config set-cluster minikube — embed-certs=true — server= https://192.168.99.100:8443 — certificate-authority=team-a-decoded.crtCluster “minikube” set. ➜ ~ kubectl config set-credentials team-a-service-account — token=ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJ….

User “team-a-service-account” set. ➜ ~ kubectl config set-context team-a-context — cluster=minikube — user=team-a-service-account — namespace=team-a

Context “team-a-context” created. ➜ ~ kubectl config use-context team-a-context

Switched to context “team-a-context”.

A few details, the server https://192.168.99.100:8443 is the address of our Minikube cluster, this may be different for you and would definitely be different in a real life scenario, your Kubernetes administrator should be able to provide this to you.

If you are the cluster administrator and you’re unsure of where the master is running, you can find it easily with cluster-info



Kubernetes master is running at ➜ kubectl cluster-infoKubernetes master is running at https://192.168.99.100:8443

That’s all folks

There’s not too much more too it. In a real world scenario you might have a couple of scripts to automate some of this setup for you, and the rules you grant to your roles might be a little tighter here and there, but this should help you hit the ground running!

Read this article and more at my blog, https://fable.sh!