Installing Halyard

Halyard manages the lifecycle of your Spinnaker deployment, including writing & validating your deployment’s configuration, deploying each of Spinnaker’s microservices, and updating the deployment.

The official documentation recommends installing Spinnaker using halyard, if you require any support with your installation and you did not use halyard you will be on your own, this quote is from the Spinnaker official installation guide.

Though it’s possible to install Spinnaker without Halyard, we don’t recommend it, and if you get stuck we’re just going to tell you to use Halyard.

That said, lets get started by installing halyard.

I used the “Install on Debian/Ubuntu and macOS” installation, and installed it on my MacOS

Creating a Kubernetes cluster and service account

If you don’t already have a cluster, you can create a Kubernetes cluster on GKE using the official documentation

Follow the instructions shown in the official documentation to download credentials. We will use these credentials to create the service account below

Given that all pods on GKE share the same service account, granting Spinnaker on GKE permission also grants permission to all pods running alongside Spinnaker. For this reason, we will configuring a Kubernetes Service account for Spinnaker to authenticate as.

Create a file cluster-admin-role.yml and paste this content into it



kind: ClusterRoleBinding

metadata:

name: spinnaker-admin-delete-me-after

roleRef:

apiGroup: rbac.authorization.k8s.io

kind: ClusterRole

name: cluster-admin

subjects:

- kind: User

name: apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata:name: spinnaker-admin-delete-me-afterroleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: cluster-adminsubjects:- kind: Username: y our-gke-user-account@example.com

The your-gke-user-account@example.com can be found using the command

gcloud auth list --filter=status:ACTIVE --format="value(account)"

It should be the same email address as the one in the credentials file that was downloaded

Create the Role: kubectl apply -f cluster-admin-role.yml

Now that we have the correct permissions we can go ahead and create the Kubernetes service account for Spinnaker.

First, create a k8s-spinnaker-service-account.yml file and paste this contents into it, which can be found in here

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRole

metadata:

name: spinnaker-role

rules:

- apiGroups: [""]

resources: ["namespaces", "configmaps", "events", "replicationcontrollers", "serviceaccounts", "pods/logs"]

verbs: ["get", "list"]

- apiGroups: [""]

resources: ["pods", "services", "secrets"]

verbs: ["create", "delete", "deletecollection", "get", "list", "patch", "update", "watch"]

- apiGroups: ["autoscaling"]

resources: ["horizontalpodautoscalers"]

verbs: ["list", "get"]

- apiGroups: ["apps"]

resources: ["controllerrevisions", "statefulsets"]

verbs: ["list"]

- apiGroups: ["extensions", "apps"]

resources: ["deployments", "replicasets", "ingresses"]

verbs: ["create", "delete", "deletecollection", "get", "list", "patch", "update", "watch"]

# These permissions are necessary for halyard to operate. We use this role also to deploy Spinnaker itself.

- apiGroups: [""]

resources: ["services/proxy", "pods/portforward"]

verbs: ["create", "delete", "deletecollection", "get", "list", "patch", "update", "watch"]

---

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

name: spinnaker-role-binding

roleRef:

apiGroup: rbac.authorization.k8s.io

kind: ClusterRole

name: spinnaker-role

subjects:

- namespace: spinnaker

kind: ServiceAccount

name: spinnaker-service-account

---

apiVersion: v1

kind: ServiceAccount

metadata:

name: spinnaker-service-account

namespace: spinnaker

Now we can apply that to our cluster.

kubectl apply --context spinnaker -f k8s-spinnaker-service-account.yml

Grab the token that this service account relies on:

TOKEN=$(kubectl get secret --context spinnaker \

$(kubectl get serviceaccount spinnaker-service-account \

--context spinnaker \

-n spinnaker \

-o jsonpath='{.secrets[0].name}') \

-n spinnaker \

-o jsonpath='{.data.token}' | base64 --decode)

Place this token into your kubeconfig under a new user called

spinnaker-token-user

kubectl config set-credentials spinnaker-token-user --token $TOKEN

Configure your spinnaker context to use this new user

kubectl config set-context spinnaker --user spinnaker-token-user

Now your spinnaker context will authenticate with the token we created above.

Check out these docs to learn more about contexts and why they are useful.

With those steps out of the way we can jump back to the official guide and add an account.

Choosing an environment

Once you have added an account you need to choose an environment. If you plan on using this in production the recommended environment is a Distributed Installation

Configuring persistent storage

Spinnaker requires an external storage provider for persisting your Application settings and configured Pipelines. For my installation I used the Spinnaker GCS guide

With everything configured we are now ready to deploy Spinnaker

Deploying Spinnaker

Deploying Spinnaker is the easy part, simply follow these steps

That’s it. That should be all that is required to get Spinnaker setup on GKE.

Configure Spinnaker to be publicly reachable (with authentication)

To make it easier to access Spinnaker you want to be able to reach it publicly.

For this section of the guide let us assume we are using the following $DOMAIN cznconsulting.com

Part 0 and Part 1 of this guide are easy to follow, and no changes are required.

I was not able to get Part 2 working on GKE, the external IP on the spin-deck and spin-gate services were stuck in a pending state.

As a work around, and in my opinion a better solution for GKE, I setup an Ingress to route traffic to these services, but more on that later.

First navigate back to the Google credentials manager, and edit the Spinnaker client ID using your value of $DOMAIN :

Where in our example it would look like this: http://spinnaker-api.cznconsulting.com/login

Now authorize the UI and API servers to receive requests at these urls using Halyard:

hal config security ui edit \

--override-base-url http://spinnaker.cznconsulting.com



hal config security api edit \

--override-base-url http://spinnaker-api.cznconsulting.com

Now, before finalizing these changes by deploying Spinnaker, we need to edit the Kubernetes Services fronting the UI & API servers, spin-deck and spin-gate in the spinnaker namespace respectively.

Run the following command

kubectl edit svc spin-deck -n spinnaker

You will have the service definition open in your text editor. Make the changes noted below in bold

# Please edit the object below. Lines beginning with a '#' will be ignored,

# and an empty file will abort the edit. If an error occurs while saving this file will be

# reopened with the relevant failures.

#

apiVersion: v1

kind: Service

metadata:

creationTimestamp: 2017-06-01T00:57:34Z

name: spin-deck

namespace: spinnaker

resourceVersion: "6038615"

selfLink: /api/v1/namespaces/spinnaker/services/spin-deck

uid: 4c1fb82f-4165-11e7-888f-42020a8a0a12

spec:

clusterIP: 10.127.244.30

ports:

- port: 9000

protocol: TCP

targetPort: 9000

selector:

load-balancer-spin-deck: "true"

sessionAffinity: None

type: NodePort



status:

loadBalancer: {}

Now repeat this but for spin-gate

kubectl edit svc spin-gate -n spinnaker

You will have the service definition open in your text editor. Make the changes noted below in bold

# Please edit the object below. Lines beginning with a '#' will be ignored,

# and an empty file will abort the edit. If an error occurs while saving this file will be

# reopened with the relevant failures.

#

apiVersion: v1

kind: Service

metadata:

creationTimestamp: 2017-06-01T00:57:32Z

name: spin-gate

namespace: spinnaker

resourceVersion: "6038615"

selfLink: /api/v1/namespaces/spinnaker/services/spin-gate

uid: 1c4fd288-818a-166e-888f-45251eee0d92

spec:

clusterIP: 10.127.244.29

ports:

- port: 8084

protocol: TCP

targetPort: 8084

selector:

load-balancer-spin-gate: "true"

sessionAffinity: None

type: NodePort status:

loadBalancer: {}

The native GKE Ingress Controller only supports NodePort at the moment

Finally, redeploy Spinnaker

hal deploy apply

We are eventually going to update our DNS settings and create A records to point our subdomains at a static IP, so we need to create the static IP first.

gcloud compute addresses create spinnaker-static-ip --global

Now we are ready to create the Ingress resource. I recommend reading this excellent article explaining what an Ingress is.

First, create a file called spinnaker-ingress.yml and paste the following contents into it.

apiVersion: extensions/v1beta1

kind: Ingress

metadata:

name: spinnaker-ingress

annotations:

kubernetes.io/ingress.global-static-ip-name: "spinnaker-static-ip"

namespace: spinnaker

spec:

rules:

- host: spinnaker-api.cznconsulting.com

http:

paths:

- backend:

serviceName: spin-gate

servicePort: 8084

- host: spinnaker.cznconsulting.com

http:

paths:

- backend:

serviceName: spin-deck

servicePort: 9000

Note: The annotations section specifies the newly created static IP should be associated with this Ingress

Now we can apply this to our cluster, which will create a new load balancer for us.

kubectl apply -f spinnaker-ingress.yml

There is one final step we need to take to get this to work, that is because of an existing github issue with ingress-gce

We need to manually update the health check on our newly created load balancer.

First, identify the failing health check

kubectl describe ingress spinnaker-ingress -n spinnaker

That command will show something like this, except there will be one service that is UNHEALTHY

Once you have identified the UNHEALTHY service, head over to Google Cloud console and update the health check from / to /health by editing the health check that is failing. This is because spin-gate does not return a http status code of 20X on the / path of it’s service, but spin-deck does, so you only have to update/edit the one health check.

If you give that a minute or two and run the following command again you should see all backends now reporting as healthy.

kubectl describe ingress spinnaker-ingress -n spinnaker

There is an additional piece of information on that command that is useful for the final step, and that is the static IP that is reported in the Address block.

Use that IP to configure your DNS provider and create two A records for the sub-domains, spinnaker.yourdomainhere.com and spinnaker-api.yourdomainhere.com and point them to the static IP above.

If all went well you will have a functioning spinnaker installation that is publicly available with Google OAuth authentication.