I did say ‘ part deux' was coming in my previous Blog Post Istio-Proxy Ingress SSL Certificate Integration the old fashioned* way(no SDS) — so here it is.

At first, you work with Containers and this is it! then you discover the new world of Kubernetes, and this becomes it again. Now add the ‘steroids’ of Istio Service Mesh, and we’re integrating the whole system end-to-end with Google Cloud, trying to piece together how we can make do with The Best of both worlds. Easy, right?

Well, spoiler alert, It works a treat. Grab a coffee, this will be interesting, I promise.

HTTP(S) LoadBalancer GKE Ingress + Istio (NEG) + SSL Manager Integration

Since this is quite a convoluted integration spanning multiple features and products, let’s include a few relevant keywords for easier discovery, as we’re touching on;

- HTTP(S) Loadbalancer

- GKE GCE Ingress Controller

- BackendConfig for GKE(GCE) Ingress Controller

- CloudArmor + WAF protection for Istio (NEG)

- Istio Ingressgatway integration as an HTTP(S) Backend

- Istio Ingressgateway + VirtualService; health check and application traffic routing

What we are set out to achieve is to Configure Istio Ingressgateway (managed or not) on GKE Cluster, as a backend (NEG) for Google HTTP(S) load balancer.

By doing so, we are able to benefit from BackendConfig configuration features, one of which allows us to enable ClourArmor Google security add-on product, which offers Enterprise DDoS defense, at the Edge.

This full integration will enable such a setup to have:

Full SSL termination at GCE Loadbalancer (not Istio Ingress Gateway)

Anycast IP

Full Istio Mesh features integration, packet routing, traffic control, release management

Cloud Armor support, DDoS protection + WAF features

Cloud Identity Aware Proxy

And more, see Backendconfig again

This requires provisioning HTTP(s) load balancer and configuring Istio to accept traffic via GKE (gce) Ingress, with the associated BackendConfig

Besides CloudArmor feature, there are other additional benefits as you can use a BackendConfig to configure these features of HTTP(S) Load Balancing:

- Cloud CDN

- Identity-Aware Proxy (IAP)

- Timeout, Connection draining timeout, Session affinity

This is different from the default Istio Service mesh model implementation, where the latter is provisioned with service type instead of a traditional ingress .

An effective double-filtered, double-secured setup, which works in each independent layer.

Once you factor the Istio Service Mesh, the Ingress type is then no longer a BAU option with Istio. The Istio Ingress Gateway Service is actually a service with the LoadBalancer type.

This can and does work BAU, alongside the classic HorizontalPodAutoscaler However, this results in non-compatible service-provisioning within Google Console, whereby additional offerings like Cloud Armor/WAF, IAP are not possible.

The Action List to achieve the How-To Objectives with 5 “easy” steps:

Create a GKE cluster with Cloud Run and Istio enabled

Create an Istio virtual service to handle health check requests

Modify the Kubernetes Service object of the Istio ingress gateway so it can be exposed by a Kubernetes Ingress object

Configure HTTP(S) Load Balancing by creating a Kubernetes Ingress object

Deploy a sample Nginx service to verify the solution.

Kudos where it’s due, — the inspiration of this implementation was taken from the Google documentation on HTTP(S) load balancer implementation with Istio and CloudRun

Diving through each step in their glorious details

Provision your GKE Cluster with HttpLoadBalancing And Istio add-on enabled

And enabled Deploy the DNS-Manager — like in previous posts, to do away with DNS management toil. And also because we can.

SSL Manager — HOW-TO guide deploying cert-manager, and use ACME DNS verification for Let’s Encrypt CA to issue certificates

Configure the GKE Ingress to work with Istio ingressgateway as NEG backend + BackendConfig, proxying through to Istio Ingress Gateway PODs, NEGs Healtchecks. Then, another VirtualService mapping to reach our demo Nginx App:

Reserve Static IP — as should be relevant

Configure Cloud Armor Policy — any ‘block my IP’ trial would do

The reference above within `BackendConfig`

Patch Istio Ingressgateway service, to be ingress friendly with this configuration update. Its worth to point out that service will become of NodePort type from loadbalancer the type it gets provisioned.

Copy-Paste JSON

[ { "op": "add", "path": "/metadata/annotations/cloud.google.com~1neg", "value": "{\"ingress\": true}" }, { "op": "replace", "path": "/spec/type", "value": "NodePort" }, { "op": "remove", "path": "/status" }, { "op": "add", "path": "/metadata/annotations/beta.cloud.google.com~1backend-config", "value": "{\"ports\": {\"80\":\"cloudarmor-staging-test\"}}" }]

Configure the VirtualService to tweak and accept LB health check calls to NEG (Istio Ingressgateway PODs). Without this HTTP(S) loadbalancer would never be healthy, and thus no traffic forwarded to the backend — Istio ingressgateway as our NEGs

A must for HTTP(S) load balancer to mark NEG as Ready — which is the Istio Ingressgateway Pod

Configure (GCE) Ingress with TLS secret, if relevant. You can reference it below since cert-manager and external-dns should be operating on your cluster reserving DNS entries and issuing certificates as and when necessary.

default backend to be as istio-ingressgateway.

Deploy the Nginx demo service to test against

kubectl run nginx --image=nginx && k expose deploy/nginx --name nginx --type ClusterIP --port 80 --target-port 80

Create a separate VirtualService to route domain request to the dedicated Nginx demo service.

Another VirtualService routing configuration to redirect traffic based on the host to the dummy nginx service-> deployment I have provisioned

3. Test Cloud Armor policy with blacklisting your own IP address!

Hope you enjoyed this. Enjoying the read?

Like, Clap and Share this post along!

Join the conversation. See you in #Kubernetes-users Kubernetes slack group

PS There is quite a number of exciting Kubernetes projects taking place at Contino. If you are looking to work on the latest-greatest infrastructure stack or looking for a challenge, — Get in touch! We’re hiring, looking for bright minds at every level. At Contino, we pride ourselves on delivering the best practices cloud transformation projects, for medium-sized businesses to large enterprises.