Apart from all the recent discussions about Kubernetes and whether you should Dockerize your database or not, today I would like to show you why those two things might be good solutions when scalability and elasticity is a big requirement in your architecture.

The secret sauce here is simple: deploying both your application and database on Kubernetes and a combination of NoSQL and Spring Data.

Why NoSQL and Spring Data?

With document databases, you can avoid lots of unnecessary joins as the whole structure is stored in a single document. Therefore, it will naturally perform faster than a relational model as your data grows.

If you are using any of the JVM languages, Spring Data might be something quite familiar to you. Thus, you can quickly start with NoSQL even without any previous knowledge.

Why Kubernetes?

Kubernetes allows you to scale up and down your stateless application in a cloud-agnostic environment. In the last few versions, K8s also added the ability to run stateful applications such as databases in it, that is one of the (many) reasons why it is such a hot topic nowadays.

I have shown in my previous blog post how to deploy Couchbase on K8s and how to make it “elastic” by easily scaling up and down. If you haven’t read it yet, please spend a few extra minutes going through the video transcript as it is an important part of what we are going to talk about here.

Creating a User Profile Microservice

In the majority of the systems, the user (and all related entities) is the most frequently accessed data. Consequently, it is one of the first parts of the system that has to go through some sort of optimization as your data grows.

Adding a Cache layer is the first type of optimization we can think of. However, it is not the “final solution” yet. Things might get a little bit more complicated if you have thousands of users, or if you need to store user-related entities also in memory.

Managing massive amounts of user profiles is a well-known good fit for document databases. Just take a look at the Pokémon Go use case, for instance. Therefore, building a highly scalable and elastic User Profile Service seems to be a challenge good enough to demonstrate how to design a highly scalable microservice.

What you are going to need:

Couchbase

JDK and Lombok’s plugin for Eclipse or Intellij

Maven

A Kubernetes cluster – I’m running this example on 3 nodes on AWS (I do not recommend using minikube). If you don’t know how to set up one, watch this video.

The Code

You can clone the whole project here:

https://github.com/couchbaselabs/kubernetes-starter-kit 1 https : //github.com/couchbaselabs/kubernetes-starter-kit

Let’s start by creating our main entity called User:

@Document @Data @AllArgsConstructor @NoArgsConstructor @EqualsAndHashCode public class User extends BasicEntity { @NotNull @Id private String id; @NotNull @Field private String name; @Field private Address address; @Field private List<Preference> preferences = new ArrayList<>(); @Field private List<String> securityRoles = new ArrayList<>(); } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 @Document @Data @AllArgsConstructor @NoArgsConstructor @EqualsAndHashCode public class User extends BasicEntity { @NotNull @Id private String id ; @NotNull @Field private String name ; @Field private Address address ; @Field private List <Preference> preferences = new ArrayList < > ( ) ; @Field private List <String> securityRoles = new ArrayList < > ( ) ; }

In this entity we have two important properties:

securityRoles: All roles the user can play within the system.

All roles the user can play within the system. preferences: All possible preferences the user might have, such as language, notifications, currency, etc.

Now, let’s play a little bit with our Repository. As we are using Spring Data, you can use pretty much all its capabilities here:

@N1qlPrimaryIndexed @ViewIndexed(designDoc = "user") public interface UserRepository extends CouchbasePagingAndSortingRepository<User, String> { List<User> findByName(String name); } 1 2 3 4 5 @ N1qlPrimaryIndexed @ ViewIndexed ( designDoc = "user" ) public interface UserRepository extends CouchbasePagingAndSortingRepository < User , String > { List < User > findByName ( String name ) ; }

If you want to know more about Couchbase and Spring Data, check out this tutorial.

We also implemented two other methods:

@Query("#{#n1ql.selectEntity} where #{#n1ql.filter} and ANY preference IN " + " preferences SATISFIES preference.name = $1 END") List<User> findUsersByPreferenceName(String name); @Query("#{#n1ql.selectEntity} where #{#n1ql.filter} and meta().id = $1 and ARRAY_CONTAINS(securityRoles, $2)") User hasRole(String userId, String role); 1 2 3 4 5 6 @ Query ( "#{#n1ql.selectEntity} where #{#n1ql.filter} and ANY preference IN " + " preferences SATISFIES preference.name = $1 END" ) List < User > findUsersByPreferenceName ( String name ) ; @ Query ( "#{#n1ql.selectEntity} where #{#n1ql.filter} and meta().id = $1 and ARRAY_CONTAINS(securityRoles, $2)" ) User hasRole ( String userId , String role ) ;

hasRole: Check if a user has a specified role:

Check if a user has a specified role: findUsersByPreferencyName: As the name says, it finds all users that contain a given preference.

Notice that we are using N1QL syntax in the code above as it makes things much simpler to query than using plain JQL.

Additionally, you can run all tests to make sure that everything is working properly:

Don’t forget to change your application.properties with the correct credentials of your database:

spring.couchbase.bootstrap-hosts=localhost spring.couchbase.bucket.name=test spring.couchbase.bucket.password=couchbase spring.data.couchbase.auto-index=true 1 2 3 4 spring . couchbase . bootstrap - hosts = localhost spring . couchbase . bucket . name = test spring . couchbase . bucket . password = couchbase spring . data . couchbase . auto - index = true

In order to test our microservice, I added a few Restful endpoints:

@RestController @RequestMapping("/api/user") public class UserServiceController { @Autowired private UserService userService; @RequestMapping(value = "/{id}", method = GET, produces = APPLICATION_JSON_VALUE) public User findById(@PathParam("id") String id) { return userService.findById(id); } @RequestMapping(value = "/preference", method = GET, produces = APPLICATION_JSON_VALUE) public List<User> findPreference(@RequestParam("name") String name) { return userService.findUsersByPreferenceName(name); } @RequestMapping(value = "/find", method = GET, produces = APPLICATION_JSON_VALUE) public List<User> findUserByName(@RequestParam("name") String name) { return userService.findByName(name); } @RequestMapping(value = "/save", method = POST, produces = APPLICATION_JSON_VALUE) public User findUserByName(@RequestBody User user) { return userService.save(user); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 @ RestController @ RequestMapping ( "/api/user" ) public class UserServiceController { @ Autowired private UserService userService ; @ RequestMapping ( value = "/{id}" , method = GET , produces = APPLICATION_JSON_VALUE ) public User findById ( @ PathParam ( "id" ) String id ) { return userService . findById ( id ) ; } @ RequestMapping ( value = "/preference" , method = GET , produces = APPLICATION_JSON_VALUE ) public List < User > findPreference ( @ RequestParam ( "name" ) String name ) { return userService . findUsersByPreferenceName ( name ) ; } @ RequestMapping ( value = "/find" , method = GET , produces = APPLICATION_JSON_VALUE ) public List < User > findUserByName ( @ RequestParam ( "name" ) String name ) { return userService . findByName ( name ) ; } @ RequestMapping ( value = "/save" , method = POST , produces = APPLICATION_JSON_VALUE ) public User findUserByName ( @ RequestBody User user ) { return userService . save ( user ) ; } }

Dockerizing your Microservice

First, change your application.properties to get the connection credentials from environment variables:

spring.couchbase.bootstrap-hosts=${COUCHBASE_HOST} spring.couchbase.bucket.name=${COUCHBASE_BUCKET} spring.couchbase.bucket.password=${COUCHBASE_PASSWORD} spring.data.couchbase.auto-index=true 1 2 3 4 spring . couchbase . bootstrap - hosts = $ { COUCHBASE_HOST } spring . couchbase . bucket . name = $ { COUCHBASE_BUCKET } spring . couchbase . bucket . password = $ { COUCHBASE_PASSWORD } spring . data . couchbase . auto - index = true

And now we can create our Dockerfile:

FROM openjdk:8-jdk-alpine VOLUME /tmp MAINTAINER Denis Rosa <denis.rosa@couchbase.com> ARG JAR_FILE ADD ${JAR_FILE} app.jar ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"] 1 2 3 4 5 6 FROM openjdk : 8 - jdk - alpine VOLUME / tmp MAINTAINER Denis Rosa < denis . rosa @ couchbase . com > ARG JAR_FILE ADD $ { JAR_FILE } app . jar ENTRYPOINT [ "java" , "-Djava.security.egd=file:/dev/./urandom" , "-jar" , "/app.jar" ]

Then, we build and publish our image on Docker Hub:

Create your image:

./mvnw install dockerfile:build -DskipTests 1 . / mvnw install dockerfile : build - DskipTests

Log into the Docker Hub from the command line

docker login 1 docker login

Let’s grab the imageId of our recently created image:

docker images 1 docker images



Create your new tag using the imageId:

//docker tag YOUR_IMAGE_ID YOUR_USER/REPO_NAME docker tag 3f9db98544bd deniswsrosa/kubernetes-starter-kit 1 2 //docker tag YOUR_IMAGE_ID YOUR_USER/REPO_NAME docker tag 3f9db98544bd deniswsrosa / kubernetes - starter - kit

Finally, push your image:

docker push deniswsrosa/kubernetes-starter-kit 1 docker push deniswsrosa / kubernetes - starter - kit

Your image should now be available at Docker Hub:

Configuring the Database

I wrote a whole article about it here, but to keep it short. Just run the following commands inside the kubernetes directory.

./rbac/cluster_role.sh kubectl create -f secret.yaml kubectl create -f operator.yaml kubectl create -f couchbase-cluster.yaml 1 2 3 4 . / rbac / cluster_role . sh kubectl create - f secret . yaml kubectl create - f operator . yaml kubectl create - f couchbase - cluster . yaml

After a while, all the 3 instances of our database should be running:

Let’s forward the Web Console’s port to our local machine:

kubectl port-forward cb-example-0000 8091:8091 1 kubectl port - forward cb - example - 0000 8091 : 8091

And now we can access the web console at http://localhost:8091. You can log in using the username Administrator and the password password

Go to Security -> ADD USER with the following properties:

Username: couchbase-sample

couchbase-sample Full Name: couchbase-sample

couchbase-sample Password: couchbase-sample

couchbase-sample Verify password: couchbase-sample

couchbase-sample Roles: According to the image below:

OBS: In a production environment, please don’t add your app as an admin

Deploying your Microservice

First, let’s create a Kubernetes secret where we will store the password to connect to our database:

apiVersion: v1 kind: Secret metadata: name: spring-boot-app-secret type: Opaque data: bucket_password: Y291Y2hiYXNlLXNhbXBsZQ== #couchbase-sample in base64 1 2 3 4 5 6 7 apiVersion : v1 kind : Secret metadata : name : spring - boot - app - secret type : Opaque data : bucket_password : Y291Y2hiYXNlLXNhbXBsZQ == #couchbase-sample in base64

Run the following command to create the secret:

kubectl create -f spring-boot-app-secret.yaml 1 kubectl create - f spring - boot - app - secret . yaml

The file spring-boot-app.yaml is the one responsible for deploying our app. Let’s take a look at its contents:

apiVersion: apps/v1beta1 kind: Deployment metadata: name: spring-boot-deployment spec: selector: matchLabels: app: spring-boot-app replicas: 2 # tells deployment to run 2 pods matching the template template: # create pods using pod definition in this template metadata: labels: app: spring-boot-app spec: containers: - name: spring-boot-app image: deniswsrosa/kubernetes-starter-kit imagePullPolicy: Always ports: - containerPort: 8080 name: server - containerPort: 8081 name: management env: - name: COUCHBASE_PASSWORD valueFrom: secretKeyRef: name: spring-boot-app-secret key: bucket_password - name: COUCHBASE_BUCKET value: couchbase-sample - name: COUCHBASE_HOST value: cb-example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 apiVersion : apps/v1beta1 kind : Deployment metadata : name : spring-boot-deployment spec : selector : matchLabels : app : spring-boot-app replicas : 2 # tells deployment to run 2 pods matching the template template : # create pods using pod definition in this template metadata : labels : app : spring-boot-app spec : containers : - name : spring-boot-app image : deniswsrosa/kubernetes-starter-kit imagePullPolicy : Always ports : - containerPort : 8080 name : server - containerPort : 8081 name : management env : - name : COUCHBASE_PASSWORD valueFrom : secretKeyRef : name : spring-boot-app-secret key : bucket_password - name : COUCHBASE_BUCKET value : couchbase-sample - name : COUCHBASE_HOST value : cb-example

I would like to highlight some important parts of this file:

replicas: 2 -> Kubernetes will launch 2 instances of our app

Kubernetes will launch 2 instances of our app image: deniswsrosa/kubernetes-starter-kit -> The docker image we have created before.

The docker image we have created before. containers: name: -> Here is where we define the name of the container running our application. You will use this name in Kubernetes whenever you want to define how many instances should be running, autoscaling strategies, load balancing, etc.

Here is where we define the name of the container running our application. You will use this name in Kubernetes whenever you want to define how many instances should be running, autoscaling strategies, load balancing, etc. env: -> Here is where we define the environment variables of our app. Note that we are also referring to the secret we created before.

Run the following command to deploy our app:

kubectl create -f spring-boot-app.yaml 1 kubectl create - f spring - boot - app . yaml

In a few seconds, you will notice that both instances of your application are already running:

Finally, let’s expose our microservice to the outside world. There are dozens of different possibilities of how it can be done. In our case, let’s simply create a Load Balancer:

apiVersion: v1 kind: Service metadata: name: spring-boot-load-balancer spec: ports: - port: 8080 targetPort: 8080 name: http - port: 8081 targetPort: 8081 name: management selector: app: spring-boot-app type: LoadBalancer 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 apiVersion : v1 kind : Service metadata : name : spring-boot-load-balancer spec : ports : - port : 8080 targetPort : 8080 name : http - port : 8081 targetPort : 8081 name : management selector : app : spring-boot-app type : LoadBalancer

The selector is one of the most important parts of the file above. It is where we define the containers to which the traffic will be redirected. In this case, we are just pointing to the app we have deployed before.

Run the following command to create our load balancer:

kubectl create -f spring-boot-load-balancer.yaml 1 kubectl create - f spring - boot - load - balancer . yaml

The load balancer will take a few minutes to be up and redirecting traffic to our pods. You can run the following command to check its status:

kubectl describe service spring-boot-load-balancer 1 kubectl describe service spring - boot - load - balancer

As you can see in the image above, our Load Balancer is accessible at ad84a916d65ad11e884a20266aaa53c9-1223617270.us-west-2.elb.amazonaws.com, and the targetPort 8080 will redirect traffic to two endpoints: 10.2.1.6:8080 and 10.2.2.7:8080

Finally, we can access our application and start sending requests to it:

Inserting a new user:



Searching for users:

What about being elastic?

This is where things get really interesting. What if we need to scale up the whole of our microservice? Let’s say the Black Friday is coming and we need to prepare our infrastructure to support this massive flow of users coming to our website. Well, that is an easy problem to solve:

To scale up our application, we just need to change the number of replicas in the spring-boot-app.yaml file.

... spec: selector: matchLabels: app: spring-boot-app replicas: 6 # tells deployment to run 6 pods matching the template template: # create pods using pod definition in this template metadata: labels: ... 1 2 3 4 5 6 7 8 9 10 . . . spec : selector : matchLabels : app : spring-boot-app replicas : 6 # tells deployment to run 6 pods matching the template template : # create pods using pod definition in this template metadata : labels : . . .

And then, run the following command:

kubectl replace -f spring-boot-app.yaml 1 kubectl replace - f spring - boot - app . yaml

Is there anything missing? Yes. What about our database? We should scale it up as well:

Change the size attribute in the couchbase-cluster.yaml file:

... enableIndexReplica: false servers: - size: 6 name: all_services services: - data - index ... 1 2 3 4 5 6 7 8 9 . . . enableIndexReplica : false servers : - size : 6 name : all_services services : - data - index . . .

Finally, run the following command:

kubectl replace -f couchbase-cluster.yaml 1 kubectl replace - f couchbase - cluster . yaml

How can I Scale it down?

Scaling down is as easy as scaling up; you just need to change both couchbase-cluster.yaml and spring-boot-app.yaml:

couchbase-cluster.yaml

... enableIndexReplica: false servers: - size: 1 name: all_services services: - data - index ... 1 2 3 4 5 6 7 8 9 . . . enableIndexReplica : false servers : - size : 1 name : all_services services : - data - index . . .

spring-boot-app.yaml:

... spec: selector: matchLabels: app: spring-boot-app replicas: 1 template: metadata: labels: ... 1 2 3 4 5 6 7 8 9 10 . . . spec : selector : matchLabels : app : spring - boot - app replicas : 1 template : metadata : labels : . . .

And run the following commands:

kubectl replace -f couchbase-cluster.yaml kubectl replace -f spring-boot-app.yaml 1 2 kubectl replace - f couchbase - cluster . yaml kubectl replace - f spring - boot - app . yaml

Auto-scaling microservices on Kubernetes

I will make a deep dive into this topic in part 2 of this article. In the meantime, you can check this video about pod autoscaling.

Troubleshooting your Kubernetes deployment

If your Pods fail to start, there are many ways to troubleshoot the problem. In the case below, both applications failed to start:

Since they are part of the deployment, let’s describe the deployment to try understanding what is happening:

kubectl describe deployment spring-boot-deployment 1 kubectl describe deployment spring - boot - deployment

Well, nothing is really relevant in this case. Let’s look at one of the pod’s logs then:

kubectl log spring-boot-deployment-74649f869d-74sq8 1 kubectl log spring - boot - deployment - 74649f869d - 74sq8

Gotcha! The application did not start because we forgot to create the user on Couchbase. By just creating the user, the pods will be up in a few seconds:

Conclusion

Databases are stateful applications, and scaling them is not as fast as scaling stateless ones (and probably it will never be), but if you need to make a truly elastic architecture, you should plan to scale all components of your infrastructure. Otherwise, you are just creating bottleneck somewhere else.

In this article, I tried to show just a small introduction about how you can make both your application and database on Kubernetes elastic. However, it is not a production-ready architecture yet. There are a plenty of other things to consider still, and I will address some of them in the upcoming articles.

In the meantime, if you have any questions, tweet me at @deniswsrosa or leave a comment below.

Read Also: