In previous blog posts we have described what the microservices are and how they can communicate. Today let’s focus on the infrastructure around them.

It may seem that when moving to microservices, the only thing you need to do is to code smaller projects and apply all those rules of loose coupling, independence etc, right? Actually no. Microservices need a whole organization shift, not only from the developers. This concerns management, project managers and what is most important… DevOps. Having more services emphasizes the importance of operations-related activities.

Continuous Integration & Continous Delivery

Having CI (Continuous Integration) is nowadays rather a standard in IT. There shouldn’t be any issues in supporting the build of multiple services. However, what about deployments? You don’t want to do them manually. With 20 services everything done manually takes definitely too much time. You have to think how to design your CI&CD flow and make them as least painful as possible. Maybe developers just need to create specific git tags and the service will deploy automatically? Great. Maybe they just need to promote some docker image and then they will get deployed? Good as well. But if they need to edit 20 deployments on the production manually, then it is not so great.

Logs aggregation

Getting logs from single monolith instances is quite simple. But when you have e.g. 20 different services and each of them deployed in 3 instances then browsing logs is a bit more complicated. It’s important to have a log aggregator. This may be Graylog or the ELK stack (Elasticsearch, Logstash, Kibana). With them you can quickly browse any logs and, what is more, to filter them

by service, by specific instance or by any custom field you would like to.

Metrics & Alerts

The same problems are related to metrics. It’s good to have e.g. Prometheus + Grafana. When you have all metrics in one place, then you can create dashboards allowing you to quickly know in what state is your system. Is everything working correctly? Which service instance is slow? Is there any performance change after the last deploy of one of the services? Answers to all of those questions can be quite useful.

If you hadn’t yet any opportunity to play with Grafana just take a look at the demo.

Debugging

When issues appear, then logs and metrics are definitely helpful. However, when you are using microservices, a single customer operation may be realized by a few services. How to know which log lines are related to it?! There comes tracing. Tools like Zipkin or Jaeger allow you to see when, and for how long, which service was handling which request. You can analyze the whole path, from start to the database and back to the user. But how to know which log lines are related to that execution? You just need to put the trace id in logs using MDC.

Keeping code safe & up to date

There are cases when services are just working without any maintenance. Nobody needs to touch them, change code or anything. However, after some time, their dependencies may get outdated. This may cause two problems:

Later, when you’ll need to develop new features, upgrades to current versions will be much more complicated (to go through dozen of releases) Security issues in leveraged libraries may be discovered in the meantime

How to avoid that? Two choices:

Include tools checking periodically if there are any dependencies upgrades in your CI flow. For Scala this would be e.g. Scala Steward. It creates a Pull Request for every found update. Include tools checking libraries for vulnerabilities in your CI flow. This may be e.g. snyk, or e.g. for JVM a dependency-check plugin. They will inform you if any holes for your dependencies were detected.

Run on localhost

CI/CD flow, metrics, security checks, all of them are nothing, comparing to simple possibility to run the system on developer PCs. Tests on localhost are a must — they save time and allow to avoid bugs. Always prepare scripts for local setup. You may leverage here Docker Compose or Minikube. Just make it possible and quite easy.

Conclusion

Moving to microservices definitely brings overhead. They have great advantages, but you need to remember about your infrastructure. It’s a huge pain to get into a situation when deploy takes a whole day, just because there is no sufficient automation. But don’t be scared, microservices are very useful, and with a proper team, you can go through all the challenges which are ahead of you. Just remember it’s not just about putting code in the containers.