Krzysztof Ciesielski — Software Engineer

This month I learned

Circe is a solid and well adopted Scala JSON library. One of its most appealing features is auto-derivation. You only need to import io.circe.generic.auto._ and any case class can be now automatically transformed from/to JSON, provided that class fields are case classes, primitive types or custom types with encoders in scope. Such approach is very convenient, but can seriously affect your project’s compilation time.

We don’t have many case classes for JSON API, but we suspected that our compilation times (1m 45s and 2m in two main projects) could be improved if we skipped auto derivation.

We decided to fall back to semiauto derivation, which requires you to write a bit more code.

Here’s a full example:

As you can see, you need to add declarations of implicit encoders/decoders for each case class. You can put them in different places, but the most recommended is the companion object, which should make implicit resolution fast.

Applying this change made us speed up compilation by 20%-30%, which is a pretty good gain! Alternatively, you can check out the Magnolia project which solves auto-derivation in a fast way.

This tool helped me the most

Getting confused about compilation errors? Scala-clippy can make some of them less vague. Recently I wanted to combine Futures with cats’ *> operator, and I got this compilation error:

immediately followed by

The ExecutionContext can be easily missed, and you wouldn’t know that just by reading the original error. Thanks Clippy!

This I found really inspiring

Reactive programming: lessons learned” by Tomek Nurkiewicz.

It’s a very useful talk with many interesting insights about the dark side of so called reactive apps. Sane arguments backed by practice in a complex large-scale system. Highly recommended!

Dariusz Mydlarz — Software Engineer

Currently I am working on an event sourcing based project with Akka and Cassandra as the foundation pieces. Some time ago I got stuck with an odd behaviour. When I was executing the code I was getting Received sequence numbers out of order error over and over again from the Akka Persistence library.

I didn’t know where the root cause of the problem was lying. All the things were run locally so I was surprised that I got ordering problem without external network calls. After giving it some thoughts, this is what I found out.

In Akka Persistence an actor must implement an immutable persistenceId method. The defined ID is used to store and read messages sent to the actor. Messages are stored in a common journal — in my case it was a table in Apache Cassandra. Messages also get a sequence number next to the persistenceId while being written to the journal.

In my scenario I was using UUID as persistence ID to make sure there are no duplicates.

There were two actors: Actor A and Actor B. The UUID value was taken from the User entity. There was also an assumption that a single user cannot fit into both actors at the same time.

But the assumption was wrong. There were couple of cases when user fell into two roles simultaneously. Because of that I was receiving events from two actors with the same UUID and duplicated sequence numbers. Thus, the Akka Persistence Tag Writer was reporting errors of receiving messages out of order. To differentiate sources of messages I just needed to differentiate the persistenceId implementations.

Simply adding the unique prefix to the persistenceId was the thing I was looking for. Lesson learnt is really simple.

Do not ever use only uuid for persistenceId in Akka Persistence. Prefix it with an actor type name, please.

Marcin Kubala — Guardian of Type Safety

This month I learned

When using Avro4s to generate the Avro schemas from your Scala’s case classes, keep in mind that all the magic happens at macro expansion time (in other words — at the time when the code is compiled).

In Scala you can declare default parameter values and they do not have to be static, constant values. You can call a Scala object’s method or leverage values from the surrounding context/closure.

Avro is a language agnostic RPC & serialization framework which describes protocols & data types using schema files, just like Protobuf or Thrift.

And that’s when the tricky part comes in. As you probably noted, I’ve emphasised that Avro is language agnostic. The same schema can be used by various services written in Scala, Java, Python, Haskell, Go and many others. Therefore schema cannot contain any language-specific parts, like code snippets in the data type field’s default attribute.

It’s common to use default values for case class constructors and if you’re introducing Avro to an existing project and you wish to generate Avro schema files from the code, in order to save some time on writing schemas by yourself, you may get into serious trouble.

Let’s consider the following example:

The corresponding Avro schema would look like:

I’m not going to cover Avro basics in this post. This example is simple and seems to be pretty self-explanatory (if I’m wrong — here is the documentation).

This works like a charm, but now you would like to assign further information to your jokes. E.g. the date when you heard that dubiously funny joke for the first time. Assuming that no explicit date has been passed, you’ve just heard it for the first time right now:

Looks good, except it won’t work as expected and may lead you to a plenty of hard to catch issues.

Why? Let’s take a closer look at the produced schema:

Note that the default value for the firstTimeHeard field contains a random instant! Well, it’s not so random as you may think — it matches time when the schema was exported.

What happened here?

As I wrote before — Avro is language agnostic, so it tries to resolve default expression into a static value, that other implementation would understand.

Does it mean that we cannot use Avro4s schema generation mechanics together with runtime-based default field/parameter values?

Fortunately, there is a workaround!

Avro4s derives schemas from main constructors so we could introduce an alternate way for creating case class instances. Let’s try auxiliary factory, an additional apply method in the case class’ companion object and move our runtime defaults there:

And that’s all!

Note that on the above example I did not remove the default value from the joke field — since it’s a static value, we can keep it there.

Flaws

1) You have to remember to use the new keyword. Otherwise, you might end up with an infinite recursive call of the auxiliary apply method.

2) Adding such fields is not backwards compatible. Avro will not know what to put in place of a missing value when you try to read a new Chestnut instance from older binaries.

Hence this workaround is a way to go for essential fields, like id , createdAt (for events), and others, that we know to be present from the first revision.

Alternative solutions:

But I want to keep my schema backwards compatible! What should I do then?

In that case, the only options are:

1) Avoid default values that depend on dynamic (runtime) values. At least in your DTO/Event classes:

2) Use union type Option with default set to None :

In that case, you will have to handle missing values by yourself. That’s the price of compatibility.

Takeaways

The more abstraction over the original, basic tool/library you have, the easier is to forget some implications and limitations.

While developing software, never ignore implementation details and keep asking yourself How does it work? What if.. ?

Jakub Dziworski — Software Developer

This month I learned

We’ve already got lots of Scala related topics this month so to make this post a little bit more diverse I would like to share my quick weekend raspberry pi project.

Recently I decided to move to Netherlands for a while (yes — you can do that if you work for SoftwareMill ;)). I thus travel back and forth often, constantly facing one problem — cats! They need to be fed meat twice a day and asking friends to do that every time when I’m away is a struggle. What do geeks do in such case? Build remotely controlled bowls!

The plan was to ask the neighbour to come only once in the morning to feed the cats and leave the evening meal in “intelligent bowls” that will open up in the right time. There are already timed bowls on the market but I also wanted to be able to control the device remotely while watching cats over camera. Simply to prevent one of the cats eating two portions.

Initially I wanted to use a motor to open and close the bowl lit but one of my fellows at SoftwareMill recommended me to use servo. Generally speaking servo is a motor with additional electronics and gears inside. It is used for precise rotation — usually within 0–180 degrees range. I’ve never used this component before but it turned out to be a great solution — using a plain motor would be way more difficult to control precisely.

Apart from power supply, servo utilizes just one pin for the controlling PWM signal. The wider the pulse width of PWM, the bigger the angle — it’s just that simple. I therefore grabbed two servos from my local electronics store and hooked it up to a Raspberry Pi.

When it comes to software I decided to go with simplest solution — Node.js for exposing an HTTP endpoint and PIGPIO library for controlling the device with simulated PWM signal. The frontend is just a SPA doing Ajax calls to the backend when “Open”/”Close” button is pressed.

It took only one morning to set up hardware, program and test the project. Next day I spent on actually attaching servo to bowl and cover. I used epoxy plasticine-like material — flexible, but extremely strong after 24 hours hardening time. Next weekend I plan to make project more “production ready”. Things like covering servo with something waterproof, extending wires and covering them with plastic to prevent biting, etc.

I’m really amazed how easy it is nowadays to make electronic projects as a programmer. If you are like me in the past and avoid getting your hands dirty with electronics, because you think you don’t have enough skills — it’s very likely you are wrong — as a programmer you can learn basics in no time.

Mikołaj Koziarkiewicz — Senior Software Engineer

This month I learned

Due to a clean system upgrade and the need to install some packages, I rediscovered a feature of the font I use for coding, Fira Code. That feature is the support of “programming” ligatures.

A ligature is a special joining of two or more characters into a single one. One example is Æ. However, you’re probably even more familiar with the humble ampersand &, which originally came to be as a ligature of “et” (Latin for “and” — the relevant Wikipedia page provides an illuminating overview of the character’s evolution).

So, what does that have to do with coding? Well, eventually font designers figured out that they can use ligatures to better render operators composed of multiple characters, like “==”, “->” etc.

While — apparently — most popular among people writing in languages such as Haskell, Elixir, Clojure, etc., this feature is becoming very much relevant to Java and Scala developers as well.

Observe:

and the exact same text with ligatures enabled:

(both screenshots show Fira Code, size 12 pt., 0.8 line spacing)

As you can see, not only are the relevant characters joined, but occasionally completely converted. In addition, note that the font uses ligatures to better visually group e.g. the comment operators.

And the best part of it is, it’s just the representation — no modification of code required.

I remember I was not a fan of ligatures and didn’t switch them on, when I originally started using Fira Code. Now, a couple of years later, I’ve decided to try them out.

So far, I’m happy with them — the fact that editor support has improved, and the ligatures themselves were tweaked, probably helped.

Finally, I think ligatures provide a viable and more portable alternative to the recent Scala trend of using the scalafmt formatter to rewrite various arrow-like operators into their respective Unicode symbols.

Compare:

Thanks to the fact that ligatures can actually take as much space as their original characters (in this case 2), they can be made more readable than their single-character equivalents.

The font is available for free here (the main author has a Patreon page to fund ongoing development).

Specifically, you’ll probably be interested in:

editor and terminal support,

The general examples of ligatures,

and perhaps alternative fonts with similar features.

September community snippets from us!

We’ve joined SpreadIT in Katowice to talk&learn about software architecture and software craftmanship and answer all your question on our company stand! Thank you for visiting us and having a great time!