This is Part 4 in a series examining The Cost of Kotlin Language Features in preparation for my presentation at KotlinConf in November. The series consists of

I’m publishing these results ahead of KotlinConf to give an opportunity for peer-review, so please do give me your feedback about the content, experimental method, code and conclusions. If you’re reading this before November 2017 it isn’t too late to save me from making a fool of myself in person, rather than just on the Internet. As ever the current state of the code to run the benchmarks is available for inspection and comment on GitHub.

This post looks at the cost of nullability. My aim with this batch wasn’t to measure individual aspects of nullability so much as to simply model typical usage, so I picked adding to a nullable Int as an example.

As usual we start with a baseline, consuming a constant expression passed in to the benchmark

open class KotlinPrimitives { @Benchmark fun _1_baseline(state: IntState, blackhole: Blackhole) { blackhole.consume(state._41) } }

And now a take that and add one to it

@Benchmark fun _2_sum(state: IntState, blackhole: Blackhole) { blackhole.consume(state._41 + 1) }

Irritatingly, for the data that I have, that addition isn’t statistically detectable.

@Test fun `kotlin sum is not detectable`() { assertThat(this::_1_baseline, ! probablyDifferentTo(this::_2_sum)) }

Looking at the bytecode we see that the problem is that the code that we want to measure is swamped by code that we don’t want to measure, but has to be there to make the measurements possible.

public final _2_sum(LcostOfKotlin/primitives/IntState;Lorg/openjdk/jmh/infra/Blackhole;)V @Lorg/openjdk/jmh/annotations/Benchmark;() @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0 @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 1 L0 ALOAD 1 LDC "state" INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V ALOAD 2 LDC "blackhole" INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V L1 LINENUMBER 22 L1 ALOAD 2 ALOAD 1 INVOKEVIRTUAL costOfKotlin/primitives/IntState.get_41 ()I ICONST_1 IADD INVOKEVIRTUAL org/openjdk/jmh/infra/Blackhole.consume (I)V L2 LINENUMBER 23 L2 RETURN

That ICONST_1 IADD hidden in the middle is what we want to measure! Ah well. I could do better by not using the Blackhole , and by arranging for field access to the IntState._41 , and switching off null-checks (ideally we’d have a way to do that for just this method with an annotation), but, well, I have a more detectable fish to fry.

@Benchmark fun _3_sum_nullable(state: IntState, blackhole: Blackhole) { blackhole.consume(state.nullable_41!! + 1) } @Test fun `sum nullable is slower`() { assertThat(this::_2_sum, probablyFasterThan(this::_3_sum_nullable)) assertThat(this::_2_sum, ! probablyFasterThan(this::_3_sum_nullable, byAFactorOf = 0.001)) }

Adding a not-null !! assertion is statistically slower (when testing a value that is never null), but by less than 0.1%. In the bytecode we see the cost of the nullability

DUP IFNONNULL L2 INVOKESTATIC kotlin/jvm/internal/Intrinsics.throwNpe ()V L2 INVOKEVIRTUAL java/lang/Integer.intValue ()I

is a check for Integer != null and fetching the Int out of its box.

What if we don’t know that Int can’t be null? We can use the Elvis operator to substitute 0 for null.

@Benchmark fun _4_sum_always_null(state: IntState, blackhole: Blackhole) { blackhole.consume(state.nullInt ?: 0 + 1) } @Test fun `sum always null is slower`() { assertThat(this::_2_sum, probablyFasterThan(this::_4_sum_always_null)) assertThat(this::_2_sum, ! probablyFasterThan(this::_4_sum_always_null, byAFactorOf = 0.001)) }

Uh huh - again, slower, but by less than 0.1% when our value is always null. Here’s the bytecode

DUP IFNULL L2 INVOKEVIRTUAL java/lang/Integer.intValue ()I GOTO L3 L2 POP ICONST_1 L3 INVOKEVIRTUAL org/openjdk/jmh/infra/Blackhole.consume (I)V

Argghh!

Can you see the problem?

It took me a while.

We aren’t doing the addition at all any more. The Elvis operator has a lower precedence than addition. So the expression we’re measuring turns out to be

state.nullInt ?: (0 + 1)

and the compiler is smart enough to know that 0 + 1 = 1 for all values of 0 and 1, so it just substitutes ICONST_1 .

Sigh.

I’ll have to fix that an re-run the benchmarks (which takes several hours), but in the meantime, luckily, I already have some more results.

For a long time I couldn’t detect any statistically significant different between the versions with and without null checks. I formed a hypothesis that, when the value being checked was always null or not null, branch prediction in either HotSpot or the processor was eating the cost of the check. So I measured with Int? s that were randomly null or not

@Benchmark fun _5_sum_50_50_nullable(state: IntState, blackhole: Blackhole) { blackhole.consume(state.`50 50 NullableInt` ?: 0 + 1) } @Benchmark fun _6_sum_90_10_nullable(state: IntState, blackhole: Blackhole) { blackhole.consume(state.`90 10 NullableInt` ?: 0 + 1) }

It turns out that branch prediction isn’t detectable

@Test fun `branch_prediction is undetectable_50_50`() { assertThat(this::_4_sum_always_null, ! probablyDifferentTo(this::_5_sum_50_50_nullable)) } @Test fun `branch_prediction is undetectable_90_10`() { assertThat(this::_4_sum_always_null, ! probablyDifferentTo(this::_6_sum_90_10_nullable)) }

but I suppose that Speculative Execution may play a part in the throughput of null checks.