As mentioned in my article about visualization and prioritization of technical debt: analogies can help a lot with grasping abstract concepts. And this is where a term like “Smoke Testing” comes in handy, as long as everyone involved has the same understanding of the analogy…

The story

A colleague and I were discussing how to verify the functionality of our recent production deployment. He mentioned that we could do some manual smoke testing. This confused me a little bit since that’s not how I understood “smoke testing”.

Luckily for me, I knew where the analogy came from and I could derive what smoke testing in software development actually means from that origin. So I started explaining why smoke testing is called smoke testing and why his understanding is wrong!

Hardware Smoke Testing

I pointed out that the term “smoke testing” originated from hardware development, where when you power on a circuit board for the first time and you see smoke rising you know it’s broken¹.

Therefore, I was arguing, that simply booting up the application and making sure that the application doesn’t immediately die would be a sufficient smoke test. This could be done automatically as a step in the build pipeline.

The phrase smoke test comes from electronic hardware testing. You plug in a new board and turn on the power. If you see smoke coming from the board, turn off the power. You don’t have to do any more testing. ~ Lessons Learned in Software Testing(Cem Kaner, James Bach, Brett Pettichord)

Smoke Testing in Plumbing

So far so good, right? I felt my explanation was sound and since it matched any uses of the term I had seen in software development, I had a valid reason to believe that this is the one and the only true explanation of the term and my colleague could do nothing else but agree with me.

Boy, was I wrong!

Right after I finished my explanation, my colleague (who might or might not have been a plumber in a previous life) started sharing his understanding of the origin of smoke testing and it was a very different story.

In the plumbing industry, smoke testing is done to verify that a pipe is not leaking.

Smoke is created artificially, blown into tubing, and then the pipes are examined to make sure no smoke is escaping where it shouldn’t to verify that there is no leak.

Transferring this understanding to software development, therefore, meant that after the application start, multiple simple user flows are executed to ensure basic functionality. And he added, in his understanding, this was mostly done manually in contrast to a full and automated functional test suit.

My mind was blown!

Not only did my colleague have a different story to tell about the origin of the term smoke testing but it all made sense as well! And I googled it — plumbers do actually do that and it is called smoke testing!

So, where did we go from there?

What we both agreed upon is that smoke testing:

Tests basic functionality

Is one of the first, if not the first test after deployment

Provides fast feedback to see if further testing even makes sense

However, we were indifferent about what basic functionality meant in this context. So we looked around what other people had to say about “smoke testing”. We found multiple definitions and the lines between these definitions were foggy (pun intended).

So we just agreed on something that made sense to us in our situation and went with that for the duration of the project.

Conclusion

What smoke testing really means for you depends on your team. Whatever meaning you chose does not matter, just make sure that everyone on your team has the same understanding.

When talking about it with people outside of your team be aware of different meanings and clarify first before it results in misunderstandings.

Regardless of the terminology, that you agree upon, smoke tests usually offer fast feedback to your development process and codebase.