Climbing the tower of despair… I mean product

One of the biggest and perhaps most common pitfalls not just in the world of data but in engineering as a whole is stopping to take a second and ask yourself “what am I doing, and how do I know when it’s done?” Normally this would be a business failure since ideally business and product should have a well-defined set of KPI’s for their data teams, however, should you find yourself in a situation where this is not the case it may be a good idea to define some objectives before you proceed. As part of an organisation be it a startup or a tech giant generally everything you do needs to have some measurable impact on the organisation, which brings us to the first hurdle: is what you are doing going to have an impact, and can you tell when it has?

In maturing organisations getting through that first hurdle is not always possible. It is common to encounter a situation where you can tell that what you are doing is going to bring some measure of value, but now comes the time to convince the product and business teams of this value. From my experience, it is generally not a failure on the part of anyone in the data spectrum but a galactic misalignment in the perceived place of data within an organisation.

The term “data-driven” has lost all meaning, AB Tests run rampant and expectations are almost never met, sound familiar? This brings me to the second hurdle which is, in essence, trying to bring into the fold a data-aware product team. A data-aware product team is essential for bringing the data triage (analytics, machine learning and engineering) together and bridging the business gap while reigning in those expectations.

Unfortunately, a sense of product understanding is not always clear within the domain of data. It is common to encounter rushed implementations because an algorithm that made an impact in one domain is assumed to make an impact in another and unfortunately with the misperceived smell of success in the air, it can at times be difficult to convince anyone to take a step back and reevaluate. It is in such cases that a data-aware product team is essential for creating a more manageable expectation.