“The world is changing very fast. Big will not beat small anymore. It will be the fast beating the slow.” — Rupert Murdoch, CEO, News Corporation

Early last month, we shipped VWO’s biggest product update yet: VWO Conversion Optimization Platform.

Almost 2 years in the making, this was one of the biggest product launches ever. Exhilarating? Of course! We’re thrilled to bring something this unique to our customers and to the world. The Conversion Optimization Platform aims to be a critical contributor towards success of conversion optimization programs run by our customers.

Now’s the time to rejoice and party (and yes, we did have a big bash last week!). Now’s also the time to pause and reflect back on things we did right but more importantly, things we did wrong (and learn from them) in the last two years. We Wingifighters, are a bunch of very self critical folks and I will not let this self-reflection opportunity go by so that we and rest of the world can get better at building SaaS products through these learnings.

Lesson 1: Incremental and iterative vs. big bang shock and awe

Wingify is a big advocate of Agile and Lean and makes all conscious efforts to abide by these philosophies. We realise that being Agile is not a destination, it’s an ongoing journey of constant self reflection and improvement. Execution of Conversion Optimization Platform clearly highlighted the fact that we indeed have a long way to go before we can call ourselves a truly Agile company.

Around early 2015, in the cosy Delhi winters, we figured Conversion Optimization Platform was a great idea and it wouldn’t take us more than a quarter to nail it — may be two quarters at most. Our launch strategy was to go for one big bang release and launch it all together — the new product, the new pricing, the new website — the whole enchilada. We were pumped to make it happen, we were full of confidence, we were unstoppable, and we were so WRONG!

A quarter turned into four and then eight. Obviously, we were late, we knew that, but worst of all we kept on building a product for 8 freaking quarters without it going live into production, without it making us any revenue, and worse, without it soliciting the ‘real’ feedback from our customers — the kind of feedback you get only when customers are actually shelling out their hard earned dollars for something they find valuable. Don’t get me wrong. We still did a series of closed and open betas and hundreds of customer feedback sessions all along, but that still doesn’t justify going without a major generally available product release that provides real customer value and makes you money for two straight years.

In hindsight, our approach should’ve been much more incremental and Agile. Building ambitious and earth shattering products is great but what’s better (at least in SaaS world) is to build, ship, price and launch them in incremental slices that provide ongoing stream of value to your customers and early feedback to you on the usefulness, usability, and viability of your product.

Lesson 2: Product requirements and UX must lead the development, not the other way round

Now that we’re sailing smooth in the self-thrashing boat, let’s do a little autopsy to figure out some of the reasons why it took us this long and comfortably perched on top of this list is the jesty slogan that says ‘code first, think later’.

For complete disclosure and fairness, we did have a high level product strategy to begin with. Where we lapsed was effective translation of this strategy into clearly thought through and user verified product specs and UX designs before developers started building it. There were quite a few instances where, with the misplaced sense of getting a ‘head-start’, engineering went their own way building a feature even before the product specs were fully detailed out. This resulted in us trashing and re-doing fully built features. As a learning, along the way while building Conversion Optimization Platform, we introduced following formal stages in the feature life-cycle to ensure this was not repeated and in general, it has served us well:

New Idea : a feature idea is born either based on direct customer feedback or our internal brainstorming.

: a feature idea is born either based on direct customer feedback or our internal brainstorming. Research and Specs: product team does market, customer, and user research to figure out how can we best solve for the underlying need represented by the feature idea.

product team does market, customer, and user research to figure out how can we best solve for the underlying need represented by the feature idea. Need Validation: through use of low fidelity wireframes, and paper prototypes, product team validates two things: a) are we solving for a valid customer need? b) does our solution to this need make sense?

through use of low fidelity wireframes, and paper prototypes, product team validates two things: a) are we solving for a valid customer need? b) does our solution to this need make sense? Technical Feasibility: at this stage, engineering team is involved to propose and evaluate multiple possible technical solutions for a given idea. Goal is not to freeze on the exact technical design — that happens later, as part of actual development Sprints. Rather, at this point, the goal is to provide a technically viable high level design direction for the idea. This is a very critical step that prevents us from ‘biting more than what we can chew’ and from setting upon yet another multi-month project journey.

at this stage, engineering team is involved to propose and evaluate multiple possible technical solutions for a given idea. Goal is not to freeze on the exact technical design — that happens later, as part of actual development Sprints. Rather, at this point, the goal is to provide a technically viable high level design direction for the idea. This is a very critical step that prevents us from ‘biting more than what we can chew’ and from setting upon yet another multi-month project journey. Design and Design Validation: now, our awesome design team picks up the requirements and churns our high fidelity screens that are used for design validation via usability interviews.

now, our awesome design team picks up the requirements and churns our high fidelity screens that are used for design validation via usability interviews. Dev Pipeline: this is when the feature-idea is ready to be picked up by the developers for coding. Remember, this is not the first time they’re getting involved — they’ve already contributed in bringing the feature-idea to this state.

One more important point to note, the steps above are not exactly sequential. There can be significant amount of back and forth and iterations between let’s say ‘Research and Specs’ and ‘Need Validation’ states. And before the Agile purists cry “This is God-damn water-fall”, let me clarify — it’s not. We do allow for more granular and iterative technical and UX design changes during development Sprints, but freezing upon initial direction provides a solid framework to build upon. It also helps avoid the ‘excessive internal tinkering’ described in the next lesson.

Introduction of these steps went a long way towards boosting our execution and shipping speed.

Lesson 3: When it comes to product design, never confuse fickle-mindedness and misplaced sense of perfection with Agility

Changes based on real customer feedback and data is Agility, excessive tinkering based on internal opinions is not. Period.

During the course of building Conversion Optimization Platform, there were at least two instances where the product changes that should’ve taken couple of weeks ended up taking months. Why? Here’s a little anecdote to explain why:

It’s go live day, yay! One of the many ‘approvers’ barges in and says something to this effect — “Guys, hate to do this, but that pop-up animation is not quite right. Can’t put my finger on it, but something inside tells me it can be further improved. Let’s hold off the release for couple more days and make it PERFECT.” This is just one of the examples, but it happened more number of times than I am willing to admit. I happened to be one of those approvers!

What’s wrong with shooting for perfection — you may ask — especially, for a company which has customer focus and great UX as its DNA. And you’d be right, albeit partially. Problem is that perfection is subjective and perfect UX design even more so. The trap we fell into was to try to get it absolutely right on the first go and while doing so, we violated the very essence of Agile — release fast, and iterate fast based on data and direct user (and market) feedback.

We realised this wasn’t gonna work and we did eventually solve for this. Here’s how:

Step 1 — We cut down the list of approvers to bare-minimum. Everyone was welcome to voice their opinion and we did solicit lots of it through internal and external design walkthroughs and feedback sessions. However, the list of final decision makers was reduced to just two people. DACI model for decision making was put to good use. We started seeing better results in terms of increased throughput and improved team morale as soon as we got serious about DACI. The nitty-gritties of DACI and how this framework helped us deserve a separate blog post (in the making).

— We cut down the list of approvers to bare-minimum. Everyone was welcome to voice their opinion and we did solicit lots of it through internal and external design walkthroughs and feedback sessions. However, the list of final decision makers was reduced to just two people. DACI model for decision making was put to good use. We started seeing better results in terms of increased throughput and improved team morale as soon as we got serious about DACI. The nitty-gritties of DACI and how this framework helped us deserve a separate blog post (in the making). Step 2 — Program Management was formally introduced to own up the execution, processes, delivery, and timelines. More on this in Lesson 4.

— Program Management was formally introduced to own up the execution, processes, delivery, and timelines. More on this in Lesson 4. Step 3 — Product design review and approval process as described in Lesson 2 were put into effect. There were still some lapses here and there, but in general, we got super strict about it and it paid off.

Finally, what really helped and drove all of the above changes was a shift in leadership mindset that we need to find the delicate balance between speed and quality. Shipping fast is just as important as quality and at times, it may come at the cost of releasing something less than perfect and that’s OK. It was a hard pill to swallow for some of us, but we did learn!

Lesson 4: Estimations, forecasting and planning don’t make you less Agile

This one goes hand in hand with Lesson 1. I am going to be completely honest with you. In our early days, when we were in startup mode, we never bothered with estimations, and execution plans. If it had to be done, it would be done. Fast. Simple as that. Over the time, as our product matured, complexity inevitably crept in. Supposedly ‘quick’ changes to product weren’t quick anymore. Our dev team grew manifolds with people at varying degrees of experience and motivation. Thanks to the increased complexity, what used to be a ‘one person for a day job’ turned into a week long focused effort for a full stack team. Of course, we’ve been constantly working on figuring out innovative ways to cut down this complexity and maintaining velocity. However, the simple fact of the matter is that we had to learn to be far more judicious in picking up our projects and carefully plan for them before diving head first into something that accidentally turns out to be a multi-month behemoth. With estimations and rough forecasting, such mammoth projects can be better sliced into incremental iterations that deliver ongoing stream of value to our customers.

The up-front estimations and forecasting have another very nice side-effect. If you do it right and truly involve your dev team in it, it becomes a forcing function for them to figure out at least a high level technical architecture. If they don’t do it, whatever estimations they provide would be baseless. To be clear, I am not talking every nitty gritty of technical design to be figured out before starting a project — that’d be a futile exercise. However, an understanding of high level architecture does help us plan ahead in case we need to lets say switch our database systems, which incidentally, we had to while building Conversion Optimization Platform — again an expensive change, late in the game that could’ve been avoided easily had we planned ahead.

I must add that “Responding to change over following a plan” principle of Agile manifesto is often deeply misinterpreted. Being Agile does not mean you forgo strategic and release level planning based on ‘good-enough’ estimates. The trick is to keep the longer term plan flexible enough to accommodate unforeseen market and strategic changes and equally importantly, to keep evolving it over time. The exact practices we apply for estimations and planning at Wingify is a topic worth an entire blog post (coming soon!).

Reflecting back, building VWO Conversion Optimization Platform has been a fun and educational ride. With the lessons learnt along the way, we are a far more mature and effective team compared to our 2015 version. Congratulations to our customers and to team Wingify on completing this amazing journey. Onwards to bigger and better things!