A fortnight ago I wrote 10 Embarrassing Flaws That Made My ‘Weekend’ App Possible, which was about my two-week project to recommend new board games. As it turns out, that post was picked up by Reddit and HN a couple of days later in a big way, bringing in over 50,000 hits in a matter of hours. This is what that looks like:

At peak around 4,000 hits per hour were coming in, or just over 1 per second. The effect was pretty noticeable on twitter and in my inbox, too:

About an hour into this it suddenly struck me that this would be a very good time to add the Amazon affiliate links to findanewgame that I’d been meaning to. I was in a rush, so I just added a link to the search page, something like this. I wanted to link directly to the products but hadn’t got around to implementing the ItemSearch API yet. I didn’t have time! The site was getting traffic:

Maybe 12% of the blog readers checked out the findanewgame site and some of them bought games! It was working - I’d helped people find cool new board games! By the way, if you’re one of them then I hope you’re having fun with it - I’d love to hear how it’s going.

Fourteen Days Later

Traffic from HN and Reddit doesn’t really stick around - life returned to normal pretty quickly:

The findanewgame site itself was actually submitted to reddit on its own a few days ago (which I completely missed); here’s what its traffic looks like now:

Strangely, most of the traffic to the game site is ‘direct’, which can mean almost anything in analytics. Privacy-aware browsers, links sent thorough offline email programs, who knows? Not Google, that’s for sure:

So 100 Facebook likes drives less than 100 visits? Wow. Is that normal? I don’t know. Maybe I should care, but there’s something more pressing on my mind at the moment: I broke something.

I Should Have Used A/B Testing

You see, the Saturday after the post went big I made some more time to work on the site. Top of my list was sorting out the Amazon links, which I did. Now most games link directly to the product page - saving my visitors an extra click. Great! Great? Well, here are the Amazon stats. Can you tell when I made the change?

Yeah. That’s a lot of 0.00%, huh? Actually I deployed the change on the 23rd, which enjoyed a bumper 4,48% conversion rate. I thought this was fantastic, until the next day. And the next. And every day since.

What happened? I’ve no idea. People are still coming through to Amazon, so the links can’t be broken. They seem to be linking to the right products. For some reason, though, nobody’s ordering games now. Even with a 1% conversion rate I’d expect to have seen 2-3 orders since then.

If I’d been A/B testing every deployment, I’d know whether this was something I’d done or not. I actually can’t imagine why every deployment isn’t rolled out via an A/B testing process. Does anyone know of a really slick way to integrate this with the AppEngine workflow?

Choose Your Own Adventure

Where should I go from here? There are several options:

Roll back the Amazon link changes - maybe people will start placing orders again; it’s a quick change to make, but it’ll take a while to see if it has any effect.

Add A/B testing to my workflow - I’d love to push every deployment into an automatic A/B trial and then later decide which versions to keep and which to rework. Do you know of anything like this for AppEngine?

Post recommendation links to forums - I don’t know anything about marketing, but the stickiest traffic I get seems to be from forums. I’ve recently added short urls; I can spend a few hours visiting board game forums, reading the recommendation posts and using findanewgame to offer suggestions.

Make it easy to create and share lists - this has always been a goal, because sharing things we like is fun and because Christmas is coming up. At the moment you build up a list of games you want to play; I could make this easier, then let you email yourself or someone else the list, share it on facebook or whatever. Perhaps this option could also be called 'market the site socially’.

This adventure is fun to do and even more fun to share, so which two tasks are should I do next? Vote below - I’ll do whatever wins and write about it again in a couple of weeks!

Browser not showing the poll inline? Try http://polldaddy.com/poll/4033559

Update: Hacker News discussion