This is Part 1 of my design learnings from building a Google Home agent hack week project.

I recently joined a fashion tech company and participated in my first hack week. I work as a product manager with a huge interest in experience design and a fascination with how people are engaging with and reacting to voice technology, like Amazon Echo and Google Home devices.

I saw two opportunities in our customer’s journey where our customers had pain points and we didn’t engage with our users in their home environment.

Our customers unpack their orders in their home environment, deciding what items to keep and which to return and we’re not there to provide advice.

Every day our customers decide which outfit to wear from clothing they bought from us, and we provide little support on a daily basis.

I wanted to experiment with voice to build an engaging relationship with our customers. Therefore, I chose the pain point of, ‘what to wear today’ as the use case, hoping to test personalised recommendations, like the weather, their city and current trends. We began the exploratory research.

The conversation I anticipated:

User: Hey, what should I wear today?

Google Home: Well, it’s going to be -2 today in Berlin and you have that music gig after work. I’ll send a couple of outfit suggestions to match the Zara coat you bought last week.

User: (Checks their phone) Great, thanks.

Some of the data I wanted to explore and test user responses:

Explore our trend insights and recommendations and combine this with a user’s personal calendar, current location, weather, travel, dinner, parties etc.

Use a customer’s previous purchases to support them in their decision making and suggest related trending styles and outfits

Experiment with an in-home device and observe and learn about the human interaction side, especially in their home environment

Process:

Get an awesome team together, set a goal and measure of success. Everyone wants to learn different things from a hack week, so share this upfront and align on the team goal for the hack. We had a shared goal to experiment with voice interfaces, get the use case working and learn from each other. Our measure of success was to have a functioning prototype to test with users.

Define the Minimum Viable Product (MVP), or an achievable ‘story’. Tell a compelling story that shows the business opportunity and end-to-end user journey. We wanted to test the customer response to data (advice based on the weather and contextual trend recommendations) and their transition between chatting to Google Home and receiving recommendations on the mobile phone.

Agreeing on the MVP features (weather, location, mobile interaction) & using the conversation structure helped us keep focused and keep the concept simple to get something working. This flow changed as we tested out loud.

Even in a short hack week project, it was useful to share the longer term potential to help align the team, especially the developers, so they could design and build it in a way that was flexible for future possibilities.

The ‘Post MVP’ gave the team context of what could come next. This helped us keep the MVP simple and new ideas that came up along the way could get pushed into the ‘Post MVP’ bucket.

Design the persona for the agent. Create this early and involve your developers. Developers not only have good ideas to contribute, but the persona also gives everyone a common understanding of who our ‘character’ is. The tone of voice, personality, and the type of engagement our agent will provide to our users. This process made us consider if our character would be a metaphorical ‘Retail Assistant’ or ‘Stylist’. We decided on the latter, meaning we wanted to make recommendations with a focus on building a relationship, not to push customers to purchase like a retail assistant. Deciding what you won’t do is sometimes equally as important.

Build out the conversation flow early. The conversation will continually be improved, but have a visual structure early so everyone is talking about the same thing. I thought the story maps above would be enough, but we needed the intent and flow to work through the nitty gritty of successful and unsuccessful cases. We realised we were making a lot of assumptions in the conversation, without understanding how people interacted with the Google Home device, so we took to the corridors.

NB: This is not the original document, the whiteboard sessions were totally illegible. We started with the structural intent and gradually added more detail. The hardest part was closing the conversation loop and giving the user enough useful information at every step to keep them engaged.

Get the device in front of users as soon as possible. We all make assumptions based on how we interact with the device, what we like and what we find normal. A gentle reminder — if you’re working in tech, you are no longer a ‘normal’ user. Find real users, not colleagues. Testing early helped us learn about how people reacted to interacting with Google Home and our ‘agent’. Based on these learnings we redesigned the conversation to help guide the user by asking them more questions or making suggestions. For example, ‘do you like the dress or should I recommend something else?’

Embrace all the obvious mistakes you made. Use your structural conversation flow and speak the conversation out loud with someone else. Put on a hat and disguise and pretend you’re in the theatre - get playful. We ran through the conversation a number of times out loud to realise what was terrible, and what worked to make it as natural as possible. Including some tweaks from natural ‘Kiwi’ english to ‘German’ english. The variation in possible input from users became confusing, so in the end I made a document highlighting the variables, which could change the response, meaning or conversation.

We built the conversation using api.ai and they call variables, ‘entities’. Api.ai is a really simple interface that allows you to build conversational bots cross platform, including for Messenger, Google and Alexa. No matter your skill level, get into it and play around. The interface needs a few improvements (like visualising the whole conversation), but it’s an awesome start.

This is what we used to act out the conversation and try and create a natural conversation.

Close the conversation loop and complete the story. Keep iterating on the conversation until you have both the successful conversation flows and unsuccessful conversation flows working. We got the final conversation working in api.ai, integrating the weather, location and we faked the previous purchase for our MVP. We also faked sending visual recommendations to the phone using Twilio and an Invision hack. Fakes excused, we were really stoked to get the story working to convey the end-to-end experience to users.

Go live! Test it out loud with as many people as possible. Try and break it. Give difficult, complex answers. Be as negative as possible. But remember to make sure it also works on the ‘successful’ conversation flows too.

Keep observing, keep listening, keep learning. We were lucky to have over 100 people testing it for the hack presentations, so painfully saw our mistakes over and over again, but it was also one of the most rewarding experiences seeing the absolute magic it created for people. You could see their minds whirling considering the other possibilities they could apply this to in their job, or daily lives.

The final cut:

Audio of the final demo. The user converses, then gets some recommendations sent to their phone.

This was such a rewarding few days, we started the hack with 5 of us meeting and working together for the first time. We finished, celebrating with hundreds of people asking our agent ‘what should I wear’ and getting unique responses based on their input. It came together thanks to an awesome team with agile mindsets and the flexibility to learn and change scope as was necessary. I usually spend more time designing my team, but as I didn’t know many people I let it go. It turns out new(ish) technology attracts legends. Work with these people. They are there because they are passionate, continuously learning and curious.

Experiment, hack, break and learn — this is where you’ll find success.

For an overview of our design learnings, see Part 2