It all began sometime last February, a little after Binny Bansal took over as Flipkart’s chief executive from cofounder Sachin Bansal . The new CEO authored a document outlining his vision for using new technology to transform how Indians shop online.That was a trigger for the top minds at the nearly decade-old ecommerce firm to question everything about what it was like for buyers to shop at Flipkart How easy is it for buyers to search for what they want? How personalised can recommendations get? Can Flipkart ‘talk’ with shoppers while they are searching for products online? How closely can Flipkart replicate for its buyers the experience of shopping at a regular store? How well can the backend operations be streamlined?These seemingly open-ended questions found an outlet with Project Mira, an artificial intelligence-focused effort that has been in stealth mode since last February. The ecommerce platform has been silently running various experiments on their app and website to understand their customers better. The name Mira was chosen to represent any common Indian.“On 28 February (2017), we launched the first version of our conversational search experience. Now, our users with broad intent (searching for, say, shoes or bedsheets) are guided with relevant questions, conversational filters, shopping ideas, offers and trending collections,” said Ram Papatla, vice president of product at Flipkart.Globally, Amazon and eBay have invested exhaustively in artificial intelligence to improve their marketplaces and have made significant headway in areas such as understanding natural language, making relevant product recommendations and improving search results. Flipkart’s AI project represents a massive effort at addressing similar complexities at scale but specifically for Indian shoppers still coming to grips with buying online.A few years ago, Flipkart began seeing a shift in the search patterns of its shoppers. They were traversing different screen sizes and spending more time on their mobile phones. Those in small cities were also taking to shopping online because of a lack of local options for the kind of quality products they wanted. Importantly, Flipkart had enough data to realize that had it followed an offline sort of a model-- a salesman asking a shopper specific questions on size, brand preference, etc—the company could have made searching for products on its online platform simpler for its customers.“When we looked at the (product) returns data and when we looked at data from shoes and lifestyle (categories), we saw a bunch of mismatch of expectations from our customers in terms of size and fit issues. If only we could have asked them one question we could have given the right response,” said Papatla. “It was an internal war cry. We have enough evidence (to say) that had we ‘talked’ to Mira (the online Indian shopper) we could have solved it.”The marketplace processes more than 400,000 shipments a day, of which customers return 10-11%. One-in-four fashion products such as clothing and accessories is returned because of reasons such as incorrect fit or as customers change their minds about a particular style.Flipkart’s team of experts started brainstorming for attributes that could be prompted to buyers instead of having them narrow the search results using filters. Say, someone comes to Flipkart searching for an air-conditioner. Because of Project Mira, Flipkart now asks buyers about what kind of AC they want, the tonnage, room size, brand, and such. It is a beginning in helping customers find the exact product they need in online settings that aren’t exactly easy to navigate. “It is a journey. I do not have a perfect recipe,” said Papatla. “But the good thing is I have a deep foundation base where I have built this technology based on Indian consumers and data that I can start to play around with for different experiments.”Ravi Garikipati, head of engineering at Flipkart, believes that the “deeply local Indian data” fed into Project Mira can give the company an edge in the domestic market. Through Project Mira, “we let our customers express themselves naturally (so we can) understand exactly what they are looking for or asking about,” he said. “Flipkart has every reason to expect that AI-led innovation through Project Mira will help us create solid competitive differentiation and cement our market leadership.”For Papatla, every day presents a new challenge in terms of what customers want from the marketplace. “The Kala Chasma song played the previous night, we get Kala Chasma queries the next morning. It is that insane. At that point I do not know what they are looking for--is it Katrina Kaif’s outfit or the glasses. It is hard to understand the intent,” he said.If quenching the customer’s needs was one part of the problem, the other was streamlining the backend processes. This includes a variety of tasks—accurate classification of products, accurate product descriptions, avoiding duplications.Over the past two years, Flipkart has expanded its product selection significantly. It adds more than 10 million products from around 20,000 sellers every month. The first step for any automated catalog is to accurately classify a product, which is hugely difficult given the unstructured data sellers often provide. “We now have a machine learning model that can classify the (product) vertical given an image. For verticals with similar images (say shampoos and body lotions), it uses the product description to classify products with 95% accuracy,” said Papatla.The machine learning algorithms can also detect incorrect images and morphed images. Posting the right images on the platform is crucial to getting customers interested in a product. “Another pressing problem is duplicate products. Sellers intentionally or unintentionally post duplicate products, which increases user effort in scanning to the desired set of products. Finally, we want to ensure that the descriptors of a product like colour, pattern, etc., are accurate,” said Papatla.Flipkart will soon guide sellers during their product listing process on what a buyer’s perception is likely to be for a particular image. “We have computed so much training models that if it is an image mistake, we can tell (sellers) to not upload as these kinds of images in the past have seen lower conversions. We are doing it with about 300 sellers now, we are handholding them. Through our sales team, we are providing the content to the sellers, saying this is your quality score, this is the score card for this month, and by the way, here are five opportunities to improve,” said Papatla.While Mira is an internal project, Flipkart is also outsourcing the task of finding solutions to several other problems online marketplaces struggle with. University collaborations play a pivotal role. And for this, Flipkart banks on the expertise of Muthusamy Chelliah, Director, Academic Engagement. Chelliah, who has worked with Yahoo! India in a similar role, ‘marries’ Flipsters with university professors to drive research in core technologies that can improve business for the company.“We are converging towards match-making between customer problems and published research literature. This process, in turn, could help identify the best faculty member to solve the consequent technical challenge,” said Chelliah.“In addition to IIT-Kharagpur/Kanpur/Bombay, Carnegie Mellon University and IIIT-Hyderabad, where we already have published results and have ongoing projects, we have engaged with the Indian Institute of Science on funded projects. Professors from Carnegie and Columbia are on the verge of active collaboration,” he said. Pabitra Mitra, associate professor at IIT-Kharagpur’s Department of Computer Science and Engineering, is working on developing a recommender system that can prod a customer to upgrade to more upmarket products, just as a salesman would.“Artificial intelligence has been developed to mimic a selling agent and consider the buying capacity of the customer, the nature of upsell, and personalized preferences, to provide the recommendation.Experimental evaluation of the algorithm is being performed on past purchase data before actual deployment,” Mitra said. “Upsell is a major source of revenue enhancement. It not only increases the value of the particular transaction but also induces subsequent purchase by the customer by exposing her to items that are of higher utility while still being affordable.”IIT-Kharagpur is also working on a chatbot that can answer product queries from Flipkart’s customers. “We have built the initial prototype using sequence-tosequence deep-learning framework that, given a customer query, provides up to five possible responses,” said Pawan Goyal, assistant professor at the Department of Computer Science. “During the initial evaluation, the correct response was found within these top five responses returned by the system.”Project Mira is still in its infancy and has a lot of ground to cover, but it certainly is seeing encouraging results. “Mira is expanded to several verticals. As a customer-focused company, that is less interesting. What’s interesting is we cover 50% of search volume through Mira, and through Mira, we are currently seeing 12% cart additions. We are seeing a dramatic improvement of seller experience as well,” said Papatla.Next on the agenda for Mira are issues such as product returns and quicker delivery. “We have a fair amount of returns; they are extremely expensive for us and they happen for a variety of reasons. How do you employ (machine learning) models to see if (a delivery) will lead to returns and for what reasons. We are also trying to estimate if we can deliver, say, in (an optimal) two days, because if it takes longer customers fall off, they cancel midway,” said Mayur Datar , chief data scientist at Flipkart. Partha Talukdar, assistant professor at Bengaluru’s Indian Institute of Science, believes that artificial intelligence has a lot more areas to explore in ecommerce, and marketplaces are just at the tip of the iceberg.“Traditional problem of relevance, personalization... in all of these things some amount of work is there. Clearly, a lot more needs to happen. Conversational aspect is important and not solved at all. Many of those things are one-off. (A buyer) should be able to ask complex queries. Say, if they are looking for a TV with a lot of attributes, quite a bit of work needs to be on there,” said Talukdar, who heads IISc’s Machine and Language Learning Lab, whose mission is to give a “worldview” to machines. “Also, mining of unstructured data… How can one use reviews on sites and ‘signals’ from them for product ranking? Different degrees of efforts are there but it has not been fully exploited.”