Will there be new changes in the way people shop in the future as technology continues to evolve and iterate?

With the launch of “Taobao Mixed Reality Buy”, in collaboration with Microsoft, Alibaba’s Taobao answered the above question. It is the first application of Mixed Reality (MR) technology in the consumer sector. Portable devices allow consumers to read commodity information, life information and service information. Taobao Mixed Reality Buy was officially launched at the Creativity Festival , which took place from September 13 to 16 this year.





Taobao Mixed Reality Buy was exhibited in a 300 square metre "future shopping block", which includes an iFashion store, a Taobao specialty store - the Palace Museum Taobao and a comic and animation shop.





Before entering the venue, users can open the Taobao app on their phones to scan the QR code to the bound account on the Hololens (a holographic computing device developed by Microsoft, and also the most important device on Taobao MR Buy). During MR shopping, all items put into the shopping cart by users will be synced with their Taobao account on their phones.





In the empty venue, there was another new world seen through Hololens, such as the dolphins greeting visitors on the ceiling and Alibaba's mascot leaping from the 2D airfield to the physical location.









In the Palace Museum Taobao store, animated versions of the Palace Museum's characters interact with users as they enter the store, as well as audio instructions when visitors browse through the products. When the users look at the emperor figures on the wall, these figures move in front of them. When users want to check the product information, they just need to open the laptop on the bookshelf and the product information is displayed around the laptop. Users can check the product information by moving their fingers and placing the product in the shopping cart with a pinch gesture at the purchase option. Currently, Taobao MR Buy does not support payment, that is, the final settlement must be completed on the Taobao APP. The MR Buy people say that the added payment function is expected in the future.









Taobao MR Buy is supported by Microsoft's MR infrastructure, including HoloLens equipment, virtual MR hybrid rendering capability, large spatial positioning capability, 3D human-scan software and hardware solutions. Taobao is responsible for product design, including the technical framework, online systems integration, and assisting Microsoft to improve and optimise spatial positioning performance and other aspects.





Is MR a good way to connect online and offline consumption?





At the Creativity Festival two years ago, Taobao launched a VR shopping product Buy+. Since then, however, it has not been updated for two reasons: First, because of the low penetration rate of the devices. Another is that it is impossible to smoothly integrate the online and offline shopping experience across the entire virtual scene presented by VR.





Since 2015, Taobao has been focusing on shopping on mobile phones, and predicting what daily consumption would look like in the future. After the previous experiment by Buy+, Taobao found that the consumer experience combined with virtual and offline content will be the main focus in the future as offline scenes become more important. MR will be the product that best meets the requirements of the offline and online experience. That is why Alibaba has invested in Magic Leap to launch the Taobao MR Buy.





*Note: In February 2016, Alibaba participated in Magic Leap’s C round of financing, which was just under $800 million. Magic Leap is an AR company based in the United States.





The stores displayed by Taobao in the "future shopping block" were three typical MR application scenarios. The clothing store iFashion has integrated some of the most popular items on Taobao into its products. The Palace Museum Taobao shop was an upgrade to the presentation of Taobao’s distinctive cultural assets, providing consumers with a more immersive reading experience; the cosmic and animation store aimed to provide consumers with more value-added services like a mixed reality gaming experience, beyond the products they bought. More than that,if you want to buy a tea cup online, you can see its effect on your table at home and purchase it by just moving your fingers through the MR device.





“A decade ago, the smartphone was just a nascent concept, but now it has become a ubiquitous device. We believe that MR devices will one day become a ubiquitous portable device and MR will be a shopping mode within the next 5 to 10 years,” commented Taobao MR Director.





Although the seamless connection between online and offline consumer experience through MR devices is wonderful, there are still many practical issues that need to be addressed to make MR more popular, such as the high cost of equipment and holographic image modelling. The speed of MR shopping's popularity greatly depends on the speed of the entire MR generation in the world. It is still not clear whether MR shopping will become the main shopping mode in the future.