Intro

A couple weeks ago, Yan Kou and Emmanuel Ameisen traveled to Beijing to attend the O’Reilly AI conference. Insight started in the USA, but we are always keeping an eye on how AI and Data Science is being used around the world, and have recently opened our first international office in Toronto, Canada. At the conference, we were lucky enough to attend some excellent talks on cutting edge research and production applications, and present about some of the lessons we’ve learned regarding how to quickly deploy NLP applications. In this article, we will share details of some presentations that peaked our interest.

Machine Learning in Production at Chinese scale

We’d like to start by highlighting two talks from China’s top-tier companies, Xiaomi and Meituan, in which they discussed AI’s applications for voice assistants and large-scale delivery dispatching, respectively.

Voice Assistants

Xiaomi is China’s largest smartphone company, and the world’s 5th largest with a valuation of $46 billion. In recent years, it has successfully developed an ecosystem for consumer electronics and smart IoT devices. Their latest breakthrough is “XiaoAi”, a Siri-like voice assistant. Gang Wang, a software engineer at Xiaomi, discussed the engineering design of XiaoAi’s natural voice system. While the overall strategy of XiaoAi’s natural voice processing is to have a central “brain” managing dozens of parallel tasks when a user talks to XiaoAi, it embeds specialized models (e.g. one specific model to use when asked about the weather and another when asked about purchasing tickets to an event) to reduce global response time. Xiaomi also leverages its developer community by encouraging dozens of NLP vendors to solve standard challenges for text processing and understanding. The winner’s models are then integrated into XiaoAi and Xiaomi’s other smart devices. As for the core mechanism that backs up XiaoAi’s text tasks, the team heavily exploits graph-based algorithms, pushdown automaton, optimal path planning, deterministic finite automation, historical likelihood and context-free grammar models.

AI-powered dispatching

Meituan is the world’s largest online and on-demand delivery platform, with over 200 million users, 18 million daily orders, 1.5 million active local businesses and over half a million “riders” (contractors in charge of delivery). A platform with this size proposes a unique challenge for machine learning and the underlying engineering infrastructure. On the flip side, even a tiny improvement of the baseline model will translate into an immediately better user experience. Jinghua Hao, from Meituan showed us an example on how they use data science to optimize rider dispatch. The ultimate goal is to reduce time wasted at each step, including order placing, route planning, finding businesses, connecting steps, etc. In the early days, Meituan had a purely human-based dispatch system, each dispatcher being responsible for taking and assigning orders within certain areas. Once the business started growing at an exponential rate, Meituan began exploring machine learning algorithms, from genetic algorithms (which can be extremely time-consuming and lead to many false positives) and local search minima, to heuristic search, finally settling on knowledge-based models (that are ultra fast, customizable, and have fewer bad cases). In addition, Meituan developed a simulation platform to help their data scientists run experiments that are otherwise hard to test in the real world, as many parameters are dependent on sequential events. The net result? Meituan’s core models support 10 billion queries a day.

Reinforcement Learning in the real world

Finally, we wanted to highlight a different domain entirely, that has traditionally been more closely associated with research than industry.

Bonsai is building a platform to apply Reinforcement Learning (RL) to real world systems. Mark Hammond, CEO of Bonsai shared details of how its high-level platform abstracts away the low-level implementation details that can make RL so hard to reproduce, thereby making it accessible and tractable. Some of the reasons RL is hard to put into production are that it often requires an accurate simulation of the environment, which is easy for games, but much harder in robotics or manufacturing; and typical RL algorithms require a massive number of training iterations before they converge, making them prohibitively expensive. On top of this, results are often very hard to reproduce. Bonsai’s platform aims to automate some of that work, and bridge the gap between ideation and production. It turns out many manufacturing and optimization problems can be solved using RL, and Mark shared two examples of its platform being used in the real world. One was the optimization of residential HVAC systems so that they are as energy efficient as possible, while maintaining the temperature within a certain range. A common problem, where most simple heuristics end up being very inefficient. The other was an approach to automatically learn correct machine settings to produce well-dimensioned parts in manufacturing plants. As the barrier to using RL keeps getting lower, we could see such algorithms employed in any domain where we are looking to optimize a metric by tuning parameters to create systems that can learn dynamically and adapt to changing conditions.

Looking Forward

The O’Reilly AI conference allowed us to see many use cases of Deep Learning deployed at massive scale and for novel applications. We were very impressed by the quality of the engineering work and research the teams presented and are excited to bring back some of the things we’ve learned to Insight.

We hope you enjoyed these highlights as much as we did. To keep up to date on projects and events related to Insight, follow us on Twitter or Medium.