Since 2010, the annual ImageNet Large-Scale Visual Recognition Challenge has been the most widely recognized benchmark for testing image recognition algorithms. Tencent Machine Learning picks up the challenge with its new paper Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes.

Due to continuous efforts from researchers around the world, ImageNet’s Top-5 error rate dropped from around 28 percent in 2010 to 15.4 percent in 2012 (AlexNet), and finally to approximately 3 percent in 2017 — eclipsing the human rate of about 5 percent.

Through this iterative process, two neural network models were developed which proved very effective for image recognition tasks: AlexNet and ResNet-50. Although the trained model can usually achieve high accuracy, the training process is very time-consuming. For example, it takes 14 days to train ResNet-50 on ImageNet using an NVIDIA M40 GPU, a process that would take decades using only a single-core CPU with a serial program. Accordingly, how to train AlexNet and ResNet-50 on ImageNet in a shorter period of time has become a focal point for researchers.

The team from Tencent Machine Learning (腾讯机智, Jizhi) trained ResNet-50 in 6.6 minutes and AlexNet in just 4 minutes on the ImageNet dataset. The previous record time for training ResNet-50 was 15 minutes (by Preferred Network’s Chainer team); while training AlexNet had been achieved in 11 minutes by a team of UC Berkeley scholars.

Training time of ResNet-50 and AlexNet on ImageNet by different team

Training settings and performance of ResNet-50 on ImageNet by different teams

Training settings and performance of AlexNet on ImageNet by different teams, “–” indicates no data available

The Tencent Jizhi team says improving training speed on ImageNet is just a small part of their work in AI development, and plan to leverage their accelerated training capability in other AI businesses and services, such as game AI.

On July 31st, Tencent’s AI Go program “Jueyi” (绝艺, FineArt) swept its opponent “Golaxy”(星阵) 6–0 to win the World AI Go Competition, which featured 11 AI Go teams from China, Japan, South Korea, Belgium, and the US.

The Tencent Jizhi Machine Learning Platform was established by the Tencent Engineering Group’s Architecture Platform Department and Operations Management Team, and collaborates closely with Professor Xiaowen Chu’s team from the Computer Science Department at Hong Kong Baptist University.

The paper related to this study is now on Arxiv.