Following the paths of many individuals at the same time is enormously difficult, even for humans. Previous computer-based efforts to analyze dense crowd movement have focused on tracking one individual at a time in recorded video. But there are problems with that method. First, you have to run the programs over and over again for each person you want to track. Second, the programs tend to identify people in each frame of a video based on appearance—but heads and faces can be hard to distinguish from above, especially in tight crowds and low-resolution video. The new research, which will be published in IEEE Transactions on Pattern Analysis and Machine Intelligence, finds a way to increase both the efficiency and accuracy of tracking a person, enabling a software program to finally follow many people at the same time [DOI: 10.1109/TPAMI.2017.2687462] [DX].

The trick involves predicting where an individual will go next. The researchers wrote a mathematical function that analyzes five factors, based on previous frames of a video, to anticipate where each person will be in the current frame. One is appearance: Which patches of pixels resemble the target from the previous frame? Another is target motion: Where could the target be based on speed and direction? A third is neighbor motion: If the target is obscured, the program guesses on location based on the motion of the person's neighbors. Fourth is spatial proximity: The program won't guess that two people are in the same place, standing on top of one another. And last is grouping: If the program identifies a few people walking in a group, it will assume that they'll retain the same formation.