Human Detection and Tracking in Agriculture

NREC developed advanced machine vision techniques for safety around agricultural vehicles. Robotics offers the opportunity to improve efficiency on the farm, but these systems must reliably detect other workers to ensure their safety.

Application

Enabling the full promise of robotics in agriculture requires reliable detection and tracking of human coworkers so that people and machines can effectively and safely perform required tasks. Many agricultural machines are powerful and potentially dangerous, and certain tasks require humans to work closely to these machines. Other applications may need to enforce a safety buffer, and agricultural fields generally have minimal access controls. Even for smaller agricultural robots, it is often important for them to understand where the people in their environment are to effectively complete their tasks. Our previous work resulted in a spatially distributed multi-vehicle system of autonomous tractors that shared task responsibilities with multiple human co-workers to accomplish agricultural operations in a citrus orchard. This system has demonstrated over 2400km of autonomous operation and performed significant useful work at a higher productivity level than current methods. The system includes a sophisticated obstacle detection system, but a key limiting factor was the reliable detection of people when partially occluded by tree branches and weeds or when lying on the ground or in other non-standard poses. [1] [2] [1] Carnegie Mellon University. "Integrated Automation for Sustainable Specialty Crops." http://www.rec.ri.cmu.edu/usda/index.html [2] S. J. Moorehead, C. K. Wellington, B. J. Gilmore, and C. Vallespi, “Automating orchards: A system of autonomous tractors for orchard maintenance,” in IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS) Workshop on Agricultural Robotics, 2012.

Capabilities

The aim of this work was to advance the state of the art in detection and tracking of people in agricultural environments. We benchmarked many current methods in pedestrian detection and developed new ones, and released the dataset below [3,4]. We hope that this common benchmark allows the field to move forward with a spirit of both competition and cooperation. [3] T. Tabor, Z. Pezzementi, C. Vallespi and C. Wellington, 'People in the Weeds: Pedestrian Detection Goes Off-road', in 2015 IEEE International Symposium on Safety, Security, and Rescue Robotics, Purdue University, West Lafayette, IN, 2015. [4] Z. Pezzementi, T. Tabor, P. Hu, J. Chang, D. Ramanan, C. Wellington, B. Babu, and H. Herman. Comparing apples and oranges: Off-road pedestrian detection on the National Robotics Engineering Center agricultural person-detection dataset. J Field Robotics. 2017;00:1–19. https://doi.org/10.1002/rob.21760. arXiv preprint arXiv:1707.07169.



Data Set

Benchmark Results