Video-guided Camera Control for Target Tracking and Following


Journal Article

© 2019 This paper considers the problem of controlling a nonholonomic mobile ground robot equipped with an onboard camera characterized by a bounded field-of-view, tasked with detecting and following a potentially moving human target using onboard computing and video processing in real time. Computer vision algorithms have been recently shown highly effective at object detection and classification in images obtained by vision sensors. Existing methods typically assume a stationary camera and/or use pre-recorded image sequences that do not provide a causal relationship with future images. The control method developed in this paper seeks to improve the performance of the computer vision algorithms, by planning the robot/camera trajectory relative to the moving target based on the desired size and position of the target in the image plane, without the need to estimate the target's range. The method is tested and validated using a highly realistic and interactive game programming environment, known as Unreal Engine™, that allows for closed-loop simulations of the robot-camera system. Results are further validated through physical experiments using a Clearpath™ Jackal robot equipped with a camera which is capable of following a human target for long time periods. Both simulation and experimental results show that the proposed vision-based controller is capable of stabilizing the target object size and position in the image plane for extended periods of time.

Full Text

Duke Authors

Cited Authors

  • Gemerek, J; Ferrari, S; Wang, BH; Campbell, ME

Published Date

  • January 1, 2019

Published In

Volume / Issue

  • 51 / 34

Start / End Page

  • 176 - 183

Electronic International Standard Serial Number (EISSN)

  • 2405-8963

Digital Object Identifier (DOI)

  • 10.1016/j.ifacol.2019.01.062

Citation Source

  • Scopus