After flying this past weekend (together with Gabriel and Leandro) with Gabriel’s drone (which is an handmade APM 2.6 based quadcopter) in our town (Porto Alegre, Brasil), I decided to implement a tracking for objects using OpenCV and Python and check how the results would be using simple and fast methods like Meanshift. The result was very impressive and I believe that there is plenty of room for optimization, but the algorithm is now able to run in real time using Python with good results and with a Full HD resolution of 1920×1080 and 30 fps.

Here is the video of the flight that was piloted by Gabriel:

See it in Full HD for more details.

The algorithm can be described as follows and it is very simple (less than 50 lines of Python) and straightforward:

A ROI (Region of Interest) is defined, in this case the building that I want to track

The normalized histogram and back-projection are calculated

The Meanshift algorithm is used to track the ROI

The entire code for the tracking is described below:

import numpy as np import cv2 def run_main(): cap = cv2.VideoCapture('upabove.mp4') # Read the first frame of the video ret, frame = cap.read() # Set the ROI (Region of Interest). Actually, this is a # rectangle of the building that we're tracking c,r,w,h = 900,650,70,70 track_window = (c,r,w,h) # Create mask and normalized histogram roi = frame[r:r+h, c:c+w] hsv_roi = cv2.cvtColor(roi, cv2.COLOR_BGR2HSV) mask = cv2.inRange(hsv_roi, np.array((0., 30.,32.)), np.array((180.,255.,255.))) roi_hist = cv2.calcHist([hsv_roi], [0], mask, [180], [0, 180]) cv2.normalize(roi_hist, roi_hist, 0, 255, cv2.NORM_MINMAX) term_crit = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 80, 1) while True: ret, frame = cap.read() hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV) dst = cv2.calcBackProject([hsv], [0], roi_hist, [0,180], 1) ret, track_window = cv2.meanShift(dst, track_window, term_crit) x,y,w,h = track_window cv2.rectangle(frame, (x,y), (x+w,y+h), 255, 2) cv2.putText(frame, 'Tracked', (x-25,y-10), cv2.FONT_HERSHEY_SIMPLEX, 1, (255,255,255), 2, cv2.CV_AA) cv2.imshow('Tracking', frame) if cv2.waitKey(1) & 0xFF == ord('q'): break cap.release() cv2.destroyAllWindows() if __name__ == "__main__": run_main()

I hope you liked it !