Opencv Python 简明教程

OpenCV Python - Meanshift and Camshift

在本章中，我们一起来了解一下 OpenCV-Python 中的均值漂移和 CAMShift。首先，我们来了解一下什么是均值漂移。

In this chapter, let us learn about the meanshift and the camshift in the OpenCV-Python. First, let us understand what is meanshift.

Meanshift

均值漂移算法可以识别数据集中数据点高度集中或集群的位置。该算法会在每个数据点放置一个内核并对它们求和，以做出一个 Kernel Density Estimation (KDE)。

The mean shift algorithm identifies places in the data set with a high concentration of data points, or clusters. The algorithm places a kernel at each data point and sums them together to make a Kernel Density Estimation (KDE).

KDE 会有高和低数据点密度的区域，具有相应的值。均值漂移是一种很有用的方法，可以用来跟踪视频中某个指定对象。

The KDE will have places with a high and low data point density, respectfully. Meanshift is a very useful method to keep the track of a particular object inside a video.

视频中的每个实例都会以该帧的像素分布形式被检查。初始窗口（即目标区域 (ROI) ）通常是正方形或圆形。为此，会通过硬编码指定位置，并标识最高像素分布区域。

Every instance of the video is checked in the form of pixel distribution in that frame. An initial window as region of interest (ROI) is generally a square or a circle. For this, the positions are specified by hardcoding and the area of maximum pixel distribution is identified.

随着视频播放，ROI 窗口会向着最高像素分布区域移动。移动方向取决于我们的跟踪窗口中心与该窗口内所有 k 像素的质心之间的差值。

The ROI window moves towards the region of maximum pixel distribution as the video runs. The direction of movement depends upon the difference between the center of our tracking window and the centroid of all the k-pixels inside that window.

要在 OpenCV 中使用均值漂移，首先要查找我们目标的直方图（其中仅考虑色调），然后可以针对每个帧 反投影其目标，以计算均值漂移。我们还需要提供 ROI 窗口的初始位置。

In order to use Meanshift in OpenCV, first, find the histogram (of which, only Hue is considered) of our target and can back project its target on each frame for calculation of Meanshift. We also need to provide an initial location of the ROI window.

我们反复计算直方图的反投影，并计算均值漂移以获得跟踪窗口的新位置。随后，我们会使用它的尺寸在帧上绘制一个矩形。

We repeatedly calculate the back projection of the histogram and calculate the Meanshift to get the new position of track window. Later on, we draw a rectangle using its dimensions on the frame.

Functions

程序中使用的 OpenCV 函数包括：

The openCV functions used in the program are −

  1. cv.calcBackProject() − Calculates the back projection of a histogram.

  2. cv.meanShift() − Back projection of the object histogram using initial search window and Stop criteria for the iterative search algorithm.

Example

以下是均值漂移的示例程序：

Here is the example program of Meanshift −

import numpy as np
import cv2 as cv

cap = cv.VideoCapture('traffic.mp4')

ret,frame = cap.read()

# dimensions of initial location of window
x, y, w, h = 300, 200, 100, 50
tracker = (x, y, w, h)

region = frame[y:y+h, x:x+w]
hsv_reg = cv.cvtColor(region, cv.COLOR_BGR2HSV)
mask = cv.inRange(hsv_reg, np.array((0., 60.,32.)), np.array((180.,255.,255.)))
reg_hist = cv.calcHist([hsv_reg],[0],mask,[180],[0,180])
cv.normalize(reg_hist,reg_hist,0,255,cv.NORM_MINMAX)

# Setup the termination criteria
criteria = ( cv.TERM_CRITERIA_EPS | cv.TERM_CRITERIA_COUNT, 10, 1 )

while(1):
   ret, frame = cap.read()

   if ret == True:
      hsv = cv.cvtColor(frame, cv.COLOR_BGR2HSV)
      dst = cv.calcBackProject([hsv],[0],reg_hist,[0,180],1)

      # apply meanshift
      ret, tracker = cv.meanShift(dst, tracker, criteria)

      # Draw it on image
      x,y,w,h = tracker
      img = cv.rectangle(frame, (x,y), (x+w,y+h), 255,2)
      cv.imshow('img',img)

      k = cv.waitKey(30) & 0xff
      if k==115:
         cv.imwrite('capture.png', img)
      if k == 27:
         break

在运行程序时，均值漂移算法会将我们的窗口移动到密度最大的新位置。

As the program is run, the Meanshift algorithm moves our window to the new location with maximum density.

Output

以下为移动窗口的快照：

Here’s a snapshot of moving window −

meanshift

Camshift

Meanshift 算法的一个缺点是，跟踪窗口的大小保持不变，与对象到摄像机的距离无关。此外，只有当窗口处于该对象区域内时，窗口才会跟踪该对象。因此，我们必须手动对窗口进行硬编码，并且应该小心进行。

One of the disadvantages of Meanshift algorithm is that the size of the tracking window remains the same irrespective of the object’s distance from the camera. Also, the window will track the object only if it is in the region of that object. So, we must do manual hardcoding of the window and it should be done carefully.

CAMshift（代表 Continuously Adaptive Meanshift ）给出了这些问题的解决方案。一旦 meanshift 收敛，Camshift 算法就会更新窗口的大小，使得跟踪窗口的大小可能会改变，甚至旋转以更好地与跟踪对象的动作相关联。

The solution to these problems is given by CAMshift (stands for Continuously Adaptive Meanshift). Once meanshift converges, the Camshift algorithm updates the size of the window such that the tracking window may change in size or even rotate to better correlate to the movements of the tracked object.

在以下代码中，使用了 camshift() 函数，而不是 meanshift() 函数。

In the following code, instead of meanshift() function, the camshift() function is used.

首先，它使用 meanShift 查找对象中心，然后调整窗口大小并找到最佳旋转。该函数返回对象的位置、大小和方向。使用 polylines() 绘制函数在帧上绘制位置。

First, it finds an object center using meanShift and then adjusts the window size and finds the optimal rotation. The function returns the object position, size, and orientation. The position is drawn on the frame by using polylines() draw function.

Example

在较早程序中，使用 CamShift() 函数，而不是 Meanshift() 函数，如下所示：

Instead of Meanshift() function in earlier program, use CamShift() function as below −

# apply camshift
ret, tracker = cv.CamShift(dst, tracker, criteria)
pts = cv.boxPoints(ret)
pts = np.int0(pts)
img = cv.polylines(frame,[pts],True, 255,2)
cv.imshow('img',img)

Output

以下显示跟踪窗口旋转矩形的修改后的程序的结果快照：

One snapshot of the result of modified program showing rotated rectangle of the tracking window is as follows −

camshift