如何在OpenCV中模板匹配简单的2D形状？

Question

我想检测一个简单的2D棋盘上的所有棋子。问题是，我的代码只检测黑暗方块上的棋子。我需要检测所有白色的棋子。以下是我的设置：

棋盘（full_board.png）：

典当人（wp.png）：

预期输出（检测所有典当）：

实际输出（未检测到所有典当）：

代码：

import cv2
import numpy as np
import imutils

def main():
    img = cv2.imread('full_board.png', 0)   
    # Piece templates:
    img_rgb = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
    img_gray = cv2.cvtColor(img_rgb,cv2.COLOR_BGR2GRAY)

    pawn_white_template = cv2.imread('wp.png', 0)

    cv2.imshow("Template", pawn_white_template)
    cv2.waitKey(0)

    w_pawn_white, h_pawn_white = pawn_white_template.shape[::-1]

    res_pawn_white = cv2.matchTemplate(img_gray,pawn_white_template,cv2.TM_CCOEFF_NORMED)

    threshhold = 0.6
    loc = np.where(res_pawn_white >= threshhold)

    for pt in zip(*loc[::-1]):
        cv2.rectangle(img_rgb,pt,(pt[0]+w_pawn_white, pt[1]+h_pawn_white),(0,255,255),1)

    cv2.imshow('detected',img_rgb)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

更改阈值对我没有帮助。我不知道为什么它只检测暗方棋子。 你知道如何检测所有的棋子吗

Answer 1

如果删除alpha通道，将看到模板的背景为深绿色。因此，它将只匹配暗背景方块。您正在使用alpha读取模板，但在模板匹配中不会使用alpha通道。您需要提取模板的alpha通道作为遮罩，并使用matchTemplate中的遮罩选项？这应该可以解决问题

您似乎还在将输入转换为灰度，但尝试与彩色模板匹配。请注意，您可以在彩色图像上进行模板匹配

以下是不带alpha的模板：

以下是模板中的alpha通道：

见https://docs.opencv.org/4.1.1/df/dfb/group__imgproc__object.html#ga586ebfb0a7fb604b35a23d85391329be

mask Mask of searched template. It must have the same datatype and size with templ. It is not set by default. Currently, only the TM_SQDIFF and TM_CCORR_NORMED methods are supported.

In case of a color image, template summation in the numerator and each sum in the denominator is done over all of the channels and separate mean values are used for each channel. That is, the function can take a color template and a color image. The result will still be a single-channel image, which is easier to analyze.

下面是Python/OpenCV中的示例，其中包含彩色图像和蒙版模板匹配

输入：

模板：

import cv2
import numpy as np

# read chessboard image
img = cv2.imread('chessboard.png')

# read pawn image template
template = cv2.imread('pawn.png', cv2.IMREAD_UNCHANGED)
hh, ww = template.shape[:2]

# extract pawn base image and alpha channel and make alpha 3 channels
pawn = template[:,:,0:3]
alpha = template[:,:,3]
alpha = cv2.merge([alpha,alpha,alpha])

# do masked template matching and save correlation image
correlation = cv2.matchTemplate(img, pawn, cv2.TM_CCORR_NORMED, mask=alpha)

# set threshold and get all matches
threshhold = 0.89
loc = np.where(correlation >= threshhold)

# draw matches 
result = img.copy()
for pt in zip(*loc[::-1]):
    cv2.rectangle(result, pt, (pt[0]+ww, pt[1]+hh), (0,0,255), 1)
    print(pt)

# save results
cv2.imwrite('chessboard_pawn.png', pawn)
cv2.imwrite('chessboard_alpha.png', alpha)
cv2.imwrite('chessboard_matches.jpg', result)  

cv2.imshow('pawn',pawn)
cv2.imshow('alpha',alpha)
cv2.imshow('result',result)
cv2.waitKey(0)
cv2.destroyAllWindows()

不带alpha通道的模板：

提取的alpha通道作为遮罩：

输入时产生的匹配位置：

但请注意，每个位置实际上都有几个邻近的匹配项。所以一个人实际上有太多的匹配项

(83, 1052)
(252, 1052)
(253, 1052)
(254, 1052)
(423, 1052)
(592, 1052)
(593, 1052)
(594, 1052)
(763, 1052)
(932, 1052)
(933, 1052)
(934, 1052)
(1103, 1052)
(1272, 1052)
(1273, 1052)
(1274, 1052)
(82, 1053)
(83, 1053)
(84, 1053)
(252, 1053)
(253, 1053)
(254, 1053)
(422, 1053)
(423, 1053)
(424, 1053)
(592, 1053)
(593, 1053)
(594, 1053)
(762, 1053)
(763, 1053)
(764, 1053)
(932, 1053)
(933, 1053)
(934, 1053)
(1102, 1053)
(1103, 1053)
(1104, 1053)
(1272, 1053)
(1273, 1053)
(1274, 1053)
(82, 1054)
(83, 1054)
(84, 1054)
(252, 1054)
(253, 1054)
(254, 1054)
(422, 1054)
(423, 1054)
(424, 1054)
(592, 1054)
(593, 1054)
(594, 1054)
(762, 1054)
(763, 1054)
(764, 1054)
(932, 1054)
(933, 1054)
(934, 1054)
(1102, 1054)
(1103, 1054)
(1104, 1054)
(1272, 1054)
(1273, 1054)
(1274, 1054)
(82, 1055)
(83, 1055)
(84, 1055)
(252, 1055)
(253, 1055)
(254, 1055)
(422, 1055)
(423, 1055)
(424, 1055)
(592, 1055)
(593, 1055)
(594, 1055)
(762, 1055)
(763, 1055)
(764, 1055)
(932, 1055)
(933, 1055)
(934, 1055)
(1102, 1055)
(1103, 1055)
(1104, 1055)
(1272, 1055)
(1273, 1055)
(1274, 1055)

处理多个匹配的正确方法是在循环中屏蔽相关图像中的每个匹配区域，从而避免附近高于阈值的非峰值匹配

这里有一种方法可以做到这一点

import cv2
import numpy as np
import math

# read chessboard image
img = cv2.imread('chessboard.png')

# read pawn image template
template = cv2.imread('pawn.png', cv2.IMREAD_UNCHANGED)
hh, ww = template.shape[:2]

# extract pawn base image and alpha channel and make alpha 3 channels
pawn = template[:,:,0:3]
alpha = template[:,:,3]
alpha = cv2.merge([alpha,alpha,alpha])


# set threshold
threshold = 0.89

# do masked template matching and save correlation image
corr_img = cv2.matchTemplate(img, pawn, cv2.TM_CCORR_NORMED, mask=alpha)

# search for max score
result = img.copy()
max_val = 1
rad = int(math.sqrt(hh*hh+ww*ww)/4)
while max_val > threshold:

    # find max value of correlation image
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(corr_img)
    print(max_val, max_loc)

    if max_val > threshold:
        # draw match on copy of input
        cv2.rectangle(result, max_loc, (max_loc[0]+ww, max_loc[1]+hh), (0,0,255), 2)

        # write black circle at max_loc in corr_img
        cv2.circle(corr_img, (max_loc), radius=rad, color=0, thickness=cv2.FILLED)

    else:
        break

# save results
cv2.imwrite('chessboard_pawn.png', pawn)
cv2.imwrite('chessboard_alpha.png', alpha)
cv2.imwrite('chessboard_correlation.png', (255*corr_img).clip(0,255).astype(np.uint8))
cv2.imwrite('chessboard_matches2.jpg', result)

cv2.imshow('pawn',pawn)
cv2.imshow('alpha',alpha)
cv2.imshow('result',result)
cv2.waitKey(0)
cv2.destroyAllWindows()

结果匹配：

以下是他们的实际比赛成绩：

0.8956151008605957 (253, 1053)
0.8956151008605957 (593, 1053)
0.8956151008605957 (933, 1053)
0.8956151008605957 (1273, 1053)
0.89393150806427 (83, 1054)
0.89393150806427 (423, 1054)
0.89393150806427 (763, 1054)
0.89393150806427 (1103, 1054)
0.886812150478363 (1128, 1232)

具有圆形遮罩区域的相关图像：