python中openCV的多线程图像处理

Question

我对python非常陌生，在并行化算法的一部分时遇到了问题。 考虑一个输入图像，它需要在像素级以某种方式设置阈值。由于算法只考虑一个特定的区域来计算阈值，所以我想在一个单独的线程/进程中运行图像的每个块。这就是我被困的地方。我找不到这些线程在同一个图像上工作的方法，也找不到如何将结果合并到新图像中。 因为我通常来自java世界，所以我通常会解决我不想干涉其他线程的问题。因此，我只是试图通过每个进程的形象。你知道吗

def thresholding(img):
    stepSize = int(img.shape[0] / 10)
    futures = []
    with ProcessPoolExecutor(max_workers=4) as e:
        for y in range(0, img.shape[0], stepSize):
            for x in range(0, img.shape[1], stepSize):
                futures.append(e.submit(thresholdThread, y, x, img))
    concurrent.futures.wait(futures)
    return img


def thresholdThread(y, x, img):
    window_size = int(img.shape[0] / 10)
    window_shape = (window_size, window_size)
    window = img[y:y + window_shape[1], x:x + window_shape[0]]
    upper_bound, lower_bound, avg = getThresholdBounds(window, 0.6)

    for y_2 in range(0, window.shape[0]):
        for x_2 in range(0, window.shape[1]):
            tmp = img[y + y_2, x + x_2]
            img[y + y_2, x + x_2] = tmp if (tmp >= upper_bound or tmp <= lower_bound) else avg
    return str(avg)

就我对python的理解而言，这是行不通的，因为每个进程都有自己的img副本。但是由于img是numpy的floats类型ndarray，我不知道是否以及如何使用所描述的共享对象here。你知道吗

仅供参考：我使用的是python3.6.9。我知道3.7已经发布了，但是重新安装所有的东西以便我可以使用spyder和openCV并不是那么容易。你知道吗

Answer 1

您没有利用任何Numpy的矢量化技术，这些技术可以显著减少处理时间。我假设这就是为什么你想在窗口/图像块上进行多进程操作-我不知道Docker是什么，所以我不知道这是否是多进程方法的一个因素。你知道吗

这里是一个矢量化的解决方案，警告可能会从操作中排除底部和右边缘像素。如果不能接受，无需进一步阅读。你知道吗

示例中的右和下边缘窗口的大小很可能与其他窗口不同。看起来你任意选择了一个因子10来分割你的图像-如果10是一个任意的选择，你可以很容易地优化底部和右边缘的三角洲-我会在答案的末尾发布这个函数。你知道吗

图像需要重塑为面片，以矢量化操作。我使用了^{} functionsklearn.feature_extraction.image._extract_patches，因为它很方便，并且允许创建不重叠的补丁（这似乎是您想要的）。注意下划线前缀-它以前是一个公开的函数image.extract_patches，但现在已经被弃用了。这个函数使用numpy.lib.stride_tricks.as_strided—可能只是reshape数组，但我没有尝试过。你知道吗

设置

import numpy as np
from sklearn.feature_extraction import image
img = np.arange(4864*3546*3).reshape(4864,3546,3)
# all shape dimensions in the following example derived from img's shape

定义面片大小（参见下面的opt_size）并重塑图像。你知道吗

hsize, h_remainder, h_windows = opt_size(img.shape[0])
wsize, w_remainder, w_windows = opt_size(img.shape[1])

# rgb - not designed for rgba
if img.ndim == 3:
    patch_shape = (hsize,wsize,img.shape[-1])
else:
    patch_shape = (hsize,wsize)

patches = image._extract_patches(img,patch_shape=patch_shape,
                                 extraction_step=patch_shape)
patches = patches.squeeze()

patches是原始数组的一个视图，它将在原始数组中看到更改。它的形状是(8, 9, 608, 394, 3)有8x9，(608,394,3)个窗口/补丁。你知道吗

找到每个面片的上下界；将每个像素与其面片的边界进行比较；为每个在其边界之间并且需要更改的像素提取索引。你知道吗

lower = patches.min((2,3)) * .6
lower = lower[...,None,None,:]
upper = patches.max((2,3)) * .6
upper = upper[...,None,None,:]
indices = np.logical_and(patches > lower, patches < upper).nonzero()

找到每个面片的平均值，然后更改所需的像素值

avg = patches.mean((2,3))    # shape (8,9,3)
patches[indices] = avg[indices[0],indices[1],indices[-1]]

---

将所有功能组合在一起的函数

def g(img, opt_shape=False):
    original_shape = img.shape

    # determine patch shape   
    if opt_shape:
        hsize, h_remainder, h_windows = opt_size(img.shape[0])
        wsize, w_remainder, w_windows = opt_size(img.shape[1])
    else:
        patch_size = img.shape[0] // 10
        hsize, wsize = patch_size,patch_size

    # constraint checking here(?) for
    #     number of windows,
    #     orphaned pixels

    if img.ndim == 3:
        patch_shape = (hsize,wsize,img.shape[-1])
    else:
        patch_shape = (hsize,wsize)

    patches = image._extract_patches(img,patch_shape=patch_shape,
                                     extraction_step=patch_shape)
    #squeeze??
    patches = patches.squeeze()

    #assume color (h,w,3)
    lower = patches.min((2,3)) * .6
    lower = lower[...,None,None,:]
    upper = patches.max((2,3)) * .6
    upper = upper[...,None,None,:]
    indices = np.logical_and(patches > lower, patches < upper).nonzero()

    avg = patches.mean((2,3))
##    del lower, upper, mask
    patches[indices] = avg[indices[0],indices[1],indices[-1]]

---

def opt_size(size):
    '''Maximize number of windows, minimize loss at the edge

    size -> int
       Number of "windows" constrained to 4-10
       Returns (int,int,int)
           size in pixels,
           loss in pixels,
           number of windows
    '''

    size = [(divmod(size,n),n) for n in range(4,11)]
    n_windows = 0
    remainder = 99
    patch_size = 0
    for ((p,r),n) in size:
        if r <= remainder and n > n_windows:
            remainder = r
            n_windows = n
            patch_size = p
    return patch_size, remainder, n_windows

---

测试你的天真过程-我希望我执行正确。在4864x3546彩色图像的基础上改进了35倍。可能还有进一步的优化，也许一些向导会对此发表评论。你知道吗

使用块系数10进行测试：

#yours
def f(img):
    window_size = int(img.shape[0] / 10)
    window_shape = (window_size, window_size)

    for y in range(0, img.shape[0], window_size):
        for x in range(0, img.shape[1], window_size):

            window = img[y:y + window_shape[1], x:x + window_shape[0]]
            upper_bound = window.max((0,1)) * .6
            lower_bound = window.min((0,1)) * .6
            avg = window.mean((0,1))

            for y_2 in range(0, window.shape[0]):
                for x_2 in range(0, window.shape[1]):
                    tmp = img[y + y_2, x + x_2]
                    indices = np.logical_and(tmp < upper_bound,tmp > lower_bound)
                    tmp[indices] = avg[indices]


img0 = np.arange(4864*3546*3).reshape(4864,3546,3)
#get everything the same shape
size = img0.shape[0] // 10
h,w = size*10, size * (img0.shape[1]//size)
img1 = img0[:h,:w].copy()
img2 = img1.copy()

assert np.all(np.logical_and(img1==img2,img2==img0[:h,:w]))
f(img1)    # ~44 seconds
g(img2)    # ~1.2 seconds
assert(np.all(img1==img2))
if not np.all(img2==img0[:h,:w]):
    pass
else:
    raise Exception('did not change')

---

indices是一个index array。它是数组的元组，每个维度一个。indices[0][0],indices[1][0],indices[2][0]将是3d数组中一个元素的索引。完整元组可用于索引数组的多个元素。你知道吗

>>> indices
(array([1, 0, 2]), array([1, 0, 0]), array([1, 1, 1]))
>>> list(zip(*indices))
[(1, 1, 1), (0, 0, 1), (2, 0, 1)]
>>> arr = np.arange(27).reshape(3,3,3)
>>> arr[1,1,1], arr[0,0,1],arr[2,0,2]
(13, 1, 20)
>>> arr[indices]
array([13,  1, 19])

# arr[indices] <-> np.array([arr[1,1,1],arr[0,0,1],arr[2,0,1]])

np.logical_and(patches > lower, patches < upper)返回布尔数组，^{}返回值为True的所有元素的索引。你知道吗