多波段混合使接缝更加明亮和可见

Question

我正在尝试使用多波段混合将两个预扭曲的图像无缝地缝合在一起。我有两个输入图像（已经扭曲）和一个遮罩。但是，当我应用MBB时，接缝周围的区域会发光更亮，因此，它们变得更可见，这与此处的目标相反。我完全不知道我做错了什么

为了更好地解释问题，以下是图像和输出：

目标：

资料来源：

掩码：

一旦我将源图像混合到目标中，我就会得到：

以下是我的代码供参考：

import cv2 as cv2
import numpy as np
import sys

def blend(A, B, m, canvas, num_levels=6):
    trimmer = cv2.bitwise_or(canvas, m) # to trim the blurry edges around the image after blending
    m[m == 255] = 1

    GA = A.copy()
    GB = B.copy()
    GM = m.copy()

    gpA = [GA]
    gpB = [GB]
    gpM = [GM]

    for i in range(num_levels):
        GA = cv2.pyrDown(GA)
        GB = cv2.pyrDown(GB)
        GM = cv2.pyrDown(GM)

        gpA.append(np.float32(GA))
        gpB.append(np.float32(GB))
        gpM.append(np.float32(GM))

    lpA = [gpA[num_levels - 1]]
    lpB = [gpB[num_levels - 1]]
    gpMr = [gpM[num_levels - 1]]

    for i in range(num_levels - 1, 0, -1):
        size = (gpA[i - 1].shape[1], gpA[i - 1].shape[0])

        LA = np.subtract(gpA[i - 1], cv2.pyrUp(gpA[i], dstsize=size))
        LB = np.subtract(gpB[i - 1], cv2.pyrUp(gpB[i], dstsize=size))

        lpA.append(LA)
        lpB.append(LB)

        gpMr.append(gpM[i - 1])

    LS = []
    for la, lb, gm in zip(lpA, lpB, gpMr):
        ls = la * gm + lb * (1.0 - gm)
        # ls = la + lb
        LS.append(ls)

    ls_ = LS[0]
    for i in range(1, num_levels):
        size = (LS[i].shape[1], LS[i].shape[0])
        ls_ = cv2.add(cv2.pyrUp(ls_, dstsize=size), np.float32(LS[i]))
        ls_[ls_ > 255] = 255; ls_[ls_ < 0] = 0
    
    ls_ = ls_.astype(np.uint8)

    cv2.imwrite("trimmer.jpg", trimmer)
    ls_ = cv2.bitwise_and(ls_, trimmer)

    return ls_

传递给函数的画布（基本上是目标/马赛克的掩码）：

源/新映像的掩码：

我也愿意探索其他方法来无缝融合这两幅图像，以防MBB不是实现我目标的最合适的方法。请帮忙

Answer 1

这里是C++答案，但是算法很容易。

int main()
{
    std::string folder = "C:/Development/Projects/UNDIST_FISHEYE/OpenCV4_Experiments_VS2017/";
    cv::Mat mosaic_img = cv::imread(folder + "mosaic_img.jpg");
    cv::Mat newImage_img = cv::imread(folder + "newImage_img.jpg");

    //cv::Mat mosaic_mask = cv::imread(folder + "mosaic_mask.jpg", cv::IMREAD_GRAYSCALE);
    cv::Mat mosaic_mask = cv::imread(folder + "mosaic_mask_2.jpg", cv::IMREAD_GRAYSCALE);
    mosaic_mask = mosaic_mask > 230; // threshold because of jpeg artifacts

    cv::Mat newImage_mask_raw = cv::imread(folder + "newImage_mask.jpg", cv::IMREAD_GRAYSCALE);
    newImage_mask_raw = newImage_mask_raw > 230;
    // newImage_mask_raw is a few pixels too small...
    cv::Mat newImage_mask = cv::Mat::zeros(mosaic_mask.size(), mosaic_mask.type());

    newImage_mask_raw.copyTo(newImage_mask(cv::Rect(0,0, newImage_mask_raw.cols, newImage_mask_raw.rows)));

    cv::Mat mosaic_blending = cv::Mat::zeros(mosaic_mask.size(), CV_32FC1);
    cv::distanceTransform(mosaic_mask, mosaic_blending, cv::DIST_L2, cv::DIST_MASK_PRECISE);
    cv::Mat newImage_blending = cv::Mat::zeros(mosaic_mask.size(), CV_32FC1);
    cv::distanceTransform(newImage_mask, newImage_blending, cv::DIST_L2, cv::DIST_MASK_PRECISE);

    cv::imshow("mosaic blending", mosaic_blending/255);
    cv::imshow("newImage blending", newImage_blending/255);

    cv::Mat newMosaic = mosaic_img.clone();
    // now compose the mosaic:
    // for each pixel: mosaic=(m1*p1 + m2*p2)/(m1+m2)
    for (int y = 0; y < newMosaic.rows; ++y)
    {
        for (int x = 0; x < newMosaic.cols; ++x)
        {
            // for efficiency: only process pixels where the new image hits the mosaic canvas
            if (newImage_blending.at<float>(y, x) == 0) continue;

            float m1 = newImage_blending.at<float>(y, x);
            float m2 = mosaic_blending.at<float>(y, x);
            float ma = m1 + m2;

            m1 = m1 / ma;
            m2 = m2 / ma;

            cv::Vec3f mosaicPixel = m1 * newImage_img.at<cv::Vec3b>(y, x) + m2 * mosaic_img.at<cv::Vec3b>(y, x);
            newMosaic.at<cv::Vec3b>(y, x) = mosaicPixel; // maybe cast or round here
        }

    
    }

    cv::imwrite("mask1.png", mosaic_mask);
    cv::imwrite("mask2.png", newImage_mask);
    cv::imwrite("mosaic.jpg", newMosaic);

    cv::imshow("mosaic", newMosaic);


    cv::waitKey(0);
}

一般的想法是测量从遮罩边界到内部的距离，并假设边界处的像素质量较低（更有可能导致接缝），因此这些像素的混合应该更强

如果在将遮罩扭曲到马赛克画布之前测量（甚至预计算）此距离，这可能会更好

使用这些面具时

我得到这个结果：

正如您所看到的，仍然有一个接缝，但它来自中间马赛克（输入图像之一），如果先前的缝合是使用相同的混合执行的，那么它将不存在

然后将此遮罩用于中间马赛克（告知已经给定的接缝具有低像素质量）

我得到这个结果：

最好是使用前一个混合遮罩和新图像混合遮罩的每像素最大值来合成马赛克混合遮罩