自动扫描文档图像增强器

Question

我正在开发基于微软论文Whiteboard scanning and image enhancement的自动图像增强

在“白平衡和图像增强”部分，他们提供了增强的步骤：

首先：他们估计扫描文档或检测到的白板的背景：

1. "Divide the whiteboard region into rectangular cells. The cell size should be roughly the same as what we expect the size of a single character on the board (15 by 15 pixels in our implementation)."

那么

2. "Sort the pixels in each cell by their luminance values. Since the ink absorbs the incident light, the luminance of the whiteboard pixels is higher than stroke pixels’. The whiteboard color within the cell is, therefore, the color with the highest luminance. In practice, we average the colors of the pixels in the top 25 percentile in order to reduce the error introduced by sensor noise"

那么

3. "Filter the colors of the cells by locally fitting a plane in the RGB space. Occasionally there are cells that are entirely covered by pen strokes, the cell color computed in Step 2 is consequently incorrect. Those colors are rejected as outliers by the locally fitted plane and are replaced by the interpolated values from its neighbors."

我的问题是第二步和第三步：

如何获得亮度值，我应该将输入图像转换为YUV颜色空间，并从Y通道获得亮度值，还是只处理RGB颜色空间？

如何在RGB空间中拟合局部平面？

这是我的python代码，我试图从输入图像生成单元格，从YUV颜色空间中获取亮度值，以及一个简单的结果，与他们在论文中得到的结果相比似乎不正确。

Python代码：

import cv2
import numpy as np



## Return List of cells from a given Image
def SubImage(image):
    Cells = []
    CellRows = []
    for i in range(0,rows/CellSize):
        subIm = image[i*CellSize:(i+1)*CellSize,:]
        CellRows.append(subIm)
    for img in CellRows:
        for i in range(0,cols/CellSize):
            subIm = img[:,i*CellSize:(i+1)*CellSize]
            Cells.append(subIm)
    return Cells


## Sort luminosity Value
def GetLuminance(Cells):
    luminance = []
    for cel in Cells:
        luminance.append(cel.max())
    return luminance


## Estimate the background color of the white board
def UniformBackground(CelImage,img,luminance):
    a = 0

    for c in range(0,len(CelImage)):
        cel = CelImage[c]
        for i in range(0,cel.shape[0]):
            for j in range(0, cel.shape[1]):
                cel[i,j] = min(1,cel[i,j]/ luminance[c])
    for i in range(0,rows/CellSize):
        for j in range(0,cols/CellSize):
            img[i*CellSize:(i+1)*CellSize,j*CellSize:(j+1)*CellSize] = CelImage[a]
            a = a + 1

if __name__ == '__main__':
    img = cv2.imread('4.png')
    CellSize = 15
    rows,cols,depth = img.shape


    if (rows%CellSize !=0):
        rows = rows - rows%CellSize

    if (cols%CellSize !=0):
        cols = cols - cols%CellSize

    yuvImg = cv2.cvtColor(img, cv2.COLOR_BGR2YUV)
    # Get cells from Y channel
    CellsY = SubImage(yuvImg[:,:,0])
    CellsB = SubImage(img[:,:,0])
    CellsG = SubImage(img[:,:,1])
    CellsR = SubImage(img[:,:,2])

    # Get Luminance From Y cells
    LuminanceY = GetLuminance(CellsY)

    # Uniform Background
    UniformBackground(CellsB, img[:,:,0], LuminanceY)
    UniformBackground(CellsG, img[:,:,1], LuminanceY)
    UniformBackground(CellsR,img[:,:,2], LuminanceY)

    #bgrImg = cv2.cvtColor(imgB, cv2.COLOR_GRAY2BGR)
    #print imgB
    cv2.imwrite('unifrom.jpg',img)

输入白板图像：

输出图像：

预期输出：

Answer 1

temp = cel[i,j]/luminance[c]
if temp > thresh : ##Let thresh be 0.7
   cel[i,j] = 255 

亮度值越大的Cel被转换为白色，其他Cel保持原样。The output of the image with uniform background

Answer 2

让我们一步一步来：

2. "Sort the pixels in each cell by their luminance values"

是的，您必须将图像转换为其他具有亮度分量的颜色空间，例如Lab color space。在

... In practice, we average the colors of the pixels in the top 25 percentile in order to reduce the error introduced by sensor noise

也就是说，在你得到实验室图像后，你需要把它分成几个通道，L通道图像，取它的柱状图，比如说有100个箱子（我夸张了），只取那些在最白的箱子里的像素（比如从75到100）。现在，在你找到每个单元格中的白色像素后-记住它们！！！例如，您可以创建一个蒙版图像，除了那些被选为“白色”的像素外，所有像素都是0

Filter the colors of the cells by locally fitting a plane in the RGB space

现在回到RBG空间。如你所见，白板随着时间的推移越来越暗。 如果您将白板像素RGB颜色绘制为轴为R、G和B的3d世界中的3d点，您将得到近似于平面的散射（因为所有这些白板颜色都带有灰色色调）。现在，获取在上一步中标记为“白板”的点，并为它们安装一个平面。 如何安装飞机？你可以使用像this这样的最小二乘法，但从他们在文章中的写法来看，我认为他们已经考虑到了兰萨克。在