二维最大子阵函数在Keras中的实现

Question

我试图在Keras（Tensorflow后端）中实现一个自定义的loss函数。你知道吗

我的目标是创建一个损失函数，取yèpred的大小（150，200，1）（即150x200的图像，有一个通道），取它和相应的张量yèu之间的差为真，然后扫描得到的“差”数组，寻找所有可能维度的子数组，产生一个具有最大绝对值的和（一个2D max子数组问题）。然后，函数应该输出该子数组之和的绝对值作为损失（浮点）。（我试图用本文中的“MESA”算法来模拟这个函数：https://www.robots.ox.ac.uk/~vgg/publications/2010/Lempitsky10b/lempitsky10b.pdf）

我一直在尝试阅读Keras中的自定义损失函数，我理解必须在Keras函数空间中编写损失函数。虽然我目前有一个Cython优化版的损失函数，但我不太知道如何将它转换成Keras友好的版本。我的损失函数的主要基础代码如下所示。你知道吗

#The loss function as defined in my code
def MESA(y_true, y_pred):
    diff = y_true - y_pred
    diff = K.eval(diff)
    result = CythonMESA.MaxSubArray2D(diff)
    result = np.array([result])
    result = K.variable(result)
    return result

model.compile(
    loss=MESA,
    optimizer='adam',
    metrics=['accuracy']
)

“CythonMESA”模块包含一些Cython优化的函数，我在下面附上了这些函数。具体来说，就是CythonMESA.MaxSubArray2D函数将2D数组作为输入（例如2D数组）np.N阵列对象）并输出一个double。你知道吗

#Contents of CythonMESA.pyx

import numpy as np
cimport cython

@cython.boundscheck(False)
@cython.wraparound(False)
@cython.cdivision(True)

#a helper function that is called within the main function below
#this function computes the maximum sum subarray in a 1D array using Kadane's algorithm
cdef double KadaneAbsoluteValue(double [:] array):
    cdef int length = int(array.shape[0])
    cdef double[:] maxSums = np.zeros(length, np.float64)
    cdef double kadaneMax
    cdef int i
    for i in range(length):
        if i == 0:
            maxSums[0] = array[0]
            kadaneMax = abs(maxSums[0])
        else:
            if abs(array[i]) >= abs(array[i] + maxSums[i-1]):
                maxSums[i] = array[i]
            else:
                maxSums[i] = array[i] + maxSums[i-1]
            if abs(maxSums[i]) > kadaneMax:
                kadaneMax = abs(maxSums[i])
    return kadaneMax

#The main basis for the loss function
#Loops through a 2D array and uses the function above to compute maximum subarray
cpdef double MaxSubArray2D(double [:,:] array):
    cdef double maxSum = 0.
    cdef double currentSum
    cdef int height = int(array.shape[0])
    cdef int width = int(array.shape[1])
    cdef int i, j
    cdef double [:] tempArray
    if height >= width:
        for i in range(width):
            for j in range(i,width):
                tempArray = np.sum(array[:,i:j+1], axis=1)
                currentSum = KadaneAbsoluteValue(tempArray)
                if currentSum > maxSum:
                    maxSum = currentSum
    else:
        for i in range(height):
            for j in range(i, height):
                tempArray = np.sum(array[i:j+1,:], axis=0)
                currentSum = KadaneAbsoluteValue(tempArray)
                if currentSum > maxSum:
                    maxSum = currentSum
    return maxSum

实际上，我已经尝试直接使用上述函数在Keras中编译一个网络，但是正如预期的那样，它抛出了一个错误。你知道吗

如果有人能给我指出正确的方向，我可以找到如何把这个转换成一个Keras友好的功能，等等，我会非常感激！你知道吗

Answer 1

一个简单的卷积1个过滤器与所有的紧跟着一个maxpooling可以做到这一点。你知道吗

subArrayX = 3
subArrayY = 3
inputChannels = 1
outputChannels = 1
convFilter = K.ones((subArrayX, subArrayY, inputChannels, outputChannels))

def local_loss(true, pred):

    diff = K.abs(true-pred) #you might also try K.square instead of abs

    localSums = K.conv2d(diff, convFilter)
    localSums = K.batch_flatten(localSums) 
        #if using more than 1 channel, you might want a different thing here

    return K.max(localSums, axis=-1)


model.compile(loss = local_loss, ....)

对于所有可能的形状：

convWeights = []
for i in range(1, maxWidth+1):
    for j in range(1, maxHeight+1):
        convWeights.append(K.ones((i,j,1,1)))

def custom_loss(true,pred):

    diff = true - pred

    #sums for each array size
    sums = [K.conv2d(diff, w) for w in convWeights]

    # I didn't understand if you want the max abs sum or abs of max sum
    # add this line depending on the answer:
    sums = [K.abs(s) for s in sums] 

    #get the max sum for each array size
    sums = [K.batch_flatten(s) for s in sums]
    sums = [K.max(s, axis=-1) for s in sums]

    #global sums for all sizes
    sums = K.stack(sums, axis=-1)
    sums = K.max(sums, axis=-1)

    return K.abs(sums)

尝试类似于卡丹的东西（分开维度）

让我们从不同的维度来做：

if height >= width:
    convFilters1 = [K.ones((1, i, 1, 1)) for i in range(1,width+1)]
    convFilters2 = [K.ones((i, 1, 1, 1) for i in range(1,height+1)]
    concatDim1 = 2
    concatDim2 = 1
else:
    convFilters1 = [K.ones((i, 1, 1, 1)) for i in range(1,height+1)]
    convFilters2 = [K.ones((1, i, 1, 1) for i in range(1,width+1)]
    concatDim1 = 1
    concatDim2 = 2


def custom_loss_2_step(true,pred):
    diff = true-pred #shape (samp, h, w, 1)

    sums = [K.conv2d(diff, f) for f in convFilters1] #(samp, h, var, 1) 
                                                     #(samp, var, w, 1)    
    sums = K.concatenate(sums, axis=concatDim1) #(samp, h, superW, 1)
                                                #(samp, superH, w, 1)
    sums = [K.conv2d(sums, f) for f in convFilters2] #(samp, var, superW, 1)
                                                     #(samp, superH, var, 1)
    sums = K.concatenate(sums, axis=concatDim2) #(samp, superH, superW, 1)
    sums = K.batch_flatten(sums) #(samp, allSums)

    #??? sums = K.abs(sums)
    maxSum = K.max(sums, axis-1) #(samp,)
    #??? maxSum = K.abs(maxSum)

    return maxSum