使用SGD而不使用sklearn（LogLoss随历元增加）

Question

def train(X_train,y_train,X_test,y_test,epochs,alpha,eta0):
    w,b = initialize_weights(X_train[0])
    loss_test=[]
    N=len(X_train)
    for i in range(0,epochs):
        print(i)
        for j in range(N-1):
            grad_dw=gradient_dw(X_train[j],y_train[j],w,b,alpha,N)
            grad_db=gradient_db(X_train[j],y_train[j],w,b)
            w=np.array(w)+(alpha*(np.array(grad_dw)))
            b=b+(alpha*(grad_db))                
               predict2 = []
    for m in range(len(y_test)):
        z=np.dot(w[0],X_test[m])+b
        if sigmoid(z) == 0: # sigmoid(w,x,b) returns 1/(1+exp(-(dot(x,w)+b)))
            predict2.append(0.000001)
        elif sigmoid(z) == 1:
            predict2.append(0.99999)
        else:
            predict2.append(sigmoid(z)) 
            
    loss_test.append(logloss(y_test,predict2))       
    return w,b,loss_test

我的梯度dw函数

def gradient_dw(x,y,w,b,alpha,N):
    dw=[]
    for i in range(len(x)):
        dw.append((x[i]*(y-1/(1+np.exp(abs(w.T[0][i]*x[i]+b)))))+(alpha/N)*(w.T[0][i]))
    return dw

My gradient db函数：

 def gradient_db(x,y,w,b):
        db=0
        for i in range(len(x)):
            db=(y-1/(1+np.exp(abs(w.T[0][i]*x[i]+b))))
        return db

我的损失函数：

def logloss(y_true,y_pred):
    loss=0
    for i in range(len(y_true)):
        loss+=((y_true[i]*math.log10(y_pred[i]))+((1-y_true[i])*math.log10(1-y_pred[i])))
    loss=-1*(1/len(y_true))*loss
    return loss

我的问题是，每个时代过后，我的损失都在增加。为什么？

任何帮助都将不胜感激

谢谢你

Answer 1

1. The problem was of weight function

2. i was taking weight array as of dim(15,1)

3. but it should be (15)

4. So all the changes need to be done according with it in this code

5. Thank You