numpy真除法中遇到溢出

Question

我正试图在python中从头开始使用动量实现梯度下降。在对指数加权平均值进行偏差校正时，我遇到了一条运行时警告：“numpy真除法中遇到溢出”。我正在尝试用一个非常小的浮点数（在代码部分提供）分割一个numpy数组

for i in range(1, len(self.layers)):
            self.dvw['dvw'+str(i)] = (self.beta1*self.dvw['dvw'+str(i)] + (1-self.beta1)*self.dw['dw'+str(i)])
            self.dvb['dvb'+str(i)] = (self.beta1*self.dvb['dvb'+str(i)] + (1-self.beta1)*self.db['db'+str(i)])
            self.dvw['dvw'+str(i)] = self.dvw['dvw'+str(i)].astype('float')
            self.dvb['dvb'+str(i)] = self.dvb['dvb'+str(i)].astype('float')
            self.dvw['dvw'+str(i)] = np.divide(self.dvw['dvw'+str(i)], (1-np.power(self.beta1, t))) # encounterd error in this line
            self.dvb['dvb'+str(i)] = np.divide(self.dvb['dvb'+str(i)], (1-np.power(self.beta1, t))) # and this line aswell

dvw和dvb都是以numpy数组作为值的字典。beta1值为0.9，t为实数

Answer 1

尝试通过添加以下内容来提高代码的数值稳定性：

eps=1e-6
for i in range(1, len(self.layers)):
    self.dvw['dvw'+str(i)] = (self.beta1*self.dvw['dvw'+str(i)] + (1-self.beta1)*self.dw['dw'+str(i)])
    self.dvb['dvb'+str(i)] = (self.beta1*self.dvb['dvb'+str(i)] + (1-self.beta1)*self.db['db'+str(i)])
    self.dvw['dvw'+str(i)] = self.dvw['dvw'+str(i)].astype('float')
    self.dvb['dvb'+str(i)] = self.dvb['dvb'+str(i)].astype('float')
    self.dvw['dvw'+str(i)] = np.divide(self.dvw['dvw'+str(i)], (eps + 1-np.power(self.beta1, t)))
    self.dvb['dvb'+str(i)] = np.divide(self.dvb['dvb'+str(i)], (eps + 1-np.power(self.beta1, t)))