keras.model.fit_生成器如何迭代验证_数据？

Question

我在用Keras软件包做神经网络。现在，我正在构建用于预测问题的NN。我在google colabolatory上训练我的模型，其中RAM限制为12 GB。
为了解决RAM的问题，我正在使用model.fit_generator功能，这里我的问题开始了。因为一张图片超过1000字，所以我创建了这个图表：

详情：

• 资料

df = pd.DataFrame({'x': [i for i in range(50)], 'y': [i**2 for i in range(50)]})

• 常数

num_rows = len(df) # 50
validate_size = 1
train_size = 4
batch_size = validate_size + train_size # 5
steps_per_epoch = 1
epochs = num_rows // batch_size // steps_per_epoch # 10

• 发电机

def validate_generator(df, validate_size, train_size):
    batch_size = validate_size + train_size
    for i in range(0, len(df), batch_size):
        X = df[i + train_size:i + batch_size, 0]
        y = df[i + train_size:i + batch_size, 1]
        yield (X.reshape(-1, 1), y.reshape(-1, 1))
        print('\n\nval', X, '\n')
        

def train_generator(df, validate_size, train_size):
    batch_size = validate_size + train_size
    for i in range(0, len(df), batch_size):
        X = df[i:i + train_size, 0]
        y = df[i:i + train_size, 1]
        yield (X.reshape(-1, 1), y.reshape(-1, 1))
        print('\n\ntrain', X)

• keras model.fit_发电机功能

model.fit_generator(
    train_generator(df, validate_size, train_size),
    steps_per_epoch=steps_per_epoch,
    validation_data=validate_generator(df, validate_size, train_size),
    validation_steps=steps_per_epoch,
    epochs=epochs,
)

培训过程日志：

Epoch 1/10
train [0 1 2 3]
1/1 [==============================] - ETA: 0s - loss: 17.7931 - mae: 2.8934
val [4]
1/1 [==============================] - 0s 105ms/step - loss: 17.7931 - mae: 2.8934 - val_loss: 205.8619 - val_mae: 14.3479

Epoch 2/10
train [5 6 7 8]
1/1 [==============================] - ETA: 0s - loss: 1865.1838 - mae: 40.8180
val [9]
val [14]
1/1 [==============================] - 0s 37ms/step - loss: 1865.1838 - mae: 40.8180 - val_loss: 36154.7812 - val_mae: 190.1441

Epoch 3/10
train [10 11 12 13]
1/1 [==============================] - ETA: 0s - loss: 17199.1582 - mae: 128.6884
val [19]
val [24]
1/1 [==============================] - 0s 32ms/step - loss: 17199.1582 - mae: 128.6884 - val_loss: 320162.5000 - val_mae: 565.8290

Epoch 4/10
train [15 16 17 18]
1/1 [==============================] - ETA: 0s - loss: 72351.9141 - mae: 266.5040
val [29]
val [34]
1/1 [==============================] - 0s 40ms/step - loss: 72351.9141 - mae: 266.5040 - val_loss: 1302790.6250 - val_mae: 1141.3986

Epoch 5/10
train [20 21 22 23]
 1/1 [==============================] - ETA: 0s - loss: 208619.7188 - mae: 454.2613
val [39]
val [44]
1/1 [==============================] - 0s 36ms/step - loss: 208619.7188 - mae: 454.2613 - val_loss: 3674302.5000 - val_mae: 1916.8470

Epoch 6/10
train [25 26 27 28]
1/1 [==============================] - ETA: 0s - loss: 482259.9688 - mae: 691.9575
val [49]
WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least `steps_per_epoch * epochs` batches (in this case, 1 batches). You may need to use the repeat() function when building your dataset.
1/1 [==============================] - 0s 44ms/step - loss: 482259.9688 - mae: 691.9575

Epoch 7/10
train [30 31 32 33]
1/1 [==============================] - 0s 18ms/step - loss: 964487.8750 - mae: 979.5891

Epoch 8/10
train [35 36 37 38]
1/1 [==============================] - 0s 25ms/step - loss: 1741474.2500 - mae: 1317.1534

Epoch 9/10
train [40 41 42 43]
1/1 [==============================] - 0s 16ms/step - loss: 2914341.0000 - mae: 1704.6472

Epoch 10/10
train [45 46 47 48]
1/1 [==============================] - 0s 16ms/step - loss: 4599159.5000 - mae: 2142.0679

我对validate_generator有问题。您可以在培训日志中看到，在6个历元之后，有一个关于validation_data的警告——没有更多的数据可以迭代。我认为这个警告是因为在以前的历元中validate_generator每个历元产生两倍的数据。
链接到colab:https://colab.research.google.com/drive/1INO5UfPDr_7Jvx6xM4FVBYkGnLG6IFUf?usp=sharing