深入学习keras模型生命周期管理备份/恢复nano框架
sizif的Python项目详细描述
DL备份/恢复nano框架
深度学习模型的自动备份/还原模型快照:
- 到/从本地文件系统
- 到/从远程ftp服务器
当前版本仅支持Keras>;=2.2型号。欢迎您投稿。
用法
pip3 install sizif
ftp keras检查点备份/还原:
fromsizif.kerasimportKerasModelWrapperfromsizif.storageimportFTPFileCheckpointsMonitor# your compiled Keras Model instancemodel=build_model()# Local filesystem snapshots monitor with FTP backup/restore # Different model architectures should have different version parameter# other parameters similar to Keras ModelCheckpoint# See sizif.storage.FileCheckpointsMonitor for local file only backup/restore cpm=FTPFileCheckpointsMonitor(1,'weights.{epoch:03d}-vl{val_loss:.3f}-va{val_acc:.3f}.hdf5',local_folder='/snapshots_local_dir',remote_folder='/snapshots_ftp_dir',host='ftp.your-host.com',login='your_ftp_login',password='your_ftp_password',die_on_ftperrors=True,rotate_number=3,monitor='val_loss',verbose=1,save_best_only=False,save_weights_only=True,mode='auto',period=1)# Keras wrapper, proxies all calls to the model# except `fit` and `fit_generator` — which are surrounded # by automated model state backup/recovery km=KerasModelWrapper(model,cpm)# all method parameters are proxied to Keras as is except callbacks# callbacks are extended with `cpm` listener km.fit_generator(training_set_generator,epochs=25,validation_data=test_set_generator,callbacks=[tboard])
有关详细的文档字符串,请参见来源
待办事项:
- ssh/s3/dropbox上传监视器
- TensorFlow/Pythorch型号支持
测试
python3 -m unittest
依赖关系
- 纽比~>;1.15
- 凯拉斯~>;2.2
许可证
这个项目是在麻省理工学院的许可下发布的。