命令行电池(CLB)-在不编写太难的bash代码的情况下改进bash命令功能的方法。
clb的Python项目详细描述
命令行电池(CLB)
插件驱动的python程序,在不编写太难的bash代码的情况下改进bash命令的功能。
BASH对于备份流水线(如{{CD1>})来说是好的,但是没有容易的方法来收集流水线中的每个命令的退出代码,关心命令环境,检查超时,通知监控系统,编写令人敬畏的日志,发送通知到SLACK,上传备份到AWS S3等等。有了命令行电池,所有这些东西都可以用了。只需填写yaml配置文件并根据需要更改python脚本文件。这些文件设计得尽可能简单。
特点:
- 每个shell命令的超时时间 检查每个shell命令的退出代码,包括管道中的所有命令<
- shell命令的路径环境变量
- 信息日志
- 松弛通知
- 通过influxDB指标进行监控
- 带有过期策略的aws s3上载
用法
基本用法示例:
example_simple_script.py-cexample_simple_config.yml
2019-05-06 13:38:29,930 INFO in clb.config_parser: Reading config file: example_simple_config.yml
2019-05-06 13:38:29,933 INFO in clb.shell_commands: Running shell command: mkdir -p /tmp/backups
2019-05-06 13:38:29,943 INFO in clb.shell_commands: Shell command success
2019-05-06 13:38:29,944 INFO in clb.shell_commands: Running shell command: echo 'THE BACKUP BASH COMMAND' | pigz -p 4 -4 > /tmp/backups/current_backup.gz
2019-05-06 13:38:29,963 INFO in clb.shell_commands: Shell command success
2019-05-06 13:38:29,964 INFO in clb: Commands completed
高级用法示例:
example_script.py-cexample_config.yml
2019-05-30 02:02:59,815 INFO in clb.config_parser: Reading config file: /opt/command_line_batteries/example_config.yml
2019-05-30 02:02:59,822 INFO in clb.shell_commands: Running shell command: mkdir -p /tmp/backups
2019-05-30 02:02:59,833 INFO in clb.shell_commands: Shell command success
2019-05-30 02:02:59,834 INFO in clb.shell_commands: Running shell command: echo 'THE BACKUP BASH COMMAND' | pigz -p 4 -4 > /tmp/backups/current_backup.gz
2019-05-30 02:02:59,846 INFO in clb.shell_commands: Shell command success
2019-05-30 02:02:59,847 INFO in clb: Backup files created successfully
2019-05-30 02:02:59,847 INFO in clb.notifiers.slack_client: Sending message "Backup files created successfully" to channel: YOUR_SLACK_CHANNEL
2019-05-30 02:03:00,343 INFO in clb.notifiers.influxdb_client: Adding point to InfluxDB: {'measurement': 'backups', 'tags': {'status': 'Backup files created', 'host': 'YOUR_HOST'}, 'fields': {'value': 0}}
2019-05-30 02:03:00,420 INFO in clb.storages.aws_s3: Upload /tmp/backups/current_backup.gz to s3/YOUR_AWS_BUCKET_FOR_BACKUPS/YOUR_HOST/2019-05-30-current_backup.gz
2019-05-30 02:03:01,156 INFO in clb.storages.aws_s3: Update bucket YOUR_AWS_BUCKET_FOR_BACKUPS lifecycle rules
2019-05-30 02:03:02,184 INFO in clb: Upload to s3 completed successfully
2019-05-30 02:03:02,184 INFO in clb.notifiers.slack_client: Sending message "Upload to s3 completed successfully" to channel: YOUR_SLACK_CHANNEL
2019-05-30 02:03:02,394 INFO in clb.notifiers.influxdb_client: Adding point to InfluxDB: {'measurement': 'backups', 'tags': {'status': 'Upload to s3 completed', 'host': 'YOUR_HOST'}, 'fields': {'value': 0}}
2019-05-30 02:03:02,408 INFO in clb: Backup completed successfully
2019-05-30 02:03:02,408 INFO in clb.notifiers.slack_client: Sending message "Backup completed successfully" to channel: YOUR_SLACK_CHANNEL
2019-05-30 02:03:02,894 INFO in clb.notifiers.influxdb_client: Adding point to InfluxDB: {'measurement': 'backups', 'tags': {'status': 'Backup completed', 'host': 'YOUR_HOST'}, 'fields': {'value': 0}}
Grafana可视化示例:
安装
最简单的方法:
下载并运行安装脚本(python2/3):
$ wget -O /tmp/install_clb.py https://raw.githubusercontent.com/sharkman-devops/command_line_batteries/master/install_clb.py
$ sudo python /tmp/install_clb.py
创建cron作业(crontab-e):
51 23 * * * /opt/command_line_batteries/venv/bin/python /opt/command_line_batteries/example_script.py -c /opt/command_line_batteries/example_config.yml >> /var/log/command_line_batteries/example.log 2>&1
别忘了填写example_config.yml并检查example_script.py以获取所需的逻辑!
更灵活的方式:
创建安装目录:
mkdir -p /opt/command_line_batteries
创建虚拟环境:
python3 -m venv /opt/command_line_batteries/venv
激活虚拟环境:
source /opt/command_line_batteries/venv/bin/activate
安装命令行电池:
pip install clb --upgrade
下载示例脚本和配置:
wget -O /opt/command_line_batteries/example_script.py https://raw.githubusercontent.com/sharkman-devops/command_line_batteries/master/example_script.py
wget -O /opt/command_line_batteries/example_config.yml https://raw.githubusercontent.com/sharkman-devops/command_line_batteries/master/example_config.yml
使示例脚本可执行:
chmod +x /opt/command_line_batteries/example_script.py
创建日志目录:
mkdir -p /var/log/command_line_batteries
创建cron作业(crontab-e):
51 23 * * * /opt/command_line_batteries/venv/bin/python /opt/command_line_batteries/example_script.py -c /opt/command_line_batteries/example_config.yml >> /var/log/command_line_batteries/example.log 2>&1
别忘了填写example_config.yml并检查example_script.py以获取所需的逻辑!
兼容性
环境应包括:
- 击壳
- awk
- python 3.6或3.7
在gnu-linux上使用gnu-bash 4、gnu-awk 4和python 3.6&;3.7测试命令行电池