LitePipeline：分布式管道系统
litepipeline的Python项目详细描述
莱特管道
一个轻量级、可扩展的分布式管道系统，基于Python3、tornado、venv-pack、pyinstaller。在
所有基于Python3的代码，不要使用Python2！在
它还在开发中，所以，可能有一些错误或不够稳定！在
您可以将LitePipeline与LiteDFS一起使用，LiteDFS是一种分布式文件系统，基于Python3，tornado，灵感来自HDFS。在
请参阅https://github.com/fiefdx/LiteDFS上的LiteDFS
更多详细信息请访问https://github.com/fiefdx/LitePipeline
安装
# install from pip $ pip3 install -U litepipeline # or install from source code $ cd ./litepipeline $ python3 ./setup.py install # this will install 5 commands: liteconfig, litemanager, litenode, litepipeline, liteviewer# liteconfig: to generate manager's or node's configuration file# litemanager: to start LitePipeline manager# litenode: to start LitePipeline node# litepipeline: command line tool to communicate with LitePipeline cluster# liteviewer: the web UI service, for communicate with LitePipeline cluster with web browser.
跑
运行管理器
^{pr2}$运行节点
# create node's data directory $ mkdir ./node_data # generate node's configuration file $ cd ./node_data # this will generate a configuration.yml file under ./node_data $ liteconfig -s node -o ./ # run node# after start node, node will register to manager, and get a unique node id $ litenode -c ./configuration.yml # test $ curl localhost:8001 # return this message{"message": "LitePipeline node service"}
运行查看器
# create viewer's data directory $ mkdir ./viewer_data # generate viewer's configuration file $ cd ./viewer_data # this will generate a configuration.yml file and other scripts under ./viewer_data $ liteconfig -s viewer -o ./ # run viewer $ liteviewer -c ./configuration.yml # test# use web browser open: http://localhost:8088
使用Web浏览器与litepineline集群通信
使用web浏览器在liteviewer的配置文件中打开其主机和端口
使用命令行与LitePipeline群集通信
$ litepipeline localhost:8000 cluster info # | node_id | http_host | http_port | action_slots | app_path | data_path 1|32313239-e7ee-4f90-8c05-4e08fb48be70 |127.0.0.1 |8001|2| /usr/local/lib/python3.7/dist-packages/litepipeline/node | /home/pi/Develop/litenode/data # use -h/--help parameter to see help message $ litepipeline --help usage: litepipeline [-h][-W COLUMN_WIDTH][-v] address {app,app_history,task,cluster,workspace,workflow,work,schedule,service} ... positional arguments: address manager address, host:port {app,app_history,task,cluster,workspace,workflow,work,schedule,service} sub-command help app operate with app API app_history operate with app_history API task operate with task API cluster operate with cluster API workspace operate with workspace API workflow operate with workflow API work operate with work API schedule operate with schedule API service operate with service API optional arguments: -h, --help show this help message and exit -W COLUMN_WIDTH, --column_width COLUMN_WIDTH column max width -v, --version show program's version number and exit$ litepipeline localhost:8000 app --helpusage: litepipeline address app [-h] {create,delete,update,list,info,download} ...positional arguments: {create,delete,update,list,info,download} sub-command app help create create application delete delete application update update application list list applications info application's info download download application optional arguments: -h, --help show this help message and exit $ litepipeline localhost:8000 task --help usage: litepipeline address task [-h]{create,delete,list,info,rerun,recover,stop} ... positional arguments: {create,delete,list,info,rerun,recover,stop} sub-command task help create create task delete delete task list list tasks info task's info rerun rerun task recover recover task stop stop taskoptional arguments: -h, --help show this help message and exit$ litepipeline localhost:8000 cluster --helpusage: litepipeline address cluster [-h] {info} ...positional arguments: {info} sub-command cluster help info cluster's info optional arguments: -h, --help show this help message and exit
- 项目
标签：
欢迎加入QQ群-->： 979659372
