在conda环境中运行气流,且系统D无法启动

2024-10-06 12:02:04 发布

您现在位置:Python中文网/ 问答频道 /正文

我想通过conda安装气流,并使用systemd在Ubuntu上控制气流。我已经能够通过以下步骤成功地将airflow安装到conda环境中,但是我无法正确地配置systemd以使用气流。在

详细信息:

  • Ubuntu 18.04版
  • Python 3.7.4
  • 气流1.10.4

我见过SO #52310217,但问题仍然没有解决。在

以下是从命令行安装和运行airflow的步骤:

$ conda create --name airflow -c conda-forge airflow
$ conda activate airflow
$ export AIRFLOW_HOME=~/airflow
$ airflow initdb

$ airflow scheduler
$ airflow webserver

从这里开始,我在/etc/systemd/system/中创建了以下文件:


^{pr2}$
# /etc/systemd/system/airflow-webserver.service
[Unit]
Description=Airflow webserver daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
Environment="PATH=/home/ubuntu/python/envs/airflow/bin"
User=airflow
Group=airflow
Type=simple
ExecStart=/home/curtis/miniconda3/envs/airflow/bin/airflow webserver -p 8085 --pid /home/curtis/airflow/airflow-webserver.pid
Restart=on-failure
RestartSec=5s
PrivateTmp=true

[Install]
WantedBy=multi-user.target

在创建了上面的文件之后,我发出了以下命令,但是什么都没有发生。在

sudo systemctl daemon-reload
sudo systemctl enable airflow-scheduler
sudo systemctl start airflow-scheduler
sudo systemctl enable airflow-webserver
sudo systemctl start airflow-webserver

如果我直接在终端窗口中输入ExecStart命令,airflow scheduler和airflow Web服务器将按预期启动。出于这个原因,我不认为这是气流的问题,而是我在systemd配置中缺少的东西。在

我不知道该从中得到什么,但这里是气流调度程序和气流Web服务器的状态。在


[curtis:~/airflow/logs/scheduler] [airflow] 4 $ sudo systemctl status airflow-scheduler
● airflow-scheduler.service - Airflow scheduler daemon
   Loaded: loaded (/etc/systemd/system/airflow-scheduler.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Mon 2019-08-19 22:36:35 PDT; 3s ago
  Process: 5623 ExecStart=/home/curtis/miniconda3/envs/airflow/bin/airflow scheduler (code=exited, status=217/USER)
 Main PID: 5623 (code=exited, status=217/USER)

[curtis:~/airflow/logs/scheduler] [airflow] 3 $ sudo systemctl status airflow-webserver
● airflow-webserver.service - Airflow webserver daemon
   Loaded: loaded (/etc/systemd/system/airflow-webserver.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Mon 2019-08-19 22:36:46 PDT; 895ms ago
  Process: 5805 ExecStart=/home/curtis/miniconda3/envs/airflow/bin/airflow webserver -p 8085 --pid /home/curtis/airflow/airflow-webserver
 Main PID: 5805 (code=exited, status=217/USER)


Tags: homestatusserviceetcsudocodesystemconda
1条回答
网友
1楼 · 发布于 2024-10-06 12:02:04

我发现了这个问题,并想把这个贴在这里给其他遇到同样问题的人。在

  • User应该是您当前的用户名,或者安装Airflow的用户名。对我来说那是“柯蒂斯”
  • Group应该是您当前的组,或者安装气流的用户组。对我来说那是“柯蒂斯”
  • Environment是到conda环境的路径。在我的例子中,那是PATH=/home/curtis/miniconda3/envs/airflow/bin

我已经包括了我更正的systemd文件供你参考。在


# /etc/systemd/system/airflow-scheduler.service
[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
Environment="PATH=/home/curtis/miniconda3/envs/airflow/bin"
User=curtis
Group=curtis
Type=simple
ExecStart=/home/curtis/miniconda3/envs/airflow/bin/airflow scheduler
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target

^{pr2}$

相关问题 更多 >