没有这样的文件或目录/airflow/xcom/return.json(No such file o

2019-10-29 08:18发布

创建映像包含/airflow/xcom/return.json使用chmod + X上的所有子DIR由于日志显示无法找到文件或目录(尝试使用chmod + x)的

strtpodbefore = KubernetesPodOperator(namespace='rguonew',
                                    image="mydockerimage",
                                    name="fail",
                                    image_pull_policy="Always",
                                    task_id="failing-task",
                                    get_logs= True,
                                    xcom_push=True,
                                    dag=dag
                                    )'

这里是日志

[2019-03-18 20:32:07,007] {logging_mixin.py:95} INFO - [2019-03-18 
 20:32:07,007] {pod_launcher.py:166} INFO - Running command... cat 
/airflow/xcom/return.json
[2019-03-18 20:32:07,007] {logging_mixin.py:95} INFO - 
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - [2019-03-18 
20:32:07,026] {pod_launcher.py:173} INFO - cat: can't open 
 '/airflow/xcom/return.json': No such file or directory
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - 
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - [2019-03-18 
 20:32:07,026] {pod_launcher.py:166} INFO - Running command... kill -s 
 SIGINT 1
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - 
[2019-03-18 20:32:07,067] {models.py:1788} ERROR - Pod Launching 
failed: Failed to extract xcom from pod: fail-e18e3dac

因此,试图用这种方式,它的工作原理,但它意味着其分配XCOM JSON从外部而不是从图像

 return_value = '{"foo": "bar"\n, "buzz": 2}'
 strtpodbefore = KubernetesPodOperator(namespace='rguonew',
                                image="python:3.6.6-stretch",
                                cmds=["bash", "-cx"],
                                name="fail",
                                task_id="failing-task",
                                arguments=['echo \'{}\' > 
 /airflow/xcom/return.json'.format(return_value)],
                                get_logs= True,
                                xcom_push=True,
                                dag=dag
                                )

所以我试图用做一个额外的参数,但仍然不工作的最终解决方案,第一个命令没有得到这样的回报目录

strtpodbefore = KubernetesPodOperator(namespace='rguonew',
                                    image="myimages",
                                    name="fail",
                                    image_pull_policy="Always",
                                    cmds=["bash", "-cx"],
                                    arguments=['echo \'{}\' > 
/airflow/xcom/return.json'.format(return_value)],
                                    task_id="failing-task",
                                    get_logs= True,
                                    xcom_push=True,
                                    dag=dag
                                    )

Answer 1:

当您设置xcom_push =真,一个三轮容器将与您执行人容器一起创建。 边三轮容器将读取执行的/airflow/xcom/return.json ,所以你实际上需要为你在第二个例子做了从执行容器写这个。

在这里看到: https://stackoverflow.com/a/53568710/10675601

当我们在我们的荚XCOM工作,也有相关的RBAC,你需要设置一些问题: 气流K8S操作XCOM -握手身份403禁止



文章来源: No such file or directory /airflow/xcom/return.json