How can I restart the airflow server on Google Com

Question

When I need to restart the webserver locally I do:

ps -ef | grep airflow | awk '{print $2}' | xargs kill -9
airflow webserver -p 8080 -D

How can I do this on Google Composer? I don't see an option to restart the server in the console.

Answer 1

A way to restart Composer server is to add a 'dummy variable' into the 'Environment Variables' of GCP Composer UI. After submitting, it will restart to include this change.

Answer 2

Since Cloud Composer is an Apache Airflow managed service, it is not possible to restart the whole service. You can restart though the single instances of the service, as described here, but this will not help to apply to the plugin changes.

To apply the plugin changes, you should install the plugin after loading it into the plugin folder, automatically created when you created the environment:

gcloud composer environments storage plugins import --environment ENVIRONMENT_NAME \
    --location LOCATION \
    --source PATH_TO_LOCAL_FILE \
    --destination PATH_IN_SUBFOLDER

DAG updates are performed automatically and some other must be applied using

gcloud composer environments update

command.