How to install airflow?

Question

I seem to be doing sth. wrong.

https://pythonhosted.org/airflow/start.html

$ export AIRFLOW_HOME=~/airflow
$ pip install apache-airflow
Requirement already satisfied
$ airflow initdb
airflow: Command not found 

python --version
Python 2.7.10

It's weird - the installation seemed to have worked fine (with some warnings - nothing serious) saying: airflow, flask, etc. successfully installed. But even after restarting the PC (Ubuntu 15.10) airflow seems not to be a command.

Answer 1

I tried both pip install apache-airflow and pip3 install apache airflow and both had issues because it installed everything in ~/.local/bin/

If you get the error that you cannot run airflow, you will find it in ~/.local/bin/airflow. Then you can add the alias to your .bashrc: alias airflow='~/.local/bin/airflow' then run bash and you will be able to run airflow.

Then when you try to run the webserver with either the python2 or python3 version it will throw an error because it cannot find gunicorn, and you can fix that by adding ~/.local/bin to the PATH:

export PATH=$PATH:~/.local/bin

Answer 2

I have macOs Mojave. Airflow comes with default sqlite db which doesnt allow parallel processing. Installation is difficult.

brew install autoconf
brew install automake
brew install pkg-config
brew install libtool

Then create a virtual env for python3.7. In venv type :

pip install "apache-airflow[celery, crypto, postgres, rabbitmq, redis]==1.10.6"

Airflow will be generally installed at

‘/Library/Frameworks/Python.framework/Versions/3.7/bin/airflow’

add this path to your path variable as shown here Link

now type following in you virtual env that you created above for airflow.

airflow initdb

If this works you have installed airflow successfully.

Answer 3

Here are the steps I followed to install Airflow:

Set the airflow home in ~/.bashrc

export AIRFLOW_HOME=~/airflow

Install from pypi using pip

pip install airflow

initialising the metadata DB

airflow initdb

starting the webserver

airflow webserver -p 8080

open browser and go to localhost:8080 to view and use UI

Answer 4

The solution the worked for was to create an environment, install airflow and then was able to run it.

-> Install virtualenv: $pip install virtualenv

-> Create environment: $python -m venv myvenv

-> Activate environment: $source myenv/bin/activate

-> Install airflow : (myenv)$pip install airflow with postgres: pip install airflow[postgres]

->Start the server: (myenv)$airflow webserver -p 8080

Answer 5

This seems like the path to airflow is not in your PATH. does this happen with other python packages?

try:

export PATH=$PATH:/usr/local/bin/

this is the default path for airflow and should make it work

Answer 6

Your steps look correct, if you haven't omitted anything else. But you could try Python virtualenv and virtualenvwrapper with following steps to have an isolated airflow environment.

pip install virtualenv
pip install virtualenvwrapper
# update and source your .profile
mkvirtualenv airflow
workon airflow
export AIRFLOW_VERSION=1.7.0
pip install airflow==${AIRFLOW_VERSION}
# optionally other modules
#pip install airflow[celery]==${AIRFLOW_VERSION}