Python Script not running in cron

Question

I am trying to run a Python script from cron. I am using crontab to run the command as a user instead of root. My Python script has the shebang at the top #! /usr/bin/env python and I did chmod +x it to make the script executable.

The script does work when running it from a shell but not when using crontab. I did check the /var/log/cron file and saw that the script runs but that absolutely nothing from stdout or stderr prints anywhere.

Finally, I made a small script that prints the date and a string and called that every minute that worked but this other script does not. I am not sure why I am getting these variable results...

Here is my entry in crontab

SHELL=/bin/bash



#min    #hour   day-of-month    month   day-of-week     command
#-------------------------------------------------------------------------
*/5     *       *                *        *             /path/to/script/script.py

Here is the source code of my script that will not run from crontab but will run from the shell it is in when called like so ./script.py. The script is executable after I use the chmod +x command on it, by the way...:

#! /usr/bin/env python

#create a file and write to it
import time

def create_and_write():
    with open("py-write-out.out", "a+") as w:
        w.write("writing to file. hello from python. :D\n")
        w.write("the time is: " + str(time.asctime()) + "\n")
        w.write("--------------------------------\n")


def main():
    create_and_write()

if __name__ == "__main__":
    main()

EDIT I figured out what was going wrong. I needed to put the absolute file paths in the script otherwise it would write to some directory I had not planned for.

Answer 1

Another reason may be that the way we judged executed or not executed is wrong.

Answer 2

Ok, I guess this thread is still going to help googlers.
I use a workaround to run python scripts with cron jobs. In fact, python scripts need to be handled with delicate care with cron job.
So I let a bash script take care of all of it.
I create a bash script which has the command to run the python script. Then I schedule a cron job to run bash script. You can even use a redirector to log the output of bash command executing the python script.
For example

@reboot /home/user/auto_delete.sh

auto_delete.sh may contain following lines:-

#!/bin/sh
echo $(date) >> bash_cron_log.txt
/usr/bin/python /home/user/auto_delete.py >> bash_cron_log.txt

So I don' need to worry about Cron Jobs crashing on python scripts.

Answer 3

I had a similar issue but with a little bit different scenario: I had a bash script (run.sh) and a Python script (hello.py). When I typed

sh run.sh

from the script directory, it worked; if I added the sh run.sh command in my crontab, it did not work.

To troubleshoot the issue I added the following line in my run.sh script:

printenv > env.txt

In this way you are able to see the environment variable used when you (or the crontab) run the script. By analyzing the differences between the env.txt generated from the manual run and the crontab run I noticed that the PWD variable was different. In my case I was able to resolve by adding the following line at the top of my .sh script:

cd /PYTHON_SCRIPT_ABSOLUTE_PATH/

Hope this could help you!

Answer 4

You can resolve this yourself by following these steps:

1. Modify the cron as /path/to/script/script.py > /tmp/log 2> &1
2. Now, let the cron run
3. Now read the file /tmp/log
4. You will find out the reason of the issue you are facing, so that you can fix it.

In my experience, the issue is mostly with the environment.
In cron, the env variables are not set. So you may have to explicitly set the env for your script in cron.

Answer 5

I would like to emphasise one more thing. I was trying to run a python script in cron using the same trick (using shell script) and this time it didn't run. It was @reboot cron. So I used redirection in crontab as I mentioned in one of the above comments. And understood the problem. I was using lot many file handlers in python script and cron runs the script from user's home directory. In that case python could not find the files used for file handlers and will crash.

What I used for troubleshooting was that I created a crontab as below. It would run the start.sh and throw all the stderror or stdoutput to file status.txt and I got error in status.txt saying that the file I used for file handler was not found. That was right because python script was executed by cron from user's home directory and then the script starts searching for files in home directory only.

@reboot /var/www/html/start.sh > /cronStatus/status.txt 2>&1

This will write everything happening during cron execution to status.txt file. You can see the error over there. I will again advice running python scripts using bash scripts for cronjobs.

SOLUTION:- Either you use full path for all the files being used in script (which wasn't feasible for me, since I don't want script to be location dependent). Or you execute script from correct directory So I created my cron as below-

@reboot cd /var/www/html/ && /var/www/html/start.sh

This cron will first change directory to correct location and then start the script. Now I don't have to worry about hardcoding full path for all files in script. Yeah, it may sound being lazy though ;)
And my start.sh looks like-

#!/bin/sh
/usr/bin/python /var/www/html/script.py

Hope it helps
Regards,
Kriss