I'm trying to run a python script on my raspberrypi using cron.
I did the following:
crontab -e # To edit a crontab job
After the cron file opened, I added the following line:
#reboot /usr/bin/python /home/pi/path/to/file/example.py > /home/pi/cronlogs/mylog.log # JOB_ID_!
If I understand the documentation correctly, this cron job should be executed every time after the system booted up. However in my case, when I reboot the computer, the script will not be executed.
What's strange:
I checked the log file and it's empty, so it seems like everything goes fine
If I run the given command manually (so basically write the following code to the terminal) it executes and works correctly: /usr/bin/python /home/pi/path/to/file/example.py > /home/pi/cronlogs/mylog.log
I guess I missed something really obvious but I can't see it. Please can I ask for any advise how to debug this. Thanks!
The cron definition looks correct; I just checked this on my Pi running Debian stretch and it works OK:
#reboot /usr/bin/python /home/pi/example.py > /home/pi/mylog.log
Some other possible reasons it might not work:
working directory issue (if you're using relative paths)
a long running script (being a scraping script it might take a while to complete) - you can check if it's still running using ps aux | grep python
the script does not output anything (would need some more details about the script)
Just to be sure you catch any errors from the script, redirect stderr to stdout by using 2>&1
Related
I use a RaspberryPi 3 with UbuntuMate 16.04. On it, I want to start a little Python (3.5) program every midnight. For that I call a little shell script, so that I can change into the wanted directory comfortably.
crontab:
5 0 * * * /path/to/script/start.sh
start.sh (yes, it's executable):
#!/bin/bash
cd /path/to/wanted/workingDir/
python3.5 ControllerQueue.py
#also tried: python3.5 ControllerQueue.py &
Now if I execute the programm or script from the terminal, everything runs fine. But if I use the crontab it starts the script and stops right after. I also tried running the programm directly but get the same outcome. The paths are correct as I copied the workingDir path from the crontab-file and started it via the terminal.
Is there something I overlook?
As stofvl suggested, I saved the error output of my shell script. It turns out that I needed to add a display. My programm is divided into two scripts. One which provides a GUI and the other main application. The script only starts the main application, without the GUI, but it seems that this didn't matter.
This discussion helped me solve the problem.
I am using dryscrape in a python script. The python script is called in a bash script, which is run by cron. For those who may not be aware, dryscrape is a headless browser (use QtWebkit in the background - so requires an xsession).
Here are the main points concerning the issue I'm having
When I run the python script from the command line, it works
When I run the bash script from the command line, it works too
I figured out that this may have something to do with different environments between my command prompt and when the cron job is running, so I modified my bash script to source my .profile as follows:
#/bin/bash
. /full/path/to/my/home/directory/.profile
python script_to_run.py
This is what my cronjob crontab entry looks like:
0,55 14-22 * * 1-5 /path/to/script.sh >> $(date "+/path/to/logs/\%Y\%m\%d.mydownload.log" )
By the way, I know that the job is being run (I can see entries in /var/log/syslog, and the script also writes to a log file - which is where I get the error message below):
In all cases, I got the following error message:
Could not connect to X server. Try calling dryscrape.start_xvfb()
before creating a session
I have installed the prerequisites, on my machine (obviously - since it runs at the command line). At the moment, I have run out of ideas.
What is causing the script to run fine at the console, and then fail when run by cron?
[[Relevant Details]]
OS: Linux 16.0.4 LTS
bash: version 4.3.46(1)
cron user: myself (i.e. same user at the command prompt)
dryscrape: version 1.0.1
The solution to this was to call the dryscrape.start_xvfb() method before starting the dryscrape session.
Cron user does not have display, so you cannot run any command which requires a display.
You need to modify the python script to do not use any type of display (check carefully, because some python commands, even though they do not open any display , they internally check for this variable).
The best way to test is to ssh into the machine without Display, and check if you can run it from there without erros.
I want to run a python script at boot of Lubuntu 15.04. This python script writes some string into a text file placed in /home/aUser/aFile.txt
My /etc/rc.local file is:
#!/bin/sh -e
python /home/aUser/theScript.py &
exit 0
And the script /home/aUser/theScript.py is:
#!/usr/bin/python
f = open('/home/aUser/aFile.txt','w');
f.write("Some string...");
f.close();
Actually the python script does more, and run an infinite loop, this is why I run the script in background with &. Of course I have python installed:
~$ python --version
Python 2.7.9
I checked if /etc/rc.local is called at boot, and it is, proof of that: I added a test into the /etc/rc.local in this way:
#!/bin/sh -e
python /home/aUser/theScript.py &
exit 0
echo "Test" >> /home/aUser/aTest.txt
and the file /home/aUser/aTest.txt is written/created at boot.
So everything looks correct, proof of that:
if I run manually ~$ /etc/rc.local the file aFile.txt is correctly written.
Instead if I start (or reboot) the OS, the file is not written at boot.
I suspect that could be a problem of permissions/user: I know that /etc/rc.local is run as root, but even if I set root or aUser as owner of the file, the situation is the same. Also run the python script in the /etc/rc.local as user aUser (with su command) does not solve the problem.
Ok I found the problem and fix it, thanks to the #Zac comment.
Actually the python script try to open a network connection before writing the file: at boot time, when the python script is run from /etc/rc.local (so, it is run), the network is still not ready (probably because it is a wireless network) and therefore an exception is raised and the entire script stops. Capturing the exception solves the problem.
So at the end it was my fault, (not) helped by the rc.local that does not provide an easy way to debug.
I have a python program that uses the ThreadPool for multithreading. The program is one step in a shell script. When I execute the shell script manually on the command line, the entire flow works as expected. However, when I execute the shell script as a cronjob, it appears that the flow goes to the next steps before the python multithreading steps are completely finished.
Inside the python program, I do call AsyncResult.get(timeout) to wait for all the results to come back before moving on.
Run your program via batch(1) (see the output of the command man batch) as well. If that works OK, but the cron version does not, then it is almost certainly a problem with your environment variable setup. To verify that, run printenv from your interactive shell to inspect your environment there. Then do the same thing inside the crontab (you will just need to temporarily set up an extra cron entry for it). Try setting the variables in your shell script before invoking Python.
On the other hand, if it doesn't work via batch(1) either, it could be something to do with the files that your code has open. Try running your shell script with input redirected from /dev/null and output going to a file:
$ /usr/local/bin/myscript </dev/null >|/tmp/outfile.txt 2>&1
Try setting "TERM=xterm" (or whatever env variable you have, figure out by command 'env' on your terminal) in your crontab.
Okay so I have a bash script that simply downloads a web page, and then I use python to pulls some data out of the downloaded page.
So my bash script is along the lines of
#!/bin/bash
html_file="web_page.html"
wget -O /home/michael/Documents/CS288/homework7/web_page.html http://markets.usatoday.com/custom/usatoday-com/html-mktscreener.asp?exchange=13\&screen=1
python hw_7_2.py $html_file
Now, when I just execute this bash script from the command line it runs fine, the wget runs and then my python script executes, however when I set it up as a cron job the wget will run but the python script never executes.
I have not really set up cron jobs so this I think may be the issue. This is basically what my crontab file looks like
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
* * * * * michael /home/michael/Documents/CS288/homework7/usatoday_runner.sh
try to replace the cron line with :
* * * * * michael /home/michael/Documents/CS288/homework7/usatoday_runner.sh > /tmp/why_is_this_failing.log 2>&1
the answer may be in the /tmp/why_is_this_failing.log
It's possible that your script doesn't have some environment variables set. When a cron job runs it doesn't have your normal profile information - it doesn't load your .profile/.bashprofile (simpler path, JAVA_HOME, etc) one possible option is to have the script source your .profile etc.
Cron frequently fails because of $PATH/working directory sorts of problems. You're setting the $PATH, but I wouldn't be surprised if neither your bash script nor your python script work if you aren't in the right directory.
Try using more absolute paths and see if that clears things up. Similarly, try running your cron command yourself from / or someplace and see if it works for you.