I've tried to run a script from crontab on my Linux system which is leaving an empty file. When I run the script logged on the terminal, it works fine.
Cron line:
#reboot sleep 1m; /bin/bash /root/start_reader_services
The script "start_reader_services" calls a Python script as below:
/root/java/tag_output >> $TAGS_PATH/tags_$DATE_FILE.log
tag_output basically prints out a series of IDs. The same mechanics used to work when I was sending the stdout to my serial port (tag_output > /dev/ttyO0), but now, writing it to the file from cron, the file is created, but is empty.
As I mentioned, running start_reader_services or any piece of that on command line, it works as expected.
Have done:
- Set bash as cron shell
- Set java environments on cron
As requested:
ls -l /root/java/tag_output
-rwxr-xr-x 1 root root 1981 Aug 6 12:06 /root/java/tag_output
First line of tag output:
#!/usr/bin/python
Any help?
Is it possible that tag_output is writing to stderr instead of stdout? Try redirecting stderr too:
/root/java/tag_output >> $TAGS_PATH/tags_$DATE_FILE.log 2>&1
As an aside, you might also want to quote and use braces with the shell variable expansion; that makes it easier to read and probably a little safer:
/root/java/tag_output >> "${TAGS_PATH}/tags_${DATE_FILE}.log" 2>&1
After initialising and updating the crontab using crontab python and before issuing
cron.write(), give the command,
cron.__reduce__()[2]['lines'] = filter(None,
cron.__reduce__()[2]['lines'])
Doing this will remove the empty lines being printed in Linux crontab.
Related
I have a bash script that I'm using to execute a python file with a specific version of Python (3.6). The Bash script is currently located on my desktop (/home/pi/Desktop/go.sh)
#!/bin/bash
python3.6 /home/pi/scriptDir/myScript.py
Here is my crontab entry, when I do crontab -l (note, I've deleted my other jobs)
* * * * * bash /home/pi/Desktop/go.sh # JOB_ID_3
When I run this file using the command line or from the GUI it executes properly.
When I have crontab do it, nothing happens.
Both my python file and the bash script are executable. chmod +x
Is there something obvious I'm missing?
**my python script does depend on other files in the same script directory, could that be the issue?
Here's what got it working for me. I was not using the full path to my python install. Unless you log the bash file there's no indication that you have an issue.
This is my bash file now. echo's were just to determine that I was indeed running the bash file.
#!/bin/bash
echo started
/home/pi/Python-3.6.0/python home/pi/myScriptFolder/myScript.py
echo finished
To break down the line executing the script:
/home/pi/Python-3.6.0/python - is where python 3.6.0 is installed on my Pi, it could be different for you. home/pi/myScriptFolder/myScript.py is the script I want to run.
And here is my cron statement:
*/15 * * * * bash /home/pi/Desktop/go.sh > /home/pi/Desktop/clog.log 2>&1 -q -f
Breaking down this line:
*/15 * * * * is the cron time, in this case every 15 mins. bash /home/pi/Desktop/go.sh specifies to run a bash file and the directory of that file. > /home/pi/Desktop/clog.log 2>&1 -q -f this last section creates a log file named clog.log so you can see what's going on.
The key here was not just logging the go.sh bash file execution, but adding the 2>&1 -q -f to the end of the log request. Before I did that there was no indication of a problem, afterwards I was getting the python files error returned into the log file.
Cron jobs are surprisingly tricky. Aside from working directory ( which you'd have to set somewhere), you also need to handle environment setup ( $PATH, for example).
Start by redirecting your shell script's standard output and error to a log file so you can get feedback.
I want to run a python script on boot of ubuntu 14.04LTS.
My rc.local file is as follows:
sudo /home/hduser/morey/zookeeper-3.3.6/bin/zkServer.sh start
echo "test" > /home/hduser/test3
sudo /home/hduser/morey/kafka/bin/kafka-server-start.sh /home/hduser/morey/kafka/config/server.properties &
echo "test" > /home/hduser/test1
/usr/bin/python /home/hduser/morey/kafka/automate.py &
echo "test" > /home/hduser/test2
exit 0
everything except my python script is working fine even the echo statement after running the python script, but the python script doesnt seem to run.
My python script is as follows
import sys
from subprocess import Popen, PIPE, STDOUT
cmd = ["sudo", "./sbt", "project java-examples", "run"]
proc = Popen(cmd, shell=False, stdout=PIPE, stdin=PIPE, stderr=STDOUT)
proc.communicate(input='1\n')
proc.stdin.close()
which works perfectly fine if executed individually.
I went through the following questions , link
I did a lot of research but couldn't find a solution
Edit : echo statements are for testing purpose only, and the second actual command (not considering the echo statements) is starting a server which keeps on running, and even the python script starts a listener which runs on an infinite loop, if this is any help
The Python script tries to launch ./sbt. Are you sure of what if the current directory when rc.local runs? The rule is always use absolute paths in system scripts
Do not run the Python script in background, run it in foreground. Do not exit from its parent script. Better call another script from "rc.local" that does all the job of "echo" and script launching.
Run that script from "rc.local"; not in background (no &).
You do not need "sudo" as "rc.local" is run as root.
If you want to run python script at system boot there is an alternate solution which i have used.
1:Create sh file like sample.sh and copy paste following content
#!/bin/bash
clear
python yourscript.py
2:Now add a cron job at reboot.If you are using linux you can use as following
a:Run crontab -e(Install sudo apt-get install cron)
b:#reboot /full path to sh file > /home/path/error.log 2>&1
And restart your device
I am trying to setup a crontab to run 6 python data scrapers. I am tired of having to restart them manually when one of them fails. When running the following:
> ps -ef | grep python
ubuntu 31537 1 0 13:09 ? 00:00:03 python /home/ubuntu/scrapers/datascraper1.py
etc... I get a list of the datascrapers 1-6 all in the same folder.
I edited my crontab like this:
sudo crontab -e
# m h dom mon dow command
* * * * * pgrep -f /home/ubuntu/scrapers/datascraper1.py || python /home/ubuntu/scrapers/datascraper1.py > test.out
Then I hit control+X to exit and hit yes to save as /tmp/crontab.M6sSxL/crontab .
However it does not work in restarting or even starting datascraper1.py whether I kill the process manually or if the process fails on its own. Next, I tried reloading cron but it still didn't work:
sudo cron reload
Finally I tried removing nohup from the cron statement and that also did not work.
How can I check if a cron.allow or cron.deny file exists?
Also, do I need to add a username before pgrep? I am also not sure what the "> test.out" is doing at the end of the cron statement.
After running
grep CRON /var/log/syslog
to check to see if cron ran at all, I get this output:
ubuntu#ip-172-31-29-12:~$ grep CRON /var/log/syslog
Jan 5 07:01:01 ip-172-31-29-12 CRON[31101]: (root) CMD (pgrep -f datascraper1.py ||
python /home/ubuntu/scrapers/datascraper1.py > test.out)
Jan 5 07:01:01 ip-172-31-29-12 CRON[31100]: (CRON) info (No MTA installed, discarding output)
Jan 5 07:17:01 ip-172-31-29-12 CRON[31115]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jan 5 08:01:01 ip-172-31-29-12 CRON[31140]: (root) CMD (pgrep -f datascraper1.py || python /home/ubuntu/scrapers/datascraper1.py > test.out)
Since there is evidence of Cron executing the command, there must be something wrong with this command, (note: I added the path to python):
pgrep -f datascraper1.py || /usr/bin/python /home/ubuntu/scrapers/datascraper1.py > test.out
Which is supposed to check to see if datascaper1.py is running, if not then restart it.
Since Cron is literally executing this statement:
(root) CMD (pgrep -f datascraper1.py || python /home/ubuntu/scrapers/datascraper1.py > test.out)
aka
root pgrep -f datascraper1.py
Running the above root command gives me:
The program 'root' is currently not installed. You can install it by typing:
sudo apt-get install root-system-bin
Is there a problem with Cron running commands from root?
Thanks for your help.
First of all, you need to see if cron is working at all.
Add this to your cron file (and ideally delete the python statement for now, to have a clear state)
* * * * * echo `date` >>/home/your_username/hello_cron
This will output the date in the file "hello_cron" every minute. Try this, and if this works, ie you see output every minute, write here and we can troubleshoot further.
You can also look in your system logs to see if cron has ran your command, like so:
grep CRON /var/log/syslog
Btw the >test.out part would redirect the output of the python program to the file test.out. I am not sure why you need the nohup part - this would let the python programs run even if you are logged out - is this what you want?
EDIT: After troubleshooting cron:
The message about no MTA installed means that cron is trying to send you an e-mail with the output of the job but cannot because you dont have an email programm installed:
Maybe this will fix it:
sudo apt-get install postfix
The line invoking the python program in cron is producing some output (an error) so it's in your best interests to see what happens. Look at this tutorial to see how to set your email address: http://www.cyberciti.biz/faq/linux-unix-crontab-change-mailto-settings/
Just in case the tutorial becomes unavailable:
MAILTO:youremail#example.com
You need to add python home to your path at the start of the job, however you have python set up. When you're running it yourself and you type python, it checks where you are, then one level down, then your $PATH. So, python home (where the python binary is) needs to be globally exported for the user that owns the cron (so, put it in a rc script in /etc/rc.d/) or, you need to append the python home to path at the start of the cron job. So,
export PATH=$PATH:<path to python>
Or, write the cron entry as
/usr/bin/python /home/ubuntu/etc/etc
to call it directly. It might not be /usr/bin, run the command
'which python'
to find out.
The 'No MTA' message means you are getting STDERR, which would normally get mailed to the user, but can't because you have no Mail Transfer Agent set up, like mailx, or mutt, so no mail for the user can get delivered from cron, so it is discarded. If you'd like STDERR to go into the log also, at the end, instead of
"command" > test.out
write
"command" 2>&1 > test.out
to redirect STDERR into STDOUT, then redirect both to test.out.
So I'm trying to have my server execute a Python script every hour.
When I go into the directory and run it with python twitter.py, it works fine.
However, I have this entry in crontab and it doesn't work:
0 * * * * /run/twitterparse/twitter.py > /run/twitterparse
I am trying to have it execute every hour, on the hour.
Here's the output to the syslog:
Aug 5 13:00:01 localhost CRON[11474]: (root) CMD (/run/twitterparse/twitter.py >/run/twitterparse/)
Aug 5 13:00:01 localhost CRON[11473]: (CRON) info (No MTA installed, discarding output)
Now what it should be doing is accessing a database and saving information from the web to that database. The script does that fine when run manually, but not automatically.
I fixed my problem by doing this
0 * * * * python /run/twitterparse/twitter.py > /run/twitterparse
and the py file should be executable
chmod +x twitter.py
You probably need to add
#!/usr/bin/python
to the beginning of the main script, with the location of YOUR python interpreter.
and change the permissions of the script to executable
chmod +x <main script name .py>
Hope this helps
Any output of a cron script that goes to the terminal (i.e. stdio or stderr) is sent to the person who's account the script is registered under.
You can see from the error message that it is trying to send this mail but "No MTA is installed" so it is discarding all the diagnostic messages.
Your first step would be to either fix the problem with the MTA so you can receive the mail or to make sure that stdio and stderr are written to a log file.
The obvious problem that I can see is that the output is being directed to a non-file.
/run/twitterparse/twitter.py > /run/twitterparse
Isn't /run/twitterparse a folder? I guess:
/run/twitterparse/twitter.py > /run/twitterparse/ouput.txt
would solve half the issues..
I have a python script in /usr/share/myscript.py
I want to execute this script from a cron job, which if the script produces any errors, emails these errors to a specific user (and does not inform the root account).
I do not want to over-ride any of the cron settings - other cron jobs should still notify root.
Currently I am using a shell wrapper, which should pipe errors to a log file and then email it to me. The cron job then executes this .sh file rather than the python script directly.
#!/bin/sh
python /usr/share/scripts/myscript.py 2>&1 > /home/me/logs/myscript.log
test -s /home/me/logs/myscript.log && cat /home/me/logs/myscript.log | mail -s "myscript errors" bob#myplace.com
In production, if nothing goes wrong, then the script executes correctly and nobody is emailed. However, if there is an error in the execution of the python script, then this is still being emailed to the root user from cron.
How should I change the .sh script to suppress this and report to me instead?
This command does the redirection of err output not in the order you want:
python /usr/share/scripts/myscript.py 2>&1 > /home/me/logs/myscript.log
instead you need to redirect stdin first, and stderr second, like so:
python /usr/share/scripts/myscript.py > /home/me/logs/myscript.log 2>&1
Also, have you appended >/dev/null 2>&1 to the end of the wrapped script call in crontab?