I am new to crontab and I'm struggling with the basics. I a lot of different issues. This is all being done on my Raspberry Pi. I am trying to schedule a python script to run every 10 minutes on weekdays. What makes this trickier is that my python script needs to use a virtual environment.
After doing some research I saw I could just activate the virtual environment with a bash script and then run the python script through that. This is the bash script:
#!/bin/bash
cd /home/pi/Desktop/projects/my_project
source env/bin/activate
python my_script.py
I have no idea if this is the best way to run python in a virtual environment through crontab but its all I could find online.
This is the code for the crontab itself:
SHELL=/bin/bash
0-59/10 * * * 1-5 /home/pi/Desktop/projects/my_project/cron_script.sh
I tested the crontab code by running this in the command line: sudo crontab -l | grep -v '^#' | cut -f 6- -d ' ' | while read CMD; do eval $CMD; done. Again I have no idea if this is the best way to test it immediately. This throws an error message:
bash: /home/pi/Desktop/projects/my_project/cron_script.sh: Permission denied
I'm really just confused at a lot of different steps here. Summary:
I don't know if I'm using a python virtual environment in crontab correctly or whether there's a cleaner way to do it.
I don't know if there is a better way to test crontab immediately. Writing the print statements from my python script to a txt file would also be nice.
I'm getting a permission denied error message.
The error you're having is likely due to the fact that your bash script is not marked as executable. You can use chmod to change access permissions.
Try this :
chmod +x /home/pi/Desktop/projects/my_project/cron_script.sh
Related
I have a bash file, it works fine when executed from terminal.
#!/bin/bash
source activate tensorflow_p36
python /home/ec2-user/abc/wsgi.py
Note: tensorflow_p36 being an in-built conda environment, does not require to be called from specific /env/bin directory. It can be activated from any directory. I think it's a feature of Amazon Deep Learning AMIs.
If I run this bash script with sudo it doesnt activate the virtual environment and works in default python environment. The python file can run in that virtual environment only.
I have tried all 3 alternatives (rc.local, .conf file, init.d config)here, also tried to use crontab as suggested here. I have also tried using supervisord to add this bash script as a program.
When the program runs from these methods, I always get the same import errors because it is using default python 3 environment which doesn't have the required dependencies.
I am working on Amazon CentOS (Deep learning AMI). Can someone please suggest a method to run this script every time system restarts?
In the rc.local, instruct root to run it as you:
su --command /path/to/bash/file --login grimlock
You can run it from your personal Crontab.
( crontab -l; printf '#reboot /path/to/bash/file\n' ) | crontab -
If you don't have a crontab there will be an error message from crontab -l but it's harmless.
crontab: no crontab for ec2-user
You just need to do this once, and the job will execute as yourself once the system comes up.
try to change source by .
. activate tensorflow_p36
python /home/ec2-user/abc/wsgi.py
also check chmod +x your path file.
I tried to make a cron job on crontab which runs a python script on AWS ec2. My python script includes a module that is only available for python3.
Using the following command I changed the ec2 default python interpreter from python2.7 to python3.4
Source /home/ec-2user/venv/python34/bin/activate
and then using pip install, I installed the required module for python3.4. So now the default interpreter is python3.4 and when I run the script on ec2-user directory using the following command:
python test.py
the program runs without any problem (so I am sure the module is installed correctly).
But when I assign python file to a cronjob
* * * * * python test.py
It does not work. Checking the mail, the error is:
“No module found named “xxxxx” “
But as I said it worked fine outside of the cron.
I was wondering if you can help me with this problem. I appreciate your time and information.
You have to make a shell script which will do the steps of changing to script directory, activating virtual environment and then running it.
Example:
#!/bin/bash
cd $YOUR_DIR
. venv/bin/activate
python3.4 test.py
Then you call this script in cron with
/bin/bash /.../script.sh
What you could do additionally is
chmod +x test.py
and add/update first line to:
#!/usr/bin/env python3.4
This way you can just run Python script with ./test.py
Create file as 'user_cron.sh'
#!/bin/bash
cd '/root/my_new_project_python'
. my_project_venv/bin/activate
python3 main.py
set the cron using crontab -e
I just started using Cron to automate this one python script I have. I understand how to use all the time parameters in nano, but I'm confused with how you would run the script. Normally just to run it right from the console, I would do:
cd /pi/home/weather/Adafruit_Python_BMP/examples
and then from there I would run the script with:
python weatherFINAL.py
Now that I'm trying to automate this in Cron, I can't do the multiple commands to cd into the directory, and then run the program. I know this is probably a really easy problem to fix, but I've been stuck on this for a while. Any help is appreciated
It is quite simple:
Write a shebang line on the top of your script in order to make it executable:
!/usr/bin/env python
Make sure you can execute that script with chmod command:
$ chmod +x
Programn a crontab task with contrab command:
$contrab -e
I am trying to run a cron script in python 3 so I had to setup a virtual environment (if there is an easier way, please let me know) and in order to run the script I need to be in the script's parent folder as it writes to text files there. So here is the long string of commands I have come up with and it works in console but does not work in cron (or I can't find the output..)
I can't type the 5 asterisks without it turning into bullet points.. but I have them in the cron tab.
cd usr/local/sbin/cronjobs && . virtualenv/secret_ciphers/bin/activate
&& cd csgostatsbot && python3 CSGO_STATS_BOT_TASK.py && deactivate
It looks like you may have a stray . in there that would likely cause an error in the command chain.
Try this:
cd usr/local/sbin/cronjobs && virtualenv/secret_ciphers/bin/activate
&& cd csgostatsbot && python3 CSGO_STATS_BOT_TASK.py && deactivate
Assuming that the virtualenv directory is in the cronjobs directory.
Also, you may want to skip the activate/deactivate, and simply run the python3 interpreter right out of the virtualenv. i.e.
/usr/local/sbin/cronjobs/virtualenv/secret_ciphers/bin/python3 /usr/local/sbin/cronjobs/csgostatsbot/CSGO_STATS_BOT_TASK.py
Edit in response to comments from OP:
The activate call is what activates the virtualenv. Not sure what the . would do aside from cause shell command parsing issues.
Both examples involve the use of the virtualenv. You don't need to explicitly call activate. As long as you invoke the interpreter out of the virtualenv's directory, you're using the virtualenv. activate is essentially a convenience method that tweaks your PATH to make python3 and other bin files refer to the virtualenv's directory instead of the system install.
2nd Edit in response to add'l comment from OP:
You should redirect stderr, i.e.:
/usr/local/sbin/cronjobs/virtualenv/secret_ciphers/bin/python3
/usr/local/sbin/cronjobs/csgostatsbot/CSGO_STATS_BOT_TASK.py >
/tmp/botlog.log 2>&1
And see if that yields any additional info.
Also, 5 asterisks in cron will run the script every minute 24/7/365. Is that really what you want?
3rd Edit in response to add'l comment from OP:
If you want it to always be running, I'm not sure you really want to use cron. Even with 5 asterisks, it will run it once per minute. That means it's not always running. It runs once per minute, and if it takes longer than a minute to run, you could get multiple copies running (which may or may not be an issue, depending on your code), and if it runs really quickly, say in a couple seconds, you'll have the rest of the minute to wait before it runs again.
It sounds like you want the script to essentially be a daemon. That is, just run the main script in a while (True) loop, and then just launch it once. Then you can quit it via <crtl>+c, else it just perpetually runs.
Try these commands. Hopefully you'll end up with a simpler, more understandable arrangement:
$ sudo apt-get install python3 # Just in case you haven't already
$ sudo apt-get install python3-pip
$ sudo pip3 install praw
$ vi CSGO_STATS_BOT_TASK.py
«Ensure that "#!/usr/bin/env python3" is the first line»
$ chmod +x CSGO_STATS_BOT_TASK.py
$ crontab -e
* * * * * /path/to/CSGO_STATS_BOT_TASK.py
I am running a command which was installed as a script of a python package i created.
I have a cronned_job_shell_script.sh file:
touch a.txt
my_script_command
where my_scrip_command was installed using pip install my_py_package.tar.gz
The cron line is:
0 * * * * cronned_job_shell_script.sh
If I run cronned_job_shell_script.sh from shell it works fine, running the python script also.
Even trying env -i /bin/bash --noprofile --norc first and then running the script works.
The problem is, when scheduled by cron, the file a.txt is touched but the script does not seem to run.
The problem is that you're putting this in the system crontab, which means it's being run by root, and root has a different PATH variable than your user. So, testing it in the shell (as you, not root) doesn't actually test the same thing.
The easiest solution is to just use the absolute path to the script—/opt/mystuff/bin/my_script_command instead of just my_script_command.