Crontab Python Script Execution (Can't find settings config file) - python

My Crontab -l
# m h dom mon dow command
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
00 8,20 * * * python /home/tomi/amaer/controller.py >>/tmp/out.txt 2>&1
My controller.py has config file settings.cfg also it uses other script in the folder it's located (I chmoded only controller.py)
The error
1;31mIOError^[[0m: [Errno 2] No such file or directory: 'settings.cfg'
I have no idea how to fix this? Please help me?
Edit: The part that read the config file
def main():
config=ConfigParser.ConfigParser()
config.readfp(open("settings.cfg"),"r")

As I initially wrote in my comment, this is because you are using relative path to the current working directory. However, that is not going to be the same when running all this via the cron executable rather than the python interpreter directly via the shebang.
Your current code would look for the "settings.cfg" in the current working directory which is where the cron executable resides, and not your script. Hence, you would need to change your code logic to using absolute paths by the help of the "os" built-in standard module.
Try to following line:
import os
...
def main():
config = ConfigParser.ConfigParser()
scriptDirectory = os.path.dirname(os.path.realpath(__file__))
settingsFilePath = os.path.join(scriptDirectory, "settings.cfg")
config.readfp(open(settingsFilePath,"r"))
This will get your the path of your script and then appends the "settings.cfg" with the appropriate dir separator for your operating system which is Linux in this particular case.
If the location of the config file changes any time in the future, you could use the argparse module for processing a command line argument to handle the config location properly, or even without it simply just using the first argument after the script name like sys.argv[1].

Your code is looking for settings.cfg in its current working directory.
This working directory will not be the same when cron executes the job, hence the error
You have two "easy" solutions:
Use an absolute path to the config file in your script (/home/tomi/amaer/config.cfg)
CD to the appropriate directory first in your crontab (cd /home/tomi/amaer/ && python /home/tomi/amaer/controller.py)
The "right" solution, though, would be to pass your script a parameter (or environment variable) that tells it where to look for the config file.
It's not exactly good practice to assume your config file will always be lying just next to your script.
You might want to have alook at this question: https://unix.stackexchange.com/questions/38951/what-is-the-working-directory-when-cron-executes-a-job

Related

Running python file through VS Code terminal

My question is I am unable to run a python file in VS Code terminal unless I specify the full path.
Whereas, when I see any python tutorial, only python filename is entered and it works.
Can somebody pls help me with this issue?
The files need to be in the same folder where you are operating / or have the other Python files.
You can use two ways for this:
first:
import os
pwd = os.getcwd()
# again, make sure you file is in the same directory.
the_file = (pwd + "\\filename.xlsx")
or Secondly,
#the below you can use wherever your file is and it will locate it.
# you specify the full path using pathlib.path and:
pathlib.path(directory here without the file itself).joinpath(file name here).

Run Python Script From Script Directory/Current Directory

[Introduction]
Hi! I'm working on a python script to automate installation of UWP-Apps, it's been a long time i'm not touching Python; until this day. The script uses Depedencies inside the script directory, i've looking up on my older scripts and found this specific code.
os.chdir(os.path.dirname(sys.argv[0]))
[Problematic]
However, using the above code doesn't work on my current script but it's working fine on older scripts. When using above, it shows:
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: ''
Already looking up on Internet about this topic; but most of them was talking about running the script from outer/different directory that leads me to dead end.
Any helps is appreciated :)
The easiest answer is probably to change your working directory, then call the .py file from where it is:
cd path/to/python/file && python ../.py
Of course you might find it even easier to write a script that does it all for you, like so:
Save this as runPython.sh in the directory where you're running the python script from, is:
#!/bin/sh
cd path/to/python/file
python ../script.py
Make it executable (for yourself):
chmod +x ./runPython.sh
Then you can simply enter your directory and run it:
./runPython.sh
If you want to only make changes to the python script:
mydir = os.getcwd() # would be the folder where you're running the file
mydir_tmp = "path/to/python/file" # get the path
mydir_new = os.chdir(mydir_tmp) # change the current working directory
mydir = os.getcwd()
The reason you got an error was because sys.argv[0] is the name of the file itself, instead you can pass the directory as a string and use sys.argv[1] instead.
import os
from os.path import abspath, dirname
os.chdir(dirname(abspath(__file__)))
You can use dirname(abspath(__file__))) to get the parent directory of the python script and os.chdir into it to make the script run in the directory where it is located.

Replicate ". ./bash.sh" shell command in python script [duplicate]

I want to implement a userland command that will take one of its arguments (path) and change the directory to that dir. After the program completion I would like the shell to be in that directory. So I want to implement cd command, but with external program.
Can it be done in a python script or I have to write bash wrapper?
Example:
tdi#bayes:/home/$>python cd.py tdi
tdi#bayes:/home/tdi$>
Others have pointed out that you can't change the working directory of a parent from a child.
But there is a way you can achieve your goal -- if you cd from a shell function, it can change the working dir. Add this to your ~/.bashrc:
go() {
cd "$(python /path/to/cd.py "$1")"
}
Your script should print the path to the directory that you want to change to. For example, this could be your cd.py:
#!/usr/bin/python
import sys, os.path
if sys.argv[1] == 'tdi': print(os.path.expanduser('~/long/tedious/path/to/tdi'))
elif sys.argv[1] == 'xyz': print(os.path.expanduser('~/long/tedious/path/to/xyz'))
Then you can do:
tdi#bayes:/home/$> go tdi
tdi#bayes:/home/tdi$> go tdi
That is not going to be possible.
Your script runs in a sub-shell spawned by the parent shell where the command was issued.
Any cding done in the sub-shell does not affect the parent shell.
cd is exclusively(?) implemented as a shell internal command, because any external program cannot change parent shell's CWD.
As codaddict writes, what happens in your sub-shell does not affect the parent shell. However, if your goal is to present the user with a shell in a different directory, you could always have Python use os.chdir to change the sub-shell's working directory and then launch a new shell from Python. This will not change the working directory of the original shell, but will leave the user with one in a different directory.
As explained by mrdiskodave
in Equivalent of shell 'cd' command to change the working directory?
there is a hack to achieve the desired behavior in pure Python.
I made some modifications to the answer from mrdiskodave to make it work in Python 3:
The pipes.quote() function has moved to shlex.quote().
To mitigate the issue of user input during execution, you can delete any previous user input with the backspace character "\x08".
So my adaption looks like the following:
import fcntl
import shlex
import termios
from pathlib import Path
def change_directory(path: Path):
quoted_path = shlex.quote(str(path))
# Remove up to 32 characters entered by the user.
backspace = "\x08" * 32
cmd = f"{backspace}cd {quoted_path}\n"
for c in cmd:
fcntl.ioctl(1, termios.TIOCSTI, c)
I shall try to show how to set a Bash terminal's working directory to whatever path a Python program wants in a fairly easy way.
Only Bash can set its working directory, so routines are needed for Python and Bash. The Python program has a routine defined as:
fob=open(somefile,"w")
fob.write(dd)
fob.close()
"Somefile" could for convenience be a RAM disk file. Bash "mount" would show tmpfs mounted somewhere like "/run/user/1000", so somefile might be "/run/user/1000/pythonwkdir". "dd" is the full directory path name desired.
The Bash file would look like:
#!/bin/bash
#pysync ---Command ". pysync" will set bash dir to what Python recorded
cd `cat /run/user/1000/pythonwkdr`

How to know path of calling shell directory and use it in my python executable

Basically, I want to call a python program from anywhere using shell so I added a shebang and copied it to /usr/local/bin with executable permission. The python program takes a command line argument which is the relative path of an input file.
I am stuck here, I have no idea what to do so that I can obtain the absolute path of the shell. I am assuming once I get the absolute path somehow, I can use sys.argv[1] to get the entered relative path of the file(which I will append to the absolute path of shell working directory) but please do correct me it won't work.
You can Print Working Directory via os.environ['PWD']. Content of your_script.py:
#!/usr/bin/python3.5
import os
print(os.environ['PWD'])
Usage:
sanyash#sanyash-ub16:/etc/nginx$ your_script.py
/etc/nginx
sanyash#sanyash-ub16:/etc/nginx$

python don't extract zipfile when ran through crontab

#!/usr/bin/python
import requests, zipfile, StringIO, sys
extractDir = "myfolder"
zip_file_url = "download url"
response = requests.get(zip_file_url)
zipDocument = zipfile.ZipFile(StringIO.StringIO(response.content))
zipinfos = zipDocument.infolist()
for zipinfo in zipinfos:
extrat = zipDocument.extract(zipinfo,path=extractDir)
System configuration
Ubuntu OS 16.04
Python 2.7.12
$ python extract.py
when I run the code on Terminal with above command, it works properly and create the folder and extract the file into it.
Similarly, when I create a cron job using sodu rights the code executes but don't create any folder or extracts the files.
crontab command:-
40 10 * * * /usr/bin/sudo /usr/bin/python /home/ubuntu/demo/directory.py > /home/ubuntu/demo/logmyshit.log 2>&1
also tried
40 10 * * * /usr/bin/python /home/ubuntu/demo/directory.py > /home/ubuntu/demo/logmyshit.log 2>&1
Notes :
I check the syslog, it says the cron is running successfully
The above code gives no errors
also made the python program executable by chmod +x filename.py
Please help where am I going wrong.
Oups, there is nothing really wrong in running a Python script in crontab, but many bad things can happen because the environment is not the one you are used to.
When you type in an interactive shell python directory.py, the PATH and all required PYTHON environment variable have been set as part of login and interactive shell initialization, and the current directory is your home directory by default or anywhere you currently are.
When the same command is run from crontab, the current directory is not specified (but may not be what you expect), PATH is only /bin:/usr/bin and python environment variables are not set. That means that you will have to tweak environment variables in crontab file until you get a correct Python environment, and set the current directory.
I had a very similar problem and it turned out cron didn’t like importing matplotlib, I ended up having to specify Agg backend. I figured it out by putting log statements after each line to see how far the program got before it crapped out. Of course, my log was empty which tipped me off that it crashed on imports.
TLDR: log each line inside the script

Categories