I'm trying to read credentials for the Google Sheets API using gspread. I wrote the following code:
class Command(BaseCommand):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def handle(self, *args, **kwargs):
scope = ['https://spreadsheets.google.com/feeds',
'https://www.googleapis.com/auth/drive']
credentials = ServiceAccountCredentials.from_json_keyfile_name('/static/json/spreadsheets.json', scope)
gc = gspread.authorize(credentials)
wks = gc.open("Where is the money Lebowski?").sheet1
self.stdout.write(self.style.SUCCESS('Succesfully ran "sheets" command'))
Reading the file returns the following error:
FileNotFoundError: [Errno 2] No such file or directory: 'static/json/spreadsheets.json'
I tried multiple paths like:
'~/static/json/spreadsheets.json'
'/json/spreadsheets.json'
'spreadsheets.json'
But none seem to work. Could anybody help me out here?
When you use an absolute path, it is taken literally i.e. from starting from the root of the filesystem i.e. /.
When you use a relative path i.e. without / at start, it is resolved from the directory where script is invoked from, not where the script actually lies in the filesystem.
So when you invoke the Django management command via e.g. ./manage.py <command>, it looks for a path starting from the current directory of manage.py i.e. os.path.dirname('manage.py'). If you give the path as static/json/spreadsheets.json, the full path it looks for is:
os.path.join(
os.path.abspath(os.path.dirname('manage.py')),
'/static/json/spreadsheets.json'
)
So you need to make sure you have the spreadsheets.json file in the right directory. A better approach would be to use an absolute path for these kind of scenarios. If you're on GNU/Linux, you can use:
readlink -f static/json/spreadsheets.json
to get the absolute path.
Also, rather than hardcoding the file, you should take the file as an argument to the management command. Django management command uses argparse for argument parsing, so you can take a peek at the doc.
First of all: ~ symbol should not work in this case, because it's just a glob which is being expanded to full path by *nix shell, so in this case it can't be expanded as shell is not involved here.
Leading / symbol will move you to the root directory.
I don't know all situation (because you didn't provided info about in which directory you run this command and where the file is located in comparison to that directory.
But you can use method getcwd from os library like this:
import os
print(os.getcwd())
Or from debugger, to get know your current location. Then you can pass absolute path to file or use ".." to get to parent directory if needed or even change current working directory using os.chdir() method, but it's not recommended to do because it can have some side-effects on Django or other libraries used in project which doesn't expect that directory could be changed in runtime.
Related
I have a python code with database file called pythontut.db (.py and db file on same folder). I used OS.path for path setting. When it is executed in thonny it works fine, I have created a systemd service to run at reboot. But at reboot, the path is different and throws 'unable to open database' error.
I tried setting path in pi-main.py like this
dbname = 'pythontut.db'
scriptdir = os.getcwd()
db_path = os.path.join(scriptdir, dbname)
print(db_path)
It shows output in thonny like this (Python file and DB are in same folder)
/home/pi/pi-project/pythontut.db
But when it runs via systemd service it throws location like this with unable to opendb error
/pythontut.db
I'm suspecting is it a path error or permission error. Probably if there is an another method for path setting.
Systemd file:
[Unit]
Description=Main Jobs Running
After=graphical.target
[Service]
Type=simple
User=pi
ExecStart=/usr/bin/python /home/pi/pi-project/pi-main.py
Restart=on-abort
[Install]
WantedBy=graphical.target
Is your file a part of your program? That is, will it always be true that the file will be in the same directory as the code? If so, then the following applies. If you are running the code at a command prompt (ie: you are writing a command line tool), this is the one case where using the current working directory makes sense. If you are locating a configuration file or something else where the user decides where to put the file, then you should somehow pass in the location of this file, like as a command line parameter or via a setting in a configuration file that your code knows how to find.
If the file you want to access is always in the same place relative to the code being executed, then you should write your code so that it is not dependent on what the current working directory is when it executes (ie: never use os.getcwd()). A very easy way to do this when you know where the file is relative to the file that contains the executing code is as follows:
# Get the directory containing this source file
code_directory = os.path.dirname(__file__)
# Relative path to get you from the directory contining your code
# to the directory containing the file you want to acceess.
relative_path = "../../some_directory"
# The name of the file you want to access
name_of_file_to_read = "somefile.txt"
# Compute the path to the file
file_path = os.path.join(code_directory, relative_path, name_of_file_to_read)
If the file you want to access is in the same directory as your code, then you can leave out relative_path or set it to ".".
Is there a way to specify the running directory of command in Python's subprocess.Popen()?
For example:
Popen('c:\mytool\tool.exe', workingdir='d:\test\local')
My Python script is located in C:\programs\python
Is is possible to run C:\mytool\tool.exe in the directory D:\test\local?
How do I set the working directory for a sub-process?
subprocess.Popen takes a cwd argument to set the Current Working Directory; you'll also want to escape your backslashes ('d:\\test\\local'), or use r'd:\test\local' so that the backslashes aren't interpreted as escape sequences by Python. The way you have it written, the \t part will be translated to a tab.
So, your new line should look like:
subprocess.Popen(r'c:\mytool\tool.exe', cwd=r'd:\test\local')
To use your Python script path as cwd, import os and define cwd using this:
os.path.dirname(os.path.realpath(__file__))
Other way is to simply do this
cwd = os.getcwd()
os.chdir('c:\some\directory')
subprocess.Popen('tool.exe')
os.chdir(cwd)
This solution works if you want to rely on relative paths, for example, if your tool's location is c:\some\directory\tool.exe. cwd keyword argument for Popen will not let you do this. Some scripts/tools may rely on you being in the given directory while invoking them. To make this code less noisy, aka detach the logic related to changing directories from the "business logic", you can use a decorator.
def invoke_at(path: str):
def parameterized(func):
def wrapper(*args, **kwargs):
cwd = os.getcwd()
os.chdir(path)
try:
ret = func(*args, **kwargs)
finally:
os.chdir(cwd)
return ret
return wrapper
return parameterized
Such decorator can be then used in a way:
#invoke_at(r'c:\some\directory')
def start_the_tool():
subprocess.Popen('tool.exe')
I'm working on a Python 3 Flask app and running dev_appserver.py to test locally.
I'm finding that when I modify my Jinja templates, the local instance doesn't seem to pick up those changes, even when I hard refresh in my browser. The only way I've been able to pick up changes is to kill the script and run it again.
I didn't find any mention of this in the official documentation and --help only mentions one relevant flag, and it says that by default the watcher ignores nothing:
--watcher_ignore_re WATCHER_IGNORE_RE
Regex string to specify files to be ignored by the
filewatcher. (default: None)
You could pass extra files to Flask().run() like the following:
from os import path, walk
extra_dirs = ['directory/to/watch',]
extra_files = extra_dirs[:]
for extra_dir in extra_dirs:
for dirname, dirs, files in walk(extra_dir):
for filename in files:
filename = path.join(dirname, filename)
if path.isfile(filename):
extra_files.append(filename)
app.run(extra_files=extra_files)
When i double click the bios.pyw it runs good but when I run this command from firm.pyw os.system("pythonw ..\\system_pbl\\bios\\bios.pyw") it gives me an error
I've tried running it with the start command but it doesn't do anything then
I've tried to run it in the same directory and with a .cmd and a .ps1 and a .bat file but no luck
firm.pyw-
import os
from tkinter import *
os.system("python ..\\system_pbl\\bios\\bios.pyw")
bios.pyw-
sil = "..\\..\\system_res files\\sil.sif"
si = PhotoImage(file=_sil)#-error
s_label = Label(app,image = si,borderwidth=0).place(y=0,x=0)
The error i get is
File "C:\Users\gavin\AppData\Local\Programs\Python\Python36-32\lib\tkinter\__init__.py", line 3498, in __init__
self.tk.call(('image', 'create', imgtype, name,) + options)
_tkinter.TclError: couldn't open "..\..\system_res files\sil.sif": no such file or directory
This is the file system:
file system:
fireware
2.S^
3.Fire-system ->firm.pyw
3.system_pbl -> bios->bios.pyw, data_change.pyw, local_update.py
-> boot->boot.pyw
3.system_res files -> sil.sif, usr_data.dde
Your script hardcodes the path to the data file so that you always have to run it in a subdirectory two levels below the directory with the data file.
If
python ./bios.pyw
works then
python ../firm.pyw
should work too, whereas
cd ..
python ./firm.pyw
will fail, because cd switches your current directory to one which isn't precisely two levels below the data file directory (you are now only one level below). Perhaps see also https://stackoverflow.com/a/55342466/874188
If you are running the scripts in a different manner (such as by double-clicking), you have to figure out how to control the current working directory within that mechanism. But a better fix is to not hard-code a relative path. You could
change the script so it accepts the path to the data file directory as a command-line argument or perhaps a configuration parameter;
change the script so it requires the data file subdirectory in the current directory. This is cumbersome to use, but at the very least simple and predictable behavior;
change the script so it figures out the absolute path to the data file directory based on something in the environment, such as, for example, the absolute path to the module you import; see How to retrieve a module's path?
More broadly, you should almost certainly not run Python as a subprocess of Python. You will probably need to reorganize your code slightly to avoid os.system() but you should find that the reorganized code is both easier to work with and much faster.
In relation to another question, how do you account for paths that may change? For example, if a program is calling a file in the same directory as the program, you can simply use the path ".\foo.py" in *nix. However, apparently Windows likes to have the path hard-coded, e.g. "C:\Python_project\foo.py".
What happens if the path changes? For example, the file may not be on the C: drive but on a thumb drive or external drive that can change the drive letter. The file may still be in the same directory as the program but it won't match the drive letter in the code.
I want the program to be cross-platform, but I expect I may have to use os.name or something to determine which path code block to use.
Simple answer: You work out the absolute path based on the environment.
What you really need is a few pointers. There are various bits of runtime and environment information that you can glean from various places in the standard library (and they certainly help me when I want to deploy an application on windows).
So, first some general things:
os.path - standard library module with lots of cross-platform path manipulation. Your best friend. "Follow the os.path" I once read in a book.
__file__ - The location of the current module.
sys.executable - The location of the running Python.
Now you can fairly much glean anything you want from these three sources. The functions from os.path will help you get around the tree:
os.path.join('path1', 'path2') - join path segments in a cross-platform way
os.path.expanduser('a_path') - find the path a_path in the user's home directory
os.path.abspath('a_path') - convert a relative path to an absolute path
os.path.dirname('a_path') - get the directory that a path is in
many many more...
So combining this, for example:
# script1.py
# Get the path to the script2.py in the same directory
import os
this_script_path = os.path.abspath(__file__)
this_dir_path = os.path.dirname(this_script_path)
script2_path = os.path.join(this_dir_path, 'script2.py')
print script2_path
And running it:
ali#work:~/tmp$ python script1.py
/home/ali/tmp/script2.py
Now for your specific case, it seems you are slightly confused between the concept of a "working directory" and the "directory that a script is in". These can be the same, but they can also be different. For example the "working directory" can be changed, and so functions that use it might be able to find what they are looking for sometimes but not others. subprocess.Popen is an example of this.
If you always pass paths absolutely, you will never get into working directory issues.
If your file is always in the same directory as your program then:
def _isInProductionMode():
""" returns True when running the exe,
False when running from a script, ie development mode.
"""
return (hasattr(sys, "frozen") or # new py2exe
hasattr(sys, "importers") # old py2exe
or imp.is_frozen("__main__")) #tools/freeze
def _getAppDir():
""" returns the directory name of the script or the directory
name of the exe
"""
if _isInProductionMode():
return os.path.dirname(sys.executable)
return os.path.dirname(__file__)
should work. Also, I've used py2exe for my own application, and haven't tested it with other exe conversion apps.
What -- specifically -- do you mean by "calling a file...foo.py"?
Import? If so, the path is totally outside of your program. Set the PYTHONPATH environment variable with . or c:\ or whatever at the shell level. You can, for example, write 2-line shell scripts to set an environment variable and run Python.
Windows
SET PYTHONPATH=C:\path\to\library
python myapp.py
Linux
export PYTHONPATH=./relative/path
python myapp.py
Execfile? Consider using import.
Read and Eval? Consider using import.
If the PYTHONPATH is too complicated, then put your module in the Python lib/site-packages directory, where it's put onto the PYTHONPATH by default for you.
I figured out by using os.getcwd(). I also learned about using os.path.join to automatically determine the correct path format based on the OS. Here's the code:
def openNewRecord(self, event): # wxGlade: CharSheet.<event_handler>
"""Create a new, blank record sheet."""
path = os.getcwd()
subprocess.Popen(os.path.join(path, "TW2K_char_rec_sheet.py"), shell=True).stdout
It appears to be working. Thanks for the ideas.