We Keep Coding

Run Python Script From Script Directory/Current Directory

[Introduction]
Hi! I'm working on a python script to automate installation of UWP-Apps, it's been a long time i'm not touching Python; until this day. The script uses Depedencies inside the script directory, i've looking up on my older scripts and found this specific code.
os.chdir(os.path.dirname(sys.argv[0]))
[Problematic]
However, using the above code doesn't work on my current script but it's working fine on older scripts. When using above, it shows:
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: ''
Already looking up on Internet about this topic; but most of them was talking about running the script from outer/different directory that leads me to dead end.
Any helps is appreciated :)

The easiest answer is probably to change your working directory, then call the .py file from where it is:
cd path/to/python/file && python ../.py
Of course you might find it even easier to write a script that does it all for you, like so:
Save this as runPython.sh in the directory where you're running the python script from, is:
#!/bin/sh
cd path/to/python/file
python ../script.py
Make it executable (for yourself):
chmod +x ./runPython.sh
Then you can simply enter your directory and run it:
./runPython.sh
If you want to only make changes to the python script:
mydir = os.getcwd() # would be the folder where you're running the file
mydir_tmp = "path/to/python/file" # get the path
mydir_new = os.chdir(mydir_tmp) # change the current working directory
mydir = os.getcwd()
The reason you got an error was because sys.argv[0] is the name of the file itself, instead you can pass the directory as a string and use sys.argv[1] instead.

import os
from os.path import abspath, dirname
os.chdir(dirname(abspath(__file__)))
You can use dirname(abspath(__file__))) to get the parent directory of the python script and os.chdir into it to make the script run in the directory where it is located.

Python3 -m run configuration in Eclipse

Update 2021: Solution is built into PyDev/Eclipse
See accepted answer for details
Original Question (and old answers) for below
As many comments/questions/rants on SO and other places will tell you, Python3 packages using relative imports want to be run from a central __main__.py file. In the case where a module, say "modA" within a package, say "packA", that uses relative imports needs to be run (for instance because a test package is run if __name__ == '__main__'), we are told to run instead run python3 -m modA.packA from the directory above modA if sys.path() does not contain the directory above modA. I may dislike this paradigm, but I can work around it.
When trying to run modA from Eclipse/PyDev, however, I can't figure out how to specify a run configuration that will execute the module properly with the -m flag. Has anyone figured out how to set up a run configuration that will do this properly?
References: Relative imports for the billionth time ; Relative import in Python 3 is not working ; Multilevel relative import

Nowadays (since PyDev 5.4.0 (2016-11-28)) you can go to the Settings > PyDev > Run and select Launch modules with python -m mod.name instead of python filename.py ;)
See: https://www.pydev.org/history_pydev.html
For older versions of PyDev (old answer)
Unfortunately, right now, it's not automatic running with -m in PyDev, so, I'll present 3 choices to work in PyDev with relative imports which are preceded by a dot (in PyDev version 4.3.0):
Don't use relative imports, only absolute imports in your __main__ modules.
Create a separate module for the __main__ which will do an absolute import for the module you want to run and run that module instead (if you're distributing your application, this is probably needed anyways as the usual way for people to launch your code in Python is by passing the script as an argument to Python and not using the -m switch).
Add the -m module to the vm arguments in your run configuration by doing:
Make the run (which will fail because of the relative import)
Right-click the editor > Copy Context Qualified Name
Open the run configuration: Alt, R, N (i.e.: Toolbar > Run > Run Configuration)
Open arguments tab and add the '-m Ctrl+V' (to add the -m and the module name you copied previously).
Although this is definitely not ideal: you'll now receive an argument with the filename (as PyDev will always pass that to run the file) and the whole process is a nuisance.
As a note, I do hope to provide a way to make runs within PyDev using the -m soon (hopefully for PyDev 4.4.0)... although this may not be possible if the file being run is not under the PYTHONPATH (i.e.: to run an external file it still has to support the option without the -m).

There's a bit nasty trick possible here to work around this issue. I'm using PyDev 9.2.0
Put your venv right in the workspace, say under the dir "venv".
Refresh your eclipse workspace and ensure that it uses this venv (through your interpreter setup).
After the refresh, go to the run configuration and edit the "Main Module" by clicking the Browse button.
The venv will now appear.
Browse into the venv/lib/python3.8/site-packages
There you will find the pip-installed module source codes and you can select the module you want to run.

Update 2021: This answer is no longer needed. See accepted answer for details.
Here's what I was able to do after Fabio's great suggestion.
Create a program called /usr/local/bin/runPy3M with world read/execute permissions, with the following code:
#!/usr/local/bin/python3 -u
'''
Run submodules inside packages (with relative imports) given
a base path and a path (relative or absolute) to the submodule
inside the package.
Either specify the package root with -b, or setenv ECLIPSE_PROJECT_LOC.
'''
import argparse
import os
import re
import subprocess
import sys
def baseAndFileToModule(basePath, pyFile):
'''
Takes a base path referring to the root of a package
and a (relative or absolute) path to a python submodule
and returns a string of a module name to be called with
python -m MODULE, if the current working directory is
changed to basePath.
Here the CWD is '/Users/cuthbert/git/t/server/tornadoHandlers/'.
>>> baseAndFileToModule('/Users/cuthbert/git/t/', 'bankHandler.py')
'server.tornadoHandlers.bankHandler'
'''
absPyFilePath = os.path.abspath(pyFile)
absBasePath = None
if basePath is not None:
absBasePath = os.path.abspath(basePath)
commonPrefix = os.path.commonprefix([absBasePath, absPyFilePath])
uncommonPyFile = absPyFilePath[len(commonPrefix):]
else:
commonPrefix = ""
uncommonPyFile = absPyFilePath
if commonPrefix not in sys.path:
sys.path.append(commonPrefix)
moduleize = uncommonPyFile.replace(os.path.sep, ".")
moduleize = re.sub("\.py.?$", "", moduleize)
moduleize = re.sub("^\.", "", moduleize)
return moduleize
def exitIfPyDevSetup():
'''
If PyDev is trying to see if this program is a valid
Python Interpreter, it expects to function just like Python.
This is a little module that does this if the last argument
is 'interpreterInfo.py' and then exits.
'''
if 'interpreterInfo.py' in sys.argv[-1]:
interarg = " ".join([sys.executable] + sys.argv[1:])
subprocess.call(interarg.split())
exit()
return
exitIfPyDevSetup()
parser = argparse.ArgumentParser(description='Run a python file or files as a module.')
parser.add_argument('file', metavar='pyfile', type=str, nargs=1,
help='filename to run, with .py')
parser.add_argument('-b', '--basepath', type=str, default=None, metavar='path',
help='path to directory to consider the root above the package')
parser.add_argument('-u', action='store_true', help='unbuffered binary stdout and stderr (assumed)')
args = parser.parse_args()
pyFile = args.file[0]
basePath = args.basepath
if basePath is None and 'ECLIPSE_PROJECT_LOC' in os.environ:
basePath = os.environ['ECLIPSE_PROJECT_LOC']
modName = baseAndFileToModule(basePath, pyFile)
allPath = ""
if basePath:
allPath += "cd '" + basePath + "'; "
allPath += sys.executable
allPath += " -m " + modName
subprocess.call(allPath, shell=True) # maybe possible with runpy instead...
Then add a new interpreter to PyDev -- call it what you'd like (e.g., runPy3M) and point the interpreter to /usr/local/bin/runPy3M. Click okay. Then move it up the list if need be. Then add to environment for the interpreter, Variable: ECLIPSE_PROJECT_LOC (name here DOES matter) with value ${project_loc}.
Now submodules inside packages that choose this interpreter will run as modules relative to the sub package.
I'd like to see baseAndFileToModule(basePath, pyFile) added to runpy eventually as another run option, but this will work for now.
EDIT: Unfortunately, after setting all this up, it appears that this configuration seems to prevent Eclipse/PyDev from recognizing builtins such as "None", "True", "False", "isinstnance," etc. So not a perfect solution.

Python, reference a file in the same directory as the running file

If I have a Python file called example.py that says with open('%s/data.csv' % sys.path[0]) as var_name: and data.csv is in the same directory as example.py, it works fine if I run example.py from the terminal, but when using unittest it does not work. My understanding is that when running it from the terminal, sys.path[0] gives me the path to the file being run (example.py), and thus if data.csv and example.py are in the same directory, sys.path[0]/data.csv will correctly find data.csv. But when unittest runs, for whatever behind-the-scenes reason sys.path[0] is empty, and /data.csv is correctly NOT found.
So I was wondering what a good strategy (and perhaps some explicit code :) ) is to make this reference always work no matter what (I'm guessing without sys.path[0] but if there's something that works with that I'll take it).
I should also mention I'm on OS X 10.9.3 and using python 2.7

just a guess but try
with open('%s/data.csv' % os.path.dirname(__file__)) as ...
or probably even preferred
file_path = os.path.join(os.path.dirname(__file__),"data.csv")

How to execute command by python?

I want to give a directory from raw_input of user and have cd to that directory by python
I used that code but it doesn't work for me:
a=raw_input("Enter The Dir :")
import os
os.system("cd "+a)
but it doesn't change the current directory of the python program!

If you want to change the current directory of a python program you should call os.chdir:
>>> import os
>>> os.getcwd() # show current working directory (cwd)
'/home/username'
>>> os.chdir('Downloads')
>>> os.getcwd()
'/home/username/Downloads'
Running the cd command via os.system doesn't work because os.system executes the command-line passed as argument in a new shell. From the documentation:
Execute the command (a string) in a subshell. This is implemented by
calling the Standard C function system(), and has the same
limitations.
The cd command actually does work, the problem is that it changes the current directory of the newly created shell, not of the current running python process. Using os.system('cd ' + x) is the same as doing:
$$(cd Downloads) #execute in a subshell
$pwd
/home/username
While calling os.chdir is equivalent to:
$cd Downloads
$pwd
/home/username/Downloads
i.e. it actually changes the current working directory as you want.
Note that os.system should be avoided. Use the subprocess module instead. This is even mentioned in the documentation for os.system:
The subprocess module provides more powerful facilities for spawning
new processes and retrieving their results; using that module is
preferable to using this function. See the Replacing Older Functions
with the subprocess Module section in the subprocess documentation for
some helpful recipes.
The subprocess module gives you much more control over the execution of the commands.
In fact your current code has a huge security flaw. For example what if the user entered a directory called (WARNING: do not provide this path to the abovementioned program. It will wipe your home directory!):
~;rm -fr .
While using subprocess you can avoid to get it interpreted as a shell command. For example:
import subprocess
dir_name = raw_input('Insert directory: ')
subprocess.call(['ls', dirname])
when provided with the above path will simply try to list the contents of a directory called ~;rm -fr . (a valid UNIX path), which probably doesn't exist.
While:
import os
dir_name = raw_input('Insert directory: ')
os.system('ls ' + dir_name=)
Will first list the contents of ~ (aka, your home directory), and then it will wipe the home.

How to change the working directory for a shell script (newbie here)

I have a python script that looks files up in a relative directory. For example: the python script is in /home/username/projectname/. I have a file that is being called within the python script that is in /home/username/projectname/subfolder.
If I run the script from the shell as python scriptname.py it runs perfectly fine.
However, i'm trying to run the script as a startup service. I'm setting it up in webmin, and I believe its using a terminal command to call it. In the startup command, I'm doing something like this to call the script:
execute python home/username/projectname/scriptname.py. The script is starting up fine, but I get an error because it cant access the files in the relative directory.
I am guessing that there is a better way to call the python program from within the startup command so that its aware of the relative path.

import os
os.chdir("/tmp")
Also have:
os.getcwd()
Maybe you need the following instead?
import os.path
dir = os.path.dirname(__file__)

The process that starts your python script (probably forkink) has a pwd (its working directory). The idea is to change the pwd of the process before to fork and execute python.
You need to look over the manual of the process that executes the shell command, and see how to set the pwd.(in shell you use cd or pushd)

__file__ is the path that was used to run your script.
Copy this into a script file and try running it:
import os
print "This script was run as: %r" % __file__
print "This script is: %r" % os.path.abspath(__file__)
script_dir = os.path.dirname(os.path.abspath(__file__))
print "This script is in directory: %r" % script_dir
print "When started, the current directory was: %r" % os.getcwd()
os.chdir(script_dir)
print "After calling os.chdir(), the current directory is: %r" % os.getcwd()