Lets assume following directories tree:
/
home/
user/
X/
scripy.py
Y/
a.c
b.c
I have a python script, with many commands using relative path, doing sth like this:
def f1(relative_path_to_a, relative_path_to_c):
abs_path = PATH_TO_PLACE_FROM_SCRIPT_WAS_CALLED
c_abs_path = abs_path + relative_path_to_c
file_a = open(relative_path_to_a)
file_a.write(c_abs_path)
close()
As it found out, I need to use it on another machine, with different directories tree. There should also be a possibility to call script from other location, than the one where it is placed.
Changing all paths to absolute will take some time, but is not impossible. On the other hand, I come up with an idea to change path on the beginning of the script, to /home/user/X (I did it by adding one more python file to X directory and calling inside of it abs_path = os.path.abspath("."), then in script.py I do os.chdir(second_script.abs_path).
What are disadvantages of second solution? Are there any situations, where it can crash?
To be precise, in the second solution, i'm doing sth line this:
/
home/
user/
X/
scripy.py
second_scripy.py
Y/
a.c
b.c
user#user:~$ pwd
/home/
user#user:~$ python user/X/script.py
To make programs reusable, they are expected to work relative to the current working directory, not change the cwd and not relative to the location of the program. So you should use something like:
# relative_path_to_a = 'Y/a.c'
# relative_path_to_c = 'Y/b.c'
def f1(relative_path_to_a, relative_path_to_c):
c_abs_path = os.path.abspath(relative_path_to_c)
a_abs_path = os.path.abspath(relative_path_to_a)
with open(a_abs_path) as file_a:
file_a.write(c_abs_path)
this means for user X you run
(cd /user/X && python script.py) # modifies /user/X/Y/a.c
and for user Z you can reuse the script using
(cd /user/Z && python ../X/script.py) # modifies /user/Z/Y/a.c
Related
Hit an interesting case with pathlib where I am using it to represent paths on a android device, not the machine the python is actively running on. Is it still possible to make use of pathlib’s sugary syntax and ability to resolve unix-wide truisms such as relative paths “../../“ without resolving device specific symlink things like “/tmp/path” -> “/private/tmp/path”?
For the most part, pathlib makes handling paths on the device super easy, but i run into problems when wanting to resolve a path on the device without using the host python machine’s symlink resolving mechanisms.
I love using pathlib instead of strings. Is it possible to do path manipulations and actions with the Path object but then send off the final file command to the device ssh’d into?
The only piece from resolve i’d like is the ability to turn (Path(“/tmp/analysis/ptool”) / “../../“).resolve take care of the .. but not change /tmp to /private/tmp, which is not a symlink on the device i’m about to use this path command with.
The example is a little trivial “just do it with strings”, but the organization and automation i want to apply this on is significantly cleaner and easier to read as Path objects instead of strings. If possible, it would be great to make it work.
In [1]: import pathlib
In [2]: from pathlib import Path
In [3]: Path('/tmp/hello').resolve()
Out[3]: PosixPath('/private/tmp/hello')
MacOS, Python3.7.3
/private/tmp is where /tmp contents are actually stored on Mac OS. On this platform, /tmp is a symlink. You can see this from the operating system, without using Python at all:
$ ls -l /tmp
lrwxr-xr-x 1 root wheel 11 Oct 22 2018 /tmp -> private/tmp
$ (cd /tmp && pwd -P)
/private/tmp
pathlib.Path.resolve() replaces symlinks with the absolute paths they point to, so it's doing exactly what you're asking for.
See Why is /tmp a symlink to /private/tmp? on our sister site Ask Different.
using a whole Path in this case seems wrong as it exposes methods like stat which will try and operate on your local system, rather than the remote device. using a PurePosixPath looks better as it doesn't expose as much, and it's easy to add in a resolve method in a derived class:
from pathlib import PurePosixPath
class ResolvingPosixPath(PurePosixPath):
def resolve(self):
if not self.is_absolute():
raise ValueError('only able to resolve absolute paths')
sep = self._flavour.sep
path = ''
for name in str(self).split(sep):
if not name or name == '.':
# current dir
continue
if name == '..':
# parent dir
path, _, _ = path.rpartition(sep)
continue
path = path + sep + name
return path or sep
I've ripped the relevant code from pathlib._PosixFlavour.resolve, and
you're obviously free to use shorter class names if that's convenient!
this can be used as you suggest you want:
hello = ResolvingPosixPath('/tmp') / 'hello'
print(hello.resolve())
giving me '/tmp/hello'
I have converted my python program to exe using pyinstaller. My exe creates a temporary folder _MEIxxxxx to keep files. In the same folder there is a file which is being changed by the program, but unfortunately it is not happening.
In the program I change the folder to go to above folder:
os.chdir('C:\\Users\\Public')
for foldername in os.listdir():
if foldername.startswith('_MEI'):
myfolder = foldername
os.chdir('C:\\Users\\Public'+myfolder+'\\Quiz')
Thanks in advance.
this doesn't work:
os.chdir('C:\\Users\\Public'+myfolder+'\\Quiz')
because myfolder doesn't contain a \ at start.
Don't hardcode C:\Users\Public, use PUBLIC env. var
And avoid chdir, it's equivalent as a global variable shared between all modules. What if some module needs one current dir, and the other another?
Your attempt looks more like a shell script ported to python cd xxx; ls; .... Break this habit.
Use absolute paths/paths passed as parameter instead. My proposal:
pubdir = os.getenv("PUBLIC")
for foldername in os.listdir(pubdir):
if foldername.startswith('_MEI'):
myfolder = foldername
quizdir = os.path.join(pubdir,myfolder,'Quiz')
# now do something with quizdir
and if you need an absolute directory to run a system call, subprocess functions have a cwd parameter for that. So you can avoid os.chdir 99% of the time.
Is there a way to use os.chdir() to go to relative user folder?
I'm making a bash and the only issue I found is the cd ~, arg[0] is undefined since I'm using this cd functions:
def cd(args):
os.chdir(args[0])
return current_status
Which I want to change to
def cd(args):
if args[0] == '~':
os.chdir('/home/')
# Here I left it to /home/ since I don't know how
# to get the user's folder name
else:
os.chdir(args[0])
return current_status
No, os.chdir won't do that, since it is just a thin wrapper around a system call. Consider that ~ is actually a legal name for a directory.
You can, however, use os.expanduser to expand ~ in a path.
def cd(path):
os.chdir(os.path.expanduser(path))
Note that this will also expand ~user to the home directory for user.
Whenever I would save a file in Python, it would save in the present working directory. For some unknown reason, Python has stopped saving in the pwd. My figures are now being saved in the directory below it. So instead of saving in /Documents/.../OrbitalMechanics/OrbitalNotes they are now being saved in /Documents/.../OrbitalMechanics. My operating system is Ubuntu 13.04, I am using Emacs, and writing the program from the terminal. When I am calling my .py file, the path is emacs ~/Documents/.../OrbitalMechanics/OrbitalNotes/stumpff.py. Therefore, I am in the correct directory.
What would cause this problem? I haven't altered anything I normal do when I save a figure.
The program where this is occurred is below:
#!/usr/bin/env ipython
# This program plots the Stumpff functions C(z) and S(z)
import numpy as np
import pylab
from matplotlib.ticker import MaxNLocator
def C(z):
if z > 0:
return (1 - np.cos(z ** 0.5)) / z
elif z < 0:
return (np.cosh(np.sqrt(-z)) - 1) / -z
return 0.5
def S(z):
if z > 0:
return (np.sqrt(z) - np.sin(z ** 0.5)) / np.sqrt(z) ** 3
elif z < 0:
return (np.sinh(np.sqrt(-z)) - np.sqrt(-z)) / np.sqrt(-z) ** 3
return 1.0 / 6.0
vC = np.vectorize(C)
vS = np.vectorize(S)
z = np.linspace(-50.0, 500.0, 500000.0)
y = vC(z)
y2 = vS(z)
fig = pylab.figure()
ax = fig.add_subplot(111)
ax.plot(z, y, 'r')
ax.plot(z, y2, 'b')
pylab.legend(('$C(z)$', '$S(z)$'), loc = 0)
pylab.xlim((-50, 0))
pylab.ylim((0, 15))
pylab.xlabel('$z$')
pylab.gca().xaxis.set_major_locator(MaxNLocator(prune = 'lower'))
pylab.savefig('stumpffneg50to0.eps', format = 'eps')
Edit:
Previously, when I saved a figure, the figure would save in the directory where the .py file was located even if I wasn't in that directory. That is, I could be in the home directory, run the file in emacs, and the file would be where I wanted it to save. Now, this isn't the case even though the file used to always save in the location of the python script.
If I run the file from the terminal as ipython ~/path/to/file, the figures will save where the .py file is located.
If I open the file in emacs and use C-c C-c to run the file, the files will save one level down from the directory the .py file is in.
If I cd to the directory then open the file in emacs, the file again will save one level down.
When you start Python, it inherits its current working directory from the shell. So, for example:
$ cd ~
$ ~/Documents/.../OrbitalMechanics/OrbitalNotes/stumpff.py
In this case, the current working directory is ~, not ~/Documents/.../OrbitalMechanics/OrbitalNotes/.
This means that any relative paths are relative to ~, so if you just create a file named foo.data, it ends up as ~/foo.data.
So, what if you want to get files into paths like ~/Documents/.../OrbitalMechanics/OrbitalNotes/foo.data? There are two basic ways to do it—either set the cwd to the script path, or use the script path explicitly. And there are two ways to do the first part, from outside or inside of the Python script. And… well, let's just go over all the variations one by one.
If you want to launch the script from its own working directory, you can of course manually cd to that directory first, or you can write a bash alias or function or script that does something like:
$(cd $(dirname $1) ./$(basename $1))
Obviously you'll want to get a bit fancier if you want to pass multiple arguments, or if you want to be able to use it on python ~/Documents/.../OrbitalMechanics/OrbitalNotes/stumpff.py instead of just ~/Documents/.../OrbitalMechanics/OrbitalNotes/stumpff.py so the script is $2, etc. But it's all basic sh stuff from here.
If you want the script to change its own working directory, you can do that at the start of the script:
import os
import sys
scriptpath = os.path.dirname(__file__)
os.chdir(scriptpath)
(In some cases, you might want to use sys.argv[1] instead of __file__, but the details are hard to explain. If you want to learn, you can read the docs for each. If you don't want to learn, and you run into a problem, print out __file__ and see if it looks right, and, if not, try the other one.)
Or, you can just explicitly use scriptpath as a base instead of using cwd-relative paths. In this case, you'll probably want to use the absolute path to your script as a base (so later calls to chdir won't affect you):
import os
import sys
scriptpath = os.path.abspath(os.dirname(__file__))
# ...
datapath = os.path.join(scriptpath, 'foo.data')
with open(datapath, 'w') as datafile:
# ...
I have three modules as:
one.py:
def abc():
print "Heeeeeeeeeeeiiiiiioooooooooo"
two.py:
import one
def defg():
one.abc()
three.py:
import os
from time import sleep
import two
two.defg()
sleep(20)
directory = os.listdir('.')
for filename in directory:
if filename[-3:] == 'pyc':
print '- ' + filename
os.remove(filename)
I have three doubts.
When I run three.py for the first time one.pyc and two.pyc will be created. I can see it since I gave 20 sec delay. After executing the statement os.remove(filename), they get removed. Until here its fine.
Again without closing the IDLE as well as the script, I ran three.py. This time no .pyc file was created. Why is this so?
If I close IDLE as well as the script, .pyc will be created as before.
Why the compiled code is not getting created again and again?
Also, if I make a change in one.py it will not be shown if I run without closing the shells. I need a solution for this also.
Third doubt is if the compiled code is getting deleted first time itself, then how the second run gives the same result without .pyc?
The .pyc is not getting created again because there is a reference to your imported module in code. When it is re-run, this reference is used.
This is why the .pyc is not generated again, and also why the extra changes you make are not getting run.
You can either remove all references and call a garbage collect, or use the built in reload() function on the modules. e.g.:
reload(three)
I think that IDLE is caching the bytecode within its own Python process, so it doesn't need to regenerate the it each time the file is run.
Edit ~/.bashrc and add this shell function to it
$ cd ; pyclean
pyclean () {
find . -type f -name "*.py[co]" -delete
find . -type d -name "__pycache__" -delete
}