This question already has answers here:
os.makedirs doesn't understand "~" in my path
(3 answers)
Closed 6 months ago.
I am trying to read a file in another directory. My current script is in path/to/dir. However, I want to read a file in ~/. I'm not sure how to do this.
I tried f = open("~/.file") but I am getting an error
IOError: [Errno 2] No such file or directory: '~/.file'
use os.path.expanduser:
import os
with open(os.path.expanduser('~/.file')) as f:
print f.read()
This should work
from os.path import expanduser
home = expanduser("~")
os.path.expanduser will help you
The short answer is: use os.path.expanduser, as m.wasowski shows:
f = open(os.path.expanduser("~/.file"))
But why do you have to do this?
Well, ~/.file doesn't actually mean what you think it does. It's just asking for a file named .file in a subdirectory named ~ in the current working directory.
Why does this work on the command line? Bcause shells don't just pass your arguments along as typed, they do all kinds of complicated processing.
This is why ./myscript.py *txt gives you ['./myscript.py', 'a.txt', 'b.txt', 'c.txt', 'd e f.txt'] in your argv—because the shell does glob expansion on *.txt by looking in the directory for everything that matches that pattern and turns it into a.txt b.txt c.txt "d e f.txt".
In the same way, it expands ~/.file into /Users/me/.file by looking at the appropriate environment variables and platform-specific defaults and so on. You can read the gory details for bash; things are similar with dash, tcsh, and most other shells (except Windows cmd and PowerShell).
Python talks to the filesystem directly, it doesn't go through the shell to do it. So, if you want to do the same things shells do, you have to do them explicitly: glob.glob, os.path.expanduser, os.path.expandvars, subprocess.check_output, etc.
The cool thing about doing it all in Python is that it works the same way everywhere. Even on Windows, where the shell doesn't know what ~/.file means, and the right answer may be some hideous thing like 'C:\\Documents and Settings\\me\\Documents' or '\\\\FileServer\\RoamingProfiles\\me', os.path.expanduser will give you the right hideous thing.
Related
I have a list of files on my filesystem which I'd like to chmod to 664 via python.
On of the filenames/dirpaths (I am not allowed to change the filename nor dirpaths!!!) is:
/home/media/Music/Ke$ha/song.mp3 (NOTE $ is a literal, not a variable!)
I receive the files in a list: ['/some/path/file1', '/some/otherpath/file2', etc...]
If I try to run the following code:
files = ['/home/media/Music/Ke$ha/song.mp3']
for file in files:
os.chmod(file, 0664)
It complains that it cannot find /home/media/Music/Ke$ha/song.mp3. Most likely (I guess) because the called shell tries to expand $ha, which is obviously wrong.
The 'Ke$ha' file is just an example, there are many more files with escape characters in it (e.g. /home/media/Music/Hill's fire/song.mp3)
The question I have is: How can I elegantly convince python and/or the shell to handle these files properly?
Kind regards,
Robert Nagtegaal.
You can do like this
files=["/home/media/Music/Ke$ha/song.mp3", "/home/media/Music/Hill's fire/song.mp3"]
import os,re
os.system("chmod 777 " + re.escape(files[i]))
How about this, a raw string? Also is your username 'media'?
files = [r'/home/media/Music/Ke$ha/song.mp3']
I am trying to make a python script that will open a directory, apply a perl script to every file in that directory and get its out put in either multiple text files or just one.
I currently have:
import shlex, subprocess
arg_str = "perl tilt.pl *.pdb > final.txt"
arg = shlex.split(arg_str)
import os
framespdb = os.listdir("prac_frames")
for frames in framespdb:
subprocess.Popen(arg, stdout=True)
I keep getting *.pdb not found. I am very new to all of this so any help trying to complete this script would help.
*.pdb not found means exactly that - there won't be a *.pdb in whatever directory you're running the script... and as I read the code - I don't see anything to imply it's within 'frames' when it runs the perl script.
you probably need os.chdir(path) before the Popen.
How do I "cd" in Python?
...using a python script to run somewhat dubious syscalls to perl may offend some people but everyone's done it.. aside from that I'd point out:
always specify full paths (this becomes a problem if you will later say, want to run your job automatically from cron or an environment that doesn't have your PATH).
i.e. 'which perl' - put that full path in.
./.pdb would be better but not as good as the fullpath/.pdb (which you could use instead of the os.chdir option).
subprocess.Popen(arg, stdout=True)
does not expand filename wildcards. To handle your *.pdb, use shell=True.
I am working on a python script that installs an 802.1x certificate on a Windows 8.1 machine. This script works fine on Windows 8 and Windows XP (haven't tried it on other machines).
I have isolated the issue. It has to do with clearing out the folder
"C:\Windows\system32\config\systemprofile\AppData\LocalLow\Microsoft\CryptURLCache\Content"
The problem is that I am using the module os and the command listdir on this folder to delete each file in it. However, listdir errors, saying the folder does not exist, when it does indeed exist.
The issue seems to be that os.listdir cannot see the LocalLow folder. If I make a two line script:
import os
os.listdir("C:\Windows\System32\config\systemprofile\AppData")
It shows the following result:
['Local', 'Roaming']
As you can see, LocalLow is missing.
I thought it might be a permissions issue, but I am having serious trouble figuring out what a next step might be. I am running the process as an administrator from the command line, and it simply doesn't see the folder.
Thanks in advance!
Edit: changing the string to r"C:\Windows\System32\config\systemprofile\AppData", "C:\Windows\System32\config\systemprofile\AppData", or C:/Windows/System32/config/systemprofile/AppData" all produce identical results
Edit: Another unusual wrinkle in this issue: If I manually create a new directory in that location I am unable to see it through os.listdir either. In addition, I cannot browse to the LocalLow or my New Folder through the "Save As.." command in Notepad++
I'm starting to think this is a bug in Windows 8.1 preview.
I encountered this issue recently.
I found it's caused by Windows file system redirector
and you can check out following python snippet
import ctypes
class disable_file_system_redirection:
_disable = ctypes.windll.kernel32.Wow64DisableWow64FsRedirection
_revert = ctypes.windll.kernel32.Wow64RevertWow64FsRedirection
def __enter__(self):
self.old_value = ctypes.c_long()
self.success = self._disable(ctypes.byref(self.old_value))
def __exit__(self, type, value, traceback):
if self.success:
self._revert(self.old_value)
#Example usage
import os
path = 'C:\\Windows\\System32\\config\\systemprofile\\AppData'
print os.listdir(path)
with disable_file_system_redirection():
print os.listdir(path)
print os.listdir(path)
ref : http://code.activestate.com/recipes/578035-disable-file-system-redirector/
You must have escape sequences in your path. You should use a raw string for file/directory paths:
# By putting the 'r' at the start, I make this string a raw string
# Raw strings do not process escape sequences
r"C:\path\to\file"
or put the slashes the other way:
"C:/path/to/file"
or escape the slashes:
# You probably won't want this method because it makes your paths huge
# I just listed it because it *does* work
"C:\\path\\to\\file"
I'm curious as to how you are able to list the contents with those two lines. You are using escape sequences \W, \S, \c, \s, \A in your code. Try escaping the back slash like this:
import os
os.listdir('C:\\Windows\\System32\\config\\systemprofile\\AppData')
import subprocess
sample_file_directory = "..." # directory where file is
SCRIPT_DIR = "..." # directory where script is
p = subprocess.Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
min_side_len, err = p.communicate()
print len(err)
So i have this script that analyzes .ply files (3d file format) and gives me data about the file. I am trying to find out in my directory which files are corrupted and which aren't. So I am trying to use the subprocess library to run the script to find an arbitrary feature of the .ply files (in this case the minimum edge length) If err has anything in it, it means it couldn't retrieve the arbitrary feature and the file is corrupted. Here, I am only running it on one file. However, I keep getting an error.
p = subprocess.Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Can anyone help me figure out why I am getting this error? Both my data directory and script directory are valid. Also, sorry if I left anything out that's important. First time posting.
Is your SCRIPT_DIR variable the absolute path or a relative path valid in the location you're running the script? If it's the latter, it may not work, as the subprocess may start in a different environment that has a different working directory.
The problem lies here:
p = subprocess.Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
Notice this part:
Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen']
Now, see, you already know that subprocess takes a list. But the problem is that you are passing the directory as part of the list. The list is formatted like this:
[command, arg1, arg2...]
Now, when you run that command, you are doing this:
[SCRIPT_DIR -> command
sample_file_directory -> arg1
'min' -> arg2
'edgelen' -> arg3]
See the problem? You are passing the script's directory as the command to run and the script's name as an argument. Import the os module and do this:
p = subprocess.Popen([os.path.join(SCRIPT_DIR,sample_file_directory),'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
This does this:
[SCRIPT_DIR and sample_file_directory joined together -> command
'min' -> arg1
'edgelen' -> arg2]
os.path.join automatically adds the path separator.
subprocess.Popen() won't combine path components for you. From what I see, you're only providing a directory. You'll need to add the target executable's name to the path being given to Popen. Try the following:
p = subprocess.Popen([os.path.join(SCRIPT_DIR,SCRIPT_NAME),sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
"One other thing it could potentially be is that you're trying to run a 32-bit binary on a 64-bit system without the 32-bit libraries installed. I actually ran into this issue earlier today. A fix is documented here. I don't know anything about your system, so this may be completely irrelevant advice, but I figured it wouldn't hurt to mention it." Dan Albert. He came up with the answer to the solution! Thanks so much Dan!
To fix a problem in code for work, I was told to "use a path relative to ~". What does ~ mean in a file path? How can I make a path that is relative to ~, and use that path to open files in Python?
it is your $HOME var in UNIX, which usually is /home/username.
"Your home" meaning the home of the user who's executing a command like cd ~/MyDocuments/ is cd /home/user_executing_cd_commnd/MyDocuments
Unless you're writing a shell script or using some other language that knows to substitute the value of $HOME for ~, tildes in file paths have no special meaning and will be treated as any other non-special character.
If you are writing a shell script, shells don't interpret tildes unless they occur as the first character in an argument. In other words, ~/file will become /path/to/users/home/directory/file, but ./~/file will be interpreted literally (i.e., "a file called file in a subdirectory of . called ~").
Used in URLs, interpretation of the tilde as a shorthand for a user's home directory (e.g., http://www.foo.org/~bob) is a convention borrowed from Unix. Implementation is entirely server-specific, so you'd need to check the documentation for your web server to see if it has any special meaning.
If you are using pathlib for filenames then you can use on both Windows and Linux (I came here for a windows answer):
from pathlib import Path
p = Path('~').expanduser()
print(p)