How do you define a local path in python - python

Good morning, I can indicate how to enter a path of internal hard disk in python, currently use the statement:
file = GETfile() or 'http://**********'
I would like to put a path to a local file, but it does not work, where am I wrong?
file = GETfile() or 'D:\xxx\xxxx\playlist\playlist.m3u'

\ is a escape character. You have three options.
1) use /. This, as a bonus works for linux as well:
'D:/xxx/xxxx/playlist/playlist.m3u'
2) escape the backslash
'D:\\xxx\\xxxx\\playlist\\playlist.m3u'
3) use raw strings:
r'D:\xxx\xxxx\playlist\playlist.m3u'

A correct answer is already given, but some additional information when working with local drive paths on Windows operating system.
Personally I would go with the r'D:\dir\subdir\filename.ext' format, however the other two methods already mentioned are valid as well.
Furthermore, file operations on Windows are limited by Explorer to a 256 character limit. Longer path names will usually result in an OS error.
However there is a workaround, by pre fixing "\\?\" to a long path.
Example of a path which does not work:
D:\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\filename.ext
Same file path which does work:
\\?\D:\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\reallyreallyreallyreallyreallylonglonglonglongdir\filename.ext
so the following code I use to change filenames to include the "\\?\":
import os
import platform
def full_path_windows(filepath):
if platform.system() == 'Windows':
if filepath[1:3] == ':\\':
return u'\\\\?\\' + os.path.normcase(filepath)
return os.path.normcase(filepath)
I use this for every path to file (or directories), it will return the path with a prefix. The path does not need to exist; so you can use this also before you create a file or directory, to ensure you are not running into the Windows Explorer limitations.
HTH

Related

Identify String as Windows or UNIX like path in Python

How do I best identify if a String is a Windows path or a UNIX style path?
Example Strings:
some_path = '/Volumes/network-drive/file.txt'
or
some_path = 'Z:\\network-drive\\file.txt'
One way is to check which slashes the String contains:
if '/' in some_path:
# do something with UNIX Style Path
elif '\\' in some_path:
# do something else with Windows Path
Is there a better way to do this? I couldn't find suited methods in os.path or pathlib. BTW, assume that the path string will come from another system so it doesn't help to check on which OS my code runs on.

os.path.split seems to be returning wrong

I can't understand what os.path.split is doing. I'm debugging a program (specifically git's interface with Perforce: git-p4) and seeing that os.path.split is splitting the incoming path in ways the script isn't expecting, and also seems inconsistent with the documentation. I made some simpler tests and can't figure out what it's doing myself.
The path I want to split is //a/b (The path is actually a Perforce path, not a local filesystem path), and I need b in the second half of the returned pair. I'm running on Windows, and suspect the issue has something to do with the path not looking very Windows-esque. When I tried running my test code in an online sandbox it worked as expected unlike my Windows machine.
I've read the documentation:
os.path.split(path)
Split the pathname path into a pair, (head, tail) where tail is the last pathname component and head is everything leading up to that. The tail part will never contain a slash; if path ends in a slash, tail will be empty. If there is no slash in path, head will be empty. If path is empty, both head and tail are empty. Trailing slashes are stripped from head unless it is the root (one or more slashes only). In all cases, join(head, tail) returns a path to the same location as path (but the strings may differ). Also see the functions dirname() and basename().
My test code:
import os
print os.path.split("//a")
print os.path.split("//a/b")
print os.path.split("//a/b/c")
What I'd expect:
('//', 'a')
('//a', 'b')
('//a/b', 'c')
What I actually get on a couple online sandboxes:
('//', 'a')
('//a', 'b')
('//a/b', 'c')
What I actually get on my PC:
('//', 'a')
('//a/b', '')
('//a/b/', 'c')
Python 2 because the git-p4 code is written for Python 2.
So my first question is just for my own understanding. What's going wrong here? An OS difference?
And then beyond my own curiosity, I need a fix. I've been able to modify git-p4, but I'd of course prefer to edit it as little as possible as I'm not trying to understand it! I'm not a python expert. Is there a comparable method that can get ('//a', 'b') returned?
You are using the wrong tool to handle these paths. On Windows, paths that start with //foo/bar or \\foo\bar are seen as UNC network paths, and os.path.split() will first use os.path.splitdrive() to make sure the UNC portion is not split. The UNC or drive portion is then re-attached after splitting the remainder.
You can use the posixpath module instead, to get the POSIX behaviour:
import posixpath
posixpath.split(yourpaths)
Quoting from the top of the os.path module documentation:
Note: Since different operating systems have different path name conventions, there are several versions of this module in the standard library. The os.path module is always the path module suitable for the operating system Python is running on, and therefore usable for local paths. However, you can also import and use the individual modules if you want to manipulate a path that is always in one of the different formats. They all have the same interface:
posixpath for UNIX-style paths
ntpath for Windows paths
[...]
On Windows, os.path is the same module as ntpath, the online sandboxes must all have been POSIX systems.
Treating your Perforce paths as POSIX paths is fine, provided you always use forward slashes as path separators.

Resolving mixed slashes from sys.path and os.path.join

I need to resolve a disparity between the separator that sys.path is providing, and the separator that os.path.join is using.
I mimicked this Esri method (Techniques for sharing Python scripts) to make my script portable. It is being used in Windows for now, but will eventually live on a Linux server; I need to let Python determine the appropriate slash.
What they suggest:
# Get the pathname to this script
scriptPath = sys.path[0]
# Get the pathname to the ToolShare folder
toolSharePath = os.path.dirname(scriptPath)
# Now construct pathname to the ToolData folder
toolDataPath = os.path.join(toolSharePath, "ToolData")
print "ToolData folder: " + toolDataPath
But this outputs ToolData folder: C:/gis\ToolData -- and obviously the mixed slashes aren't going to work.
This Question (mixed slashes with os.path.join on windows) includes the basic approach to a solution:
check your external input (the input you apparently do not control the format of) before putting it in os.path.join. This way you make sure that os.path.join does not make bad decisions based on possibly bad input
However, I'm unsure how to ensure that it will work cross-platform. If I use .replace("/","\\") on the sys.path[0] result, that's great for Windows, but isn't that going to cause the same mixed-slash problem once I transition to Unix?
How about using os.path.normpath()?
>>> import os
>>> os.path.normpath(r'c:\my/path\to/something.py')
'c:\\my\\path\\to\\something.py'
Also worth mentioning: the Windows path API doesn't care whether forward or back slashes are used. Usually it's the fault of program that doesn't handle the slashing properly. For example, in python:
with open(r'c:/path/to/my/file.py') as f:
print f.read()
will work.
After reading the documentation and trying a lot of variations:
The os.path.abspath function can "clean" the slashes, so whichever direction slash sys.path[0] decides to use, the slashes will be replaced with the preferred separator.
scriptPath = sys.path[0]
toolDataPath = os.path.join(scriptPath, "ToolData")
Result: C:/gis\ToolData
scriptPath = sys.path[0]
toolSharePath = os.path.abspath(scriptPath)
# or, in one line: toolSharePath = os.path.abspath(sys.path[0])
toolDataPath = os.path.join(toolSharePath, "ToolData")
Result: C:\gis\ToolData
There is an os.sep character in Python, which stores your OS's preferred folder separating character. Perhaps you could perform a manual string join using that?
On Linux:
>>> import os
>>> os.sep
'/'
https://docs.python.org/2/library/os.html#os.sep

os.listdir can't see my directory

I am working on a python script that installs an 802.1x certificate on a Windows 8.1 machine. This script works fine on Windows 8 and Windows XP (haven't tried it on other machines).
I have isolated the issue. It has to do with clearing out the folder
"C:\Windows\system32\config\systemprofile\AppData\LocalLow\Microsoft\CryptURLCache\Content"
The problem is that I am using the module os and the command listdir on this folder to delete each file in it. However, listdir errors, saying the folder does not exist, when it does indeed exist.
The issue seems to be that os.listdir cannot see the LocalLow folder. If I make a two line script:
import os
os.listdir("C:\Windows\System32\config\systemprofile\AppData")
It shows the following result:
['Local', 'Roaming']
As you can see, LocalLow is missing.
I thought it might be a permissions issue, but I am having serious trouble figuring out what a next step might be. I am running the process as an administrator from the command line, and it simply doesn't see the folder.
Thanks in advance!
Edit: changing the string to r"C:\Windows\System32\config\systemprofile\AppData", "C:\Windows\System32\config\systemprofile\AppData", or C:/Windows/System32/config/systemprofile/AppData" all produce identical results
Edit: Another unusual wrinkle in this issue: If I manually create a new directory in that location I am unable to see it through os.listdir either. In addition, I cannot browse to the LocalLow or my New Folder through the "Save As.." command in Notepad++
I'm starting to think this is a bug in Windows 8.1 preview.
I encountered this issue recently.
I found it's caused by Windows file system redirector
and you can check out following python snippet
import ctypes
class disable_file_system_redirection:
_disable = ctypes.windll.kernel32.Wow64DisableWow64FsRedirection
_revert = ctypes.windll.kernel32.Wow64RevertWow64FsRedirection
def __enter__(self):
self.old_value = ctypes.c_long()
self.success = self._disable(ctypes.byref(self.old_value))
def __exit__(self, type, value, traceback):
if self.success:
self._revert(self.old_value)
#Example usage
import os
path = 'C:\\Windows\\System32\\config\\systemprofile\\AppData'
print os.listdir(path)
with disable_file_system_redirection():
print os.listdir(path)
print os.listdir(path)
ref : http://code.activestate.com/recipes/578035-disable-file-system-redirector/
You must have escape sequences in your path. You should use a raw string for file/directory paths:
# By putting the 'r' at the start, I make this string a raw string
# Raw strings do not process escape sequences
r"C:\path\to\file"
or put the slashes the other way:
"C:/path/to/file"
or escape the slashes:
# You probably won't want this method because it makes your paths huge
# I just listed it because it *does* work
"C:\\path\\to\\file"
I'm curious as to how you are able to list the contents with those two lines. You are using escape sequences \W, \S, \c, \s, \A in your code. Try escaping the back slash like this:
import os
os.listdir('C:\\Windows\\System32\\config\\systemprofile\\AppData')

How do I use '~' (tilde) in the context of paths?

To fix a problem in code for work, I was told to "use a path relative to ~". What does ~ mean in a file path? How can I make a path that is relative to ~, and use that path to open files in Python?
it is your $HOME var in UNIX, which usually is /home/username.
"Your home" meaning the home of the user who's executing a command like cd ~/MyDocuments/ is cd /home/user_executing_cd_commnd/MyDocuments
Unless you're writing a shell script or using some other language that knows to substitute the value of $HOME for ~, tildes in file paths have no special meaning and will be treated as any other non-special character.
If you are writing a shell script, shells don't interpret tildes unless they occur as the first character in an argument. In other words, ~/file will become /path/to/users/home/directory/file, but ./~/file will be interpreted literally (i.e., "a file called file in a subdirectory of . called ~").
Used in URLs, interpretation of the tilde as a shorthand for a user's home directory (e.g., http://www.foo.org/~bob) is a convention borrowed from Unix. Implementation is entirely server-specific, so you'd need to check the documentation for your web server to see if it has any special meaning.
If you are using pathlib for filenames then you can use on both Windows and Linux (I came here for a windows answer):
from pathlib import Path
p = Path('~').expanduser()
print(p)

Categories