How do I best identify if a String is a Windows path or a UNIX style path?
Example Strings:
some_path = '/Volumes/network-drive/file.txt'
or
some_path = 'Z:\\network-drive\\file.txt'
One way is to check which slashes the String contains:
if '/' in some_path:
# do something with UNIX Style Path
elif '\\' in some_path:
# do something else with Windows Path
Is there a better way to do this? I couldn't find suited methods in os.path or pathlib. BTW, assume that the path string will come from another system so it doesn't help to check on which OS my code runs on.
I need to resolve a disparity between the separator that sys.path is providing, and the separator that os.path.join is using.
I mimicked this Esri method (Techniques for sharing Python scripts) to make my script portable. It is being used in Windows for now, but will eventually live on a Linux server; I need to let Python determine the appropriate slash.
What they suggest:
# Get the pathname to this script
scriptPath = sys.path[0]
# Get the pathname to the ToolShare folder
toolSharePath = os.path.dirname(scriptPath)
# Now construct pathname to the ToolData folder
toolDataPath = os.path.join(toolSharePath, "ToolData")
print "ToolData folder: " + toolDataPath
But this outputs ToolData folder: C:/gis\ToolData -- and obviously the mixed slashes aren't going to work.
This Question (mixed slashes with os.path.join on windows) includes the basic approach to a solution:
check your external input (the input you apparently do not control the format of) before putting it in os.path.join. This way you make sure that os.path.join does not make bad decisions based on possibly bad input
However, I'm unsure how to ensure that it will work cross-platform. If I use .replace("/","\\") on the sys.path[0] result, that's great for Windows, but isn't that going to cause the same mixed-slash problem once I transition to Unix?
How about using os.path.normpath()?
>>> import os
>>> os.path.normpath(r'c:\my/path\to/something.py')
'c:\\my\\path\\to\\something.py'
Also worth mentioning: the Windows path API doesn't care whether forward or back slashes are used. Usually it's the fault of program that doesn't handle the slashing properly. For example, in python:
with open(r'c:/path/to/my/file.py') as f:
print f.read()
will work.
After reading the documentation and trying a lot of variations:
The os.path.abspath function can "clean" the slashes, so whichever direction slash sys.path[0] decides to use, the slashes will be replaced with the preferred separator.
scriptPath = sys.path[0]
toolDataPath = os.path.join(scriptPath, "ToolData")
Result: C:/gis\ToolData
scriptPath = sys.path[0]
toolSharePath = os.path.abspath(scriptPath)
# or, in one line: toolSharePath = os.path.abspath(sys.path[0])
toolDataPath = os.path.join(toolSharePath, "ToolData")
Result: C:\gis\ToolData
There is an os.sep character in Python, which stores your OS's preferred folder separating character. Perhaps you could perform a manual string join using that?
On Linux:
>>> import os
>>> os.sep
'/'
https://docs.python.org/2/library/os.html#os.sep
I use python to write scripts for Autodesk Maya. Maya is a cross-platform software and internally use forward slash. If I use os.path.join operation on windows it can result paths like this:
e:/Test\\TemplatePicture.jpg
My idea is that as long as I don't use ms-dos commands easier way to join path parts like this:
pathPart1 = "e:"
pathPart2 = "Test"
pathPart3 = "TemplatePicture.jpg"
path = "s%/s%/s%" % (pathPart1, pathPart2, pathPart3)
Is there something that makes it a bad idea?
When you import os, python will create an os.path specific to your platform. On linux its posixpath and on windows its ntpath. When you are working on Maya paths, use posixpath. It will follow linux conventions even on windows. When you need to go native, convert using the realpath for your current system.
import os
import posixpath
maya_path = posixpath.join('a','b','c')
local_path = os.path.realpath(maya_path)
I don't see any problems with this.
In fact there is a related question here.
To summarize the discussion within the provided link - you either let python handle file paths or you do it all yourself
They are same, but which one should I use?
http://docs.python.org/library/os.html:
os.sep
The character used by the operating system to separate pathname components. This is '/' for POSIX and '\' for Windows. Note that knowing this is not sufficient to be able to parse or concatenate pathnames — use os.path.split() and os.path.join() — but it is occasionally useful. Also available via os.path.
I'd use os.path.sep to make it very clear that it's the path separator… But consistency is more important, so if one is already being used, use that. Otherwise, pick one and use it all the time.
Edit: Just to make sure you're not reinventing the wheel, though, the path module already has join, split, dirname, and basename functions… So you should rarely need to use path.sep:
>>> os.path.join("foo", "bar", "baz")
'foo/bar/baz'
>>> os.path.split(_)
('foo/bar', 'baz')
I recommend you use os.path.sep for clarity, since it's a path separator, not an OS separator. If you import os.path as path you can call it path.sep, which is even better.
If you are using Python 2.7, I suggest using os.sep (works) instead of os.path.sep (broken) as Jython on Windows has a bug returning a "/" slash instead of the required "\" backslash.
The following examples could highlight the differences between os.path.join and os.path.sep.join.
>>> import os
>>> os.path.join("output", "images", "saved")
'output/images/saved'
>>> os.path.sep.join(["output", "images", "saved"])
'output/images/saved'
I guess the os.path.sep.join is more robust and can be used w/o modifications for any os.
As mentioned in the python docs, both os.path.sep and os.sep return the same output.
The character used by the operating system to separate pathname
components. This is '/' for POSIX and '\' for Windows.
Both of them belongs to the same python class also.
print(type(os.sep))
print(type(os.path.sep))
# Output
<class 'str'>
<class 'str'>
Both have them have the same documentation.
print(os.path.sep.__doc__)
print(os.sep.__doc__)
# The outputs of both print statements are the same.
So, I think after Python2 where we mostly used os.sep, in Python3 only the consistency matters as far as their uses are concerned.
Can all paths in a Python program use ".." (for the parent directory) and / (for separating path components), and still work whatever the platform?
On one hand, I have never seen such a claim in the documentation (I may have missed it), and the os and os.path modules do provide facilities for handling paths in a platform agnostic way (os.pardir, os.path.join,…), which lets me think that they are here for a reason.
On the other hand, you can read on StackOverflow that "../path/to/file" works on all platforms…
So, should os.pardir, os.path.join and friends always be used, for portability purposes, or are Unix path names always safe (up to possible character encoding issues)? or maybe "almost always" safe (i.e. working under Windows, OS X, and Linux)?
I've never had any problems with using .., although it might be a good idea to convert it to an absolute path using os.path.abspath. Secondly, I would recommend always using os.path.join whereever possible. There are a lot of corner cases (aside from portability issues) in joining paths, and it's good not to have to worry about them. For instance:
>>> '/foo/bar/' + 'qux'
'/foo/bar/qux'
>>> '/foo/bar' + 'qux'
'/foo/barqux'
>>> from os.path import join
>>> join('/foo/bar/', 'qux')
'/foo/bar/qux'
>>> join('/foo/bar', 'qux')
'/foo/bar/qux'
You may run into problems with using .. if you're on some obscure platforms, but I can't name any (Windows, *nix, and OS X all support that notation).
"Almost always safe" is right. All of the platforms you care about probably work ok today and I don't think they will be changing their conventions any time soon.
However Python is very portable and runs on a lot more than the usual platforms. The reason for the os module is to help smooth things over it a platform does have different requirements.
Is there a good reason for you to not use the os functions?
os.pardir is self documenting whereas ".." isn't, and os.pardir might be easier to grep for
Here is some docs from python 1.6 when Mac was still different for everything
OS routines for Mac, DOS, NT, or Posix depending on what system we're
on.
This exports:
- all functions from posix, nt, dos, os2, mac, or ce, e.g. unlink, stat, etc.
- os.path is one of the modules posixpath, ntpath, macpath, or dospath
- os.name is 'posix', 'nt', 'dos', 'os2', 'mac', or 'ce'
- os.curdir is a string representing the current directory ('.' or ':')
- os.pardir is a string representing the parent directory ('..' or '::')
- os.sep is the (or a most common) pathname separator ('/' or ':' or '\')
- os.altsep is the alternate pathname separator (None or '/')
- os.pathsep is the component separator used in $PATH etc
- os.linesep is the line separator in text files (' ' or ' ' or ' ')
- os.defpath is the default search path for executables
Programs that import and use 'os' stand a better chance of being
portable between different platforms. Of course, they must then only
use functions that are defined by all platforms (e.g., unlink and
opendir), and leave all pathname manipulation to os.path (e.g., split
and join).
Within python, using / will always work. You will need to be aware of the OS convention if you want to execute a command in a subshell
myprog = "/path/to/my/program"
os.system([myprog, "-n"]) # 1
os.system([myprog, "C:/input/file/to/myprog"]) # 2
Command #1 will probably work as expected.
Command #2 might not work if myprog is a Windows command and expects to parse its command line arguments to get a Windows file name.
It works on Windows, so if you define "whatever the platform" to be Unix and Windows, you're fine.
On the other hand, Python also runs on VMS, RISC OS, and other odd platforms that use completely different filename conventions. However, it's probable that trying to get your application to run on VMS, blind, is kind of silly anyway - "premature portability is the root of some relatively minor evil"
I like using the os.path functions anyway because they are good for expressing intent - instead of just a string concatenation, which might be done for any of a million purposes, it reads very explicitly as a path manipulation.
Windows supports / as a path separator. The only incompatibilities between Unix filenames and Windows filenames are:
the allowed characters in filenames
the special names and
case sensitivity
Windows is more restrictive in the first two accounts (this is, it has more forbidden characters and more special names), while Unix is typically case sensitive. There are some answers here listing exactly what are these characters and names. I'll see if I can find them.
Now, if your development environment comes with a function to create or manipulate paths, you should use it, it's there for a reason, y'know. Especially given that there are a lot more platforms than Windows and Unix.
Answering your first question, yes ../dir/file will work, unless they hit some of the above mentioned incompatibilities.
OS/X and Linux are both Unix compatible, so by definition they use the format you gave at the beginning of the question. Windows allows "/" in addition to "\" so that programs could be interchangeable with Xenix, a Unix variant that Microsoft was trying out a long time ago, and that compatibility has been carried forward to the present. Thus it works too.
I don't know how many other platforms Python has been ported to, and I can't speak for them.
As others have said, a forward slash will work in all cases, but you're better off creating a list of path segments and os.path.join()-ing them.