I am writing a script to parse multiple log files and maintain a list of the files that have been processed. When I read the list of files to process I use os.walk and get names similar to the following:
C:/Users/Python/Documents/Logs\ServerUI04\SystemOut_13.01.01_20.22.25.log
This is created by the following code:
filesToProcess.extend(os.path.join(root, filename) for filename in filenames if logFilePatternMatch.match(filename))
It appears that "root" used forward slashes as the separator (I am on Windows and find that more convenient) but "filename" uses backslashes so I end up with an inconsistent path for the file as it contains a mixture of forward and back slashes as separators.
I have tried setting the separator with:
os.path.sep = "/"
and
os.sep = "/"
Before the .join but it seems to have no effect. I realize that in theory I could manipulate the string but longer term I'd like my script to run on Unix as well as Windows so would prefer that it be dynamic if possible.
Am I missing something?
Update:
Based on the helpful responses below it looks like my problem was self inflicted, for convenience I had set the initial path used as root like this:
logFileFolder = ['C:/Users/Python/Documents/Logs']
When I changed it to this:
logFileFolder = ['C:\\Users\\Python\\Documents\\Logs']
Everything works and my resulting file paths all use the "\" throughout. It looks like my approach was wrong in that I was trying to get Python to change behavior rather than correcting what I was setting as a value.
Thank you!
I would keep my fingers off os.sep and use os.path.normpath() on the result of combining the root and a filename:
filesToProcess.extend(os.path.normpath(os.path.join(root, filename))
for filename in filenames if logFilePatternMatch.match(filename))
I have preferred the following utility function.
from os.path import sep, join
def pjoin(*args, **kwargs):
return join(*args, **kwargs).replace(sep, '/')
It converts both variations (linux style and windows style) to linux style. Both windows and linux supports '/' separator in python.
I rejected the simplistic os.sep.join(['str','str','str']) because it does not take into account existing separators. Take the following case with sep.join vs vanilla join:
In[79]: os.sep.join(['/existing/my/', 'short', 'path'])
Out[79]: '/existing/my/\\short\\path'
In[80]: os.path.join('/existing/my/', 'short', 'path')
Out[80]: '/existing/my/short\\path'
The vanilla join could be repaired with the suggested:
In[75]: os.path.normpath(os.path.join('/existing/my/', 'short', 'path'))
Out[75]: '\\existing\\my\\short\\path'
So far so good. But then we introduce the following scenario where we will be interacting with linux from windows.
local_path = os.path.normpath(os.path.join('C:\\local\\base', 'subdir', 'filename.txt'))
remote_path = os.path.normpath(os.path.join('/remote/base', 'subdir', 'filename.txt'))
sftp_server.upload(local_path, remote_path)
The above will then fail because the sftp server expects a '/' separator while os.path.normpath will on windows normalize to '\'.
By using the pjoin utility function or similar, it will work cross OS, web, ftp, etc.
I use '/'.join([path1, path2]) to solve this probelm, because '/' works well in windows and linux.
You are better off not touching os.sep and os.path.sep as they are not what os.path.join is using. You could use os.path.normpath as suggested by Anthon. Another alternative is to have your own simple path join:
os.sep.join([i1,i2,i3])
Related
I know similar questions are apearing all the time, but I have read all I can find.
Solutions there helped me a bit, but I still cannot build correct full path that contains whitespaces.
After research I found solution adding \ and take everyting in Single quote. So this path is acceptable, as you can see below.
import os
os.system('\"C:\Program Files (x86)\remar\remar.exe"')
But, the problem is that I have to add console commande to it, so the path have to look like that:
C:\Program Files (x86)\remar\remar.exe --break-in=null
I tried to concatenate several parts of the path, but result is unacceptable for os.system() expression.
I can't make something like that - "'\" + '"C:\Program Files' + ' ' + "(x86)\remar\remar.exe"'" + ' ' + '--break-in=null'
What shoud I do to get all parts together to one single correct path?
PS - I tried solutions from Windows path in Python , but it doesn't work for me. Maybe it's because python version. I'm using 3.61 BTW.
Try using os.path.join here. Don't end your path variable with a backslash, join will do that for you.
import os, subprocess
path = r"C:\Program Files (x86)\test"
filename = "test.exe"
result = os.path.join(path, filename)
print(result)
subprocess.run(result)
You may also find interest in pathlib Path
Given is a variable that contains a windows file path. I have to then go and read this file. The problem here is that the path contains escape characters, and I can't seem to get rid of it. I checked os.path and pathlib, but all expect the correct text formatting already, which I can't seem to construct.
For example this. Please note that fPath is given, so I cant prefix it with r for a rawpath.
#this is given, I cant rawpath it with r
fPath = "P:\python\t\temp.txt"
file = open(fPath, "r")
for line in file:
print (line)
How can I turn fPath via some function or method from:
"P:\python\t\temp.txt"
to
"P:/python/t/temp.txt"
I've tried also tried .replace("\","/"), which doesnt work.
I'm using Python 3.7 for this.
You can use os.path.abspath() to convert it:
print(os.path.abspath("P:\python\t\temp.txt"))
>>> P:/python/t/temp.txt
See the documentation of os.path here.
I've solved it.
The issues lies with the python interpreter. \t and all the others don't exist as such data, but are interpretations of nonprint characters.
So I got a bit lucky and someone else already faced the same problem and solved it with a hard brute-force method:
http://code.activestate.com/recipes/65211/
I just had to find it.
After that I have a raw string without escaped characters, and just need to run the simple replace() on it to get a workable path.
You can use Path function from pathlib library.
from pathlib import Path
docs_folder = Path("some_folder/some_folder/")
text_file = docs_folder / "some_file.txt"
f = open(text_file)
if you would like to do replace then do
replace("\\","/")
When using python version >= 3.4, the class Path from module pathlib offers a function called as_posix, which will sort of convert a path to *nix style path. For example, if you were to build Path object via p = pathlib.Path('C:\\Windows\\SysWOW64\\regedit.exe'), asking it for p.as_posix() it would yield C:/Windows/SysWOW64/regedit.exe. So to obtain a complete *nix style path, you'd need to convert the drive letter manually.
I came across similar problem with Windows file paths. This is what is working for me:
import os
file = input(str().split('\\')
file = '/'.join(file)
This gave me the input from this:
"D:\test.txt"
to this:
"D:/test.txt"
Basically when trying to work with the Windows path, python tends to replace '' to '\'. It goes for every backslash. When working with filepaths, you won't have double slashes since those are splitting folder names.
This way you can list all folders by order by splitting '\' and then rejoining them by .join function with frontslash.
Hopefully this helps!
The following line, unless I'm mistaken, will grab the absolute path to your directory so you can access files
PATH = os.path.abspath(os.path.join(os.path.dirname(sys.argv[0])))
This is what I've been using typically access files in my current directory when I need to use images etc in the programs i've been writing.
Now, say I do the following since I'm using windows to access a specific image in the directory
image = PATH + "\\" + "some_image.gif"
This is where my question lies, this works on windows, but if I remember correctly "\\" is for windows and this will not work on other OS? I cannot directly test this myself as I don't have other operating systems or I wouldn't have bothered posting. As far as I can tell from where I've looked this isn't mentioned in the documentation.
If this is indeed the case is there a way around this?
Yes, '\\' is just for Windows. You can use os.sep, which will be '\\' on Windows, ':' on classic Mac, '/' on almost everything else, or whatever is appropriate for the current platform.
You can usually get away with using '/'. Nobody's likely to be running your program on anything but Windows or Unix. And Windows will accept '/' pathnames in most cases. But there are many Windows command-line tools that will confuse your path for a flag if it starts with /, and some even if you have a / in the middle, and if you're using \\.\ paths, a / is treated like a regular character rather than a separator, and so on. So you're better off not doing that.
The simple thing to do is just use os.path.join:
image = os.path.join(PATH, "some_image.gif")
As a side note, in your first line, you're already using join—but you don't need it there:
PATH = os.path.abspath(os.path.join(os.path.dirname(sys.argv[0])))
It's perfectly legal to call join with only one argument like this, but also perfectly useless; you just join the one thing with nothing; you will get back exactly what you passed in. Just do this:
PATH = os.path.abspath(os.path.dirname(sys.argv[0]))
One last thing: If you're on Python 3.4+, you may want to consider using pathlib instead of os.path:
PATH = Path(sys.argv[0]).parent.resolve()
image = PATH / "some_image.gif"
Use os.path.join instead of "\\":
os.path.join(PATH, "some_image.gif")
The function will join intelligently the different parts of the path.
PATH = os.path.abspath(os.path.join(os.path.dirname(sys.argv[0])))
image = os.path.join(PATH, "some_image.gif")
os.path.join will intelligently join the arguments using os.sep which uses the OS file separator for you.
I'm not able to see the bigger picture here I think; but basically I have no idea why you would use os.path.join instead of just normal string concatenation?
I have mainly used VBScript so I don't understand the point of this function.
Portable
Write filepath manipulations once and it works across many different platforms, for free. The delimiting character is abstracted away, making your job easier.
Smart
You no longer need to worry if that directory path had a trailing slash or not. os.path.join will add it if it needs to.
Clear
Using os.path.join makes it obvious to other people reading your code that you are working with filepaths. People can quickly scan through the code and discover it's a filepath intrinsically. If you decide to construct it yourself, you will likely detract the reader from finding actual problems with your code: "Hmm, some string concats, a substitution. Is this a filepath or what? Gah! Why didn't he use os.path.join?" :)
Will work on Windows with '\' and Unix (including Mac OS X) with '/'.
for posixpath here's the straightforward code
In [22]: os.path.join??
Type: function
String Form:<function join at 0x107c28ed8>
File: /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py
Definition: os.path.join(a, *p)
Source:
def join(a, *p):
"""Join two or more pathname components, inserting '/' as needed.
If any component is an absolute path, all previous path components
will be discarded."""
path = a
for b in p:
if b.startswith('/'):
path = b
elif path == '' or path.endswith('/'):
path += b
else:
path += '/' + b
return path
don't have windows but the same should be there with '\'
It is OS-independent. If you hardcode your paths as C:\Whatever they will only work on Windows. If you hardcode them with the Unix standard "/" they will only work on Unix. os.path.join detects the operating system it is running under and joins the paths using the correct symbol.
I am Trying to add a custom path filed in my GUI but the problem is that when i use the command
cmds.fileDialog2(filemode=3,dialogStyle =1)
i get a file path like
C:\Users\anoorani\Desktop\Test
However Maya only seems to be reading paths like
C:/Users/anoorani/Desktop/Test
The backticks seem to be a problem
is there a way to replace "\" with "/" in python maya.....?
Acording to #ArgiriKotsaris note, you can use os.path.normpath(path):
Normalize a pathname by collapsing redundant separators and up-level references.
So that A//B, A/B/, A/./B and A/foo/../B all become A/B.
This string manipulation may change the meaning of a path that contains symbolic links.
On Windows, it converts forward slashes to backward slashes.
So your code be:
import maya.cmds as cmds
import os
path = cmds.fileDialog2(fm=3,dialogStyle =1)
path = path and os.path.normpath(path[0])
Or if you want to always using forward slashes, then no needs to os module and change last line to:
path = path and path[0].replace('\\', '/')
Note that name of file mode argument for fileDialog2 is fileMode or fm and not filemode.
Also fileDialog2 return a list of paths or None.