Python pathlib glob function fails on WindowsError: [123]? - python

I've written the following python function that returns a python list of File Geodatabase Paths. Please note that input_folder is a raw string and contains no unicode characters.
try:
gdbs = list(Path(input_folder).glob('**/*.gdb'))
for gdb in gdbs:
print(gdb)
except WindowsError, e:
print("error")
The problem that I'm having is that pathlib glob method is failing when it encounters unicode characters in the path of files in the directory.
I tried the following but it still fails, which I assume is because I'm not converting the paths the glob generator is coming across.
try:
gdbs = list(Path(unicode(input_folder)).glob('**/*.gdb'))
for gdb in gdbs:
print(gdb)
except WindowsError, e:
print("error")
The error message that is returned is:
WindowsError: [Error 123] The filename, directory name, or volume label syntax is incorrect: 'R:\\Data\\Africa\\Tanzania\\fromDropbox\\DART\\BRT Phase 2-3 designs\\1.12 Engineering Drawings for Service\\ROAD LIGHT\\PDF\\01.Traffic Sign(Kilwa)-??04.pdf'
Any help to handle the following error will be appreciated.

Try this :
input_folder = r'R:\Data\Africa\Tanzania\fromDropbox\DART\BRT Phase 2-3 designs\1.12 Engineering Drawings for Service\ROAD LIGHT\PDF\01.Traffic Sign(Kilwa)-??04.pdf'
The correct call should have 'r' in front of the path, and using single slash.

It seems to be a problem with pathlib because of Python 2.7 not being able to handle non-ascii characters. pathlib chokes up on international characters on Python 2 on Windows

Related

Navigating between directories in python

I run the following sequence of commands. The aim was moving between various directories and folders based on the original directory (so it does not need to be apriori coded):
path=os.getcwd()
os.chdir('..')
path2=os.getcwd()
path3=path2+('\\mydir')
os.chdir('path3')
As a result I get an error:
The system cannot find the file specified: 'C:\\work_folder\\mydir'
The directory C:\work_folder\mydir exists in the system, so the problem is in my opinion in missinetrpretation of '\'.
Thus I tried to do the following:
path3=path3.replace(r'\\',r'\')
Again I am getting error:
SyntaxError: EOL while scanning string literal
I will apreciate any help in overcoming this problem. Thank you
In python, instead of using \s in your directories, you can use /s instead and it'll still work.
You can not use a single un-escaped backslash (\) in a raw string. Change the line to:
path3 = path3.replace(r'\\', '\\')
This will solve your second error (SyntaxError: EOL while scanning string literal).

Moving files with python shutil library

I am trying to move specific folders within a file directory inside a flash drive with the Python shutil library. I am getting the following error:
FileNotFoundError: [Errno 2] No such file or directory: 'D:\\New Folder\\CN00020'.
I have looked at some questions posted and I think my issue may be that I am not declaring the filepath correctly. I am using the Spyder app for Python and Windows 10.
import shutil
shutil.move('D:\\New Folder\CN00020', 'D:\\Batch Upload')
the problem is that \ has special meaning. Python interprets \C as a special character. There are three solutions:
# escape backspace
shutil.move('D:\\New Folder\\CN00020', 'D:\\Batch Upload')
# use raw strings
shutil.move(r'D:\New Folder\CN00020', r'D:\Batch Upload')
# use forward slashes which shutil happens to support
shutil.move('D:/New Folder/CN00020', 'D:/Batch Upload')

Unable to open/include a YARA file

I created a script that analyzes files based on yara rules ( the yara are the ones from this repository https://github.com/Yara-Rules/rules). My script import a yara file that include all other rules.When i try to compile it, i receive a syntax error: "can't open include file: rules_for_files\Antidebug_AntiVM_index.yar", pointing me to one of the rules. I tried to exclude it but it continue points to others.
I tried to use different versions of python: 1.i used python2.7 and i received the mentioned error in both case when i use a binary string/raw string. About python 3.5 when i mentioned a binary string like the one from my code sample, the interpreter broke/reset(in case i use a GUI). How can i resolve this? Thank you.
rules = yara.compile(filepaths={
"malware_set1 rules": b'C:/Users/g_bondrila/Desktop/phishme/functionalitati/yararules/importyara.yar'})
def yara_match(file_path, rules=rules):
try:
matches = rules.match(file_path, timeout=60)
return matches
#except TimeoutError:
# print("the time is running out")
except:
print("something")
Try giving the directory path as below:
"C:\\Users\\g_bondrila\\Desktop\\phishme\\functionalitati\\yararules\\importyara.yar"
Since Python doesn't reads single slash for a path in windows.

os.listdir can't see my directory

I am working on a python script that installs an 802.1x certificate on a Windows 8.1 machine. This script works fine on Windows 8 and Windows XP (haven't tried it on other machines).
I have isolated the issue. It has to do with clearing out the folder
"C:\Windows\system32\config\systemprofile\AppData\LocalLow\Microsoft\CryptURLCache\Content"
The problem is that I am using the module os and the command listdir on this folder to delete each file in it. However, listdir errors, saying the folder does not exist, when it does indeed exist.
The issue seems to be that os.listdir cannot see the LocalLow folder. If I make a two line script:
import os
os.listdir("C:\Windows\System32\config\systemprofile\AppData")
It shows the following result:
['Local', 'Roaming']
As you can see, LocalLow is missing.
I thought it might be a permissions issue, but I am having serious trouble figuring out what a next step might be. I am running the process as an administrator from the command line, and it simply doesn't see the folder.
Thanks in advance!
Edit: changing the string to r"C:\Windows\System32\config\systemprofile\AppData", "C:\Windows\System32\config\systemprofile\AppData", or C:/Windows/System32/config/systemprofile/AppData" all produce identical results
Edit: Another unusual wrinkle in this issue: If I manually create a new directory in that location I am unable to see it through os.listdir either. In addition, I cannot browse to the LocalLow or my New Folder through the "Save As.." command in Notepad++
I'm starting to think this is a bug in Windows 8.1 preview.
I encountered this issue recently.
I found it's caused by Windows file system redirector
and you can check out following python snippet
import ctypes
class disable_file_system_redirection:
_disable = ctypes.windll.kernel32.Wow64DisableWow64FsRedirection
_revert = ctypes.windll.kernel32.Wow64RevertWow64FsRedirection
def __enter__(self):
self.old_value = ctypes.c_long()
self.success = self._disable(ctypes.byref(self.old_value))
def __exit__(self, type, value, traceback):
if self.success:
self._revert(self.old_value)
#Example usage
import os
path = 'C:\\Windows\\System32\\config\\systemprofile\\AppData'
print os.listdir(path)
with disable_file_system_redirection():
print os.listdir(path)
print os.listdir(path)
ref : http://code.activestate.com/recipes/578035-disable-file-system-redirector/
You must have escape sequences in your path. You should use a raw string for file/directory paths:
# By putting the 'r' at the start, I make this string a raw string
# Raw strings do not process escape sequences
r"C:\path\to\file"
or put the slashes the other way:
"C:/path/to/file"
or escape the slashes:
# You probably won't want this method because it makes your paths huge
# I just listed it because it *does* work
"C:\\path\\to\\file"
I'm curious as to how you are able to list the contents with those two lines. You are using escape sequences \W, \S, \c, \s, \A in your code. Try escaping the back slash like this:
import os
os.listdir('C:\\Windows\\System32\\config\\systemprofile\\AppData')

Rename invalid filename in XP via Python

My problem is similar to Python's os.path choking on Hebrew filenames
however, I don't know the original encoding of the filename I need to rename (unlike the other post he knew it was Hebrew originally).
I was doing data recovery for a client and copied over the files to my XP SP3 machine,
and some of the file names have "?" replacing/representing invalid characters.
I tried to use Python to os.rename the files since I know it has unicode support, however, when I tell python to rename the files, it seems it's unable to pass a valid file name back to the windows API.
i.e.:
>>> os.chdir(r'F:\recovery\My Music')
>>> os.listdir(u'.')
[u'Don?t Be Them.mp3', u'That?s A Soldier.mp3']
>>> blah=os.listdir(u'.')
>>> blah[0]
Don?t Be Them.mp3
>>> os.rename(blah[0],'dont be them.mp3')
Traceback (most recent call last):
File "<pyshell#6>", line 1, in <module>
os.rename(blah[0],'dont be them.mp3')
WindowsError: [Error 123] The filename, directory name, or
volume label syntax is incorrect
I'm using Python 2.6, on Win XP SP3, with whatever encoding is standard XP behavior for US/English.
Is there a way to handle these renames without knowing the original language?
'?' is not valid character for filenames. That is the reason while your approach failed.
You may try to use DOS short filenames:
import win32api
filelist = win32api.FindFiles(r'F:/recovery/My Music/*.*')
# this will extract "short names" from WIN32_FIND_DATA structure
filelist = [i[9] if i[9] else i[8] for i in filelist]
# EXAMPLE:
# this should rename all files in 'filelist' to 1.mp3, 2.mp3, 3.mp3, ...
for (number, filename) in enumerate(filelist):
os.rename(filaname, '%d.mp3' % (number))
Try passing a unicode string:
os.rename(blah[0], u'dont be them.mp3')

Categories