We Keep Coding

Python: Unicode characters in file or folder names

We process a lot of files where path can contain an extended character set like this:
F:\Site Section\Cieślik
My Python scripts fail to open such files or chdir to such folders whatever I try.
Here is an extract from my code:
import zipfile36 as zipfile
import os
from pathlib import Path
outfile = open("F:/zip_pdf3.log", "w", encoding="utf-8")
with open('F:/zip_pdf.txt') as f: # Input file list - note the forward slashes!
for line in f:
print (line)
path, filename = os.path.split(line)
file_no_ext = os.path.splitext(os.path.basename(line))[0]
try:
os.chdir(path) # Go to the file path
except Exception as exception:
print (exception, file = outfile) #3.7
print (exception)
continue
I tried the following:
Converting path to a raw string
raw_string = r"{}".format(path)
try:
os.chdir(raw_string)
Converting a string to Path
Ppath = Path(path)
try:
os.chdir(Ppath.decode("utf8"))
Out of ideas... Anyone knows how to work with Unicode file and folder names? Using Python 3.7 or higher on Windows.
Could be as simple as that - thanks #SergeBallesta:
with open('F:/pdf_err.txt', encoding="utf-8") as f:
I may post updates after more runs with different input.
This, however, leads to a slightly different question: if, instead of reading from the file, I walk over folders and files with extended character set - how do I deal with those, i.e.
for subdir, dirs, files in os.walk(rootdir): ?
At present I'm getting either a "The filename, directory name, or volume label syntax is incorrect" or "Can't open the file".

Using terminal command to search through files for a specific string within a python script

I have a parent directory, and I'd like to go through that directory and grab each file with a specific string for editing in python. I have been using grep -r 'string' filepath in terminal, but I want to be able to do everything using python. I'm hoping to get all the files into an array and go through each of them to edit them.
Is there a way to do this by only running a python script?

changing current folder to parent
import os
os.chdir("..")
changing folder
import os
os.chdir(dir_of_your_choice)
finding files with a rule in the current folder
import glob
import os
current_dir = os.getcwd()
for f in glob.glob('*string*'):
do_things(f)

import os
#sourceFolder is the folder you're going to be looking inside for backslashes are a special character in python so they're escaped as double backslashes
sourceFolder = "C:\\FolderBeingSearched\\"
myFiles = []
# Find all files in the directory
for file in os.listdir(sourceFolder):
myFiles.append(file)
#open them for editing
for file in myFiles:
try:
open(sourceFolder + file,'r')
except:
continue
#run whatever code you need to do on each open file here
print("opened %s" % file)
EDIT: If you want to separate all files that contain a string (this just prints the list at the end currently):
import os
#sourceFolder is the folder you're going to be looking inside for backslashes are a special character in python so they're escaped as double backslashes
sourceFolder = "C:\\FolderBeingSearched\\"
myFiles = []
filesContainString = []
stringsImLookingFor = ['first','second']
# Find all files in the directory
for file in os.listdir(sourceFolder):
myFiles.append(file)
#open them for editing
for file in myFiles:
looking = sourceFolder + file
try:
open(looking,'r')
except:
continue
print("opened %s" % file)
found = 0
with open(looking,encoding="latin1") as in_file:
for line in in_file:
for x in stringsImLookingFor:
if line.find(x) != -1:
#do whatever you need to do to the file or add it to a list like I am
filesContainString.append(file)
found = 1
break
if found:
break
print(filesContainString)

How to extract a file within a folder within a zip?

I need to extract a file called Preview.pdf from a folder called QuickLooks inside of a zip file.
Right now my code looks a little like this:
with ZipFile(newName, 'r') as newName:
newName.extract(\QuickLooks\Preview.pdf)
newName.close()
(In this case, newName has been set equal to the full path to the zip).
It's important to note that the backslash is correct in this case because I'm on Windows.
The code doesn't work; here's the error it gives:
Traceback (most recent call last):
File "C:\Users\Asit\Documents\Evam\Python_Scripts\pageszip.py", line 18, in <module>
ZF.extract("""QuickLooks\Preview.pdf""")
File "C:\Python33\lib\zipfile.py", line 1019, in extract
member = self.getinfo(member)
File "C:\Python33\lib\zipfile.py", line 905, in getinfo
'There is no item named %r in the archive' % name)
KeyError: "There is no item named 'QuickLook/Preview.pdf' in the archive"
I'm running the Python script from inside Notepad++, and taking the output from its console.
How can I accomplish this?
Alternatively, how could I extract the whole QuickLooks folder, move out Preview.pdf, and then delete the folder and the rest of it's contents?
Just for context, here's the rest of the script. It's a script to get a PDF of a .pages file. I know there are bonified converters out there; I'm just doing this as an excercise with some sort of real-world application.
import os.path
import zipfile
from zipfile import *
import sys
file = raw_input('Enter the full path to the .pages file in question. Please note that file and directory names cannot contain any spaces.')
dir = os.path.abspath(os.path.join(file, os.pardir))
fileName, fileExtension = os.path.splitext(file)
if fileExtension == ".pages":
os.chdir(dir)
print (dir)
fileExtension = ".zip"
os.rename (file, fileName + ".zip")
newName = fileName + ".zip" #for debugging purposes
print (newName) #for debugging purposes
with ZipFile(newName, 'w') as ZF:
print("I'm about to list names!")
print(ZF.namelist()) #for debugging purposes
ZF.extract("QuickLook/Preview.pdf")
os.rename('Preview.pdf', fileName + '.pdf')
finalPDF = fileName + ".pdf"
print ("Check out the PDF! It's located at" + dir + finalPDF + ".")
else:
print ("Sorry, this is not a valid .pages file.")
sys.exit
I'm not sure if the import of Zipfile is redundant; I read on another SO post that it was better to use from zipfile import * than import zipfile. I wasn't sure, so I used both. =)
EDIT: I've changed the code to reflect the changes suggested by Blckknght.

Here's something that seems to work. There were several issues with your code. As I mentioned in a comment, the zipfile must be opened with mode 'r' in order to read it. Another is that zip archive member names always use forward slash / characters in their path names as separators (see section 4.4.17.1 of the PKZIP Application Note). It's important to be aware that there's no way to extract a nested archive member to a different subdirectory with Python's currentzipfilemodule. You can control the root directory, but nothing below it (i.e. any subfolders within the zip).
Lastly, since it's not necessary to rename the .pages file to .zip — the filename you passZipFile() can have any extension — I removed all that from the code. However, to overcome the limitation on extracting members to a different subdirectory, I had to add code to first extract the target member to a temporary directory, and then copy that to the final destination. Afterwards, of course, this temporary folder needs to deleted. So I'm not sure the net result is much simpler...
import os.path
import shutil
import sys
import tempfile
from zipfile import ZipFile
PREVIEW_PATH = 'QuickLooks/Preview.pdf' # archive member path
pages_file = input('Enter the path to the .pages file in question: ')
#pages_file = r'C:\Stack Overflow\extract_test.pages' # hardcode for testing
pages_file = os.path.abspath(pages_file)
filename, file_extension = os.path.splitext(pages_file)
if file_extension == ".pages":
tempdir = tempfile.gettempdir()
temp_filename = os.path.join(tempdir, PREVIEW_PATH)
with ZipFile(pages_file, 'r') as zipfile:
zipfile.extract(PREVIEW_PATH, tempdir)
if not os.path.isfile(temp_filename): # extract failure?
sys.exit('unable to extract {} from {}'.format(PREVIEW_PATH, pages_file))
final_PDF = filename + '.pdf'
shutil.copy2(temp_filename, final_PDF) # copy and rename extracted file
# delete the temporary subdirectory created (along with pdf file in it)
shutil.rmtree(os.path.join(tempdir, os.path.split(PREVIEW_PATH)[0]))
print('Check out the PDF! It\'s located at "{}".'.format(final_PDF))
#view_file(final_PDF) # see Bonus below
else:
sys.exit('Sorry, that isn\'t a .pages file.')
Bonus: If you'd like to actually view the final pdf file from the script, you can add the following function and use it on the final pdf created (assuming you have a PDF viewer application installed on your system):
import subprocess
def view_file(filepath):
subprocess.Popen(filepath, shell=True).wait()

IPython Search PYTHONPATH for file

Is there any way to make it so that IPython searches through your system path to find a file if only the name of the file is given?
I have a file foo_060112.dat that lives in a folder containing a lot of data files that are uniquely named in a folder in my path. I want to be able to simply call a load or open function on this file without specifying the full path---is it possible?

You can write your own function to search through the path and return an open file object according to the mode you select:
import sys
import os
def my_open(filename, mode):
for path in sys.path:
try:
return open(os.path.join(path, filename), mode)
except IOError:
continue
return None
Example: my_open('foo_060112.dat', 'rb')

I thinks this may help you:
import os,sys
path = os.sytem('locate foo_060112.dat')
print path
or you can also use :
_SOURCE_FILE_PATH = '/home/admin/'
_each_folder = os.walk("%s/" %(_SOURCE_FILE_PATH))
for x in _each_folder:
print x
if x[1] == []:
for y in x[2]:
if y == 'locate foo_060112.dat'
f = open(y,'r')
data = f.read()

Python Searching for String and printing the file it is in

I am working on a small program for work, and I have looked everywhere for help on this!
What I'm trying to do is to allow the user to put in string to search. The program will search multiple .txt files in a defined directory for the string and then either print the result, or open the .txt file using the default text editor.
Can someone please point me in the right direction for this search feature?
Thanks in advance!
Edit:
This is what I have so far. I cant use grep as this program wil be running on Windows as well as OSX. I have yet to test on Windows, but on OSX my results are access denied.
import os
import subprocess
text = str(raw_input("Enter the text you want to search for: "))
thedir = './f'
for file in os.listdir(thedir):
document = os.path.join(thedir, file)
for line in open(document):
if text in line:
subpocess.call(document, shell=True)

There are much better tools to do this (grep was mentioned, and it is probably the best way to go).
Now, if you want a Python solution (which would run very slow), you can start from here:
import os
def find(word):
def _find(path):
with open(path, "rb") as fp:
for n, line in enumerate(fp):
if word in line:
yield n+1, line
return _find
def search(word, start):
finder = find(word)
for root, dirs, files in os.walk(start):
for f in files:
path = os.path.join(root, f)
for line_number, line in finder(path):
yield path, line_number, line.strip()
if __name__ == "__main__":
import sys
if not len(sys.argv) == 3:
print("usage: word directory")
sys.exit(1)
word = sys.argv[1]
start = sys.argv[2]
for path, line_number, line in search(word, start):
print ("{0} matches in line {1}: '{2}'".format(path, line_number, line))
Please take this with a grain of salt: it will not use regular expressions, or be smart at all. For example, if you try to search for "hola" it will match "nicholas", but not "Hola" (in the latter case, you could add a line.lower() method.
Again, it is just a beginning to show you a possible way to start. However, please PLEASE use grep.
Cheers.
Sample run (I called this script "pygrep.py"; $ is the command prompt):
$python pygrep.py finder .
./pygrep.py matches in line 12: 'finder = find(word)'
./pygrep.py matches in line 16: 'for line_number, line in finder(path):'
./pygrep.py~ matches in line 11: 'finder = find(word)'

below are hints to your answer :)
You can use os.walk to traverse all the files in the specified directory structure, search the string in the file, use subprocess module to open the file in the required editor...

import os
import subprocess
text = str(raw_input("Enter the text you want to search for: "))
thedir = 'C:\\your\\path\\here\\'
for file in os.listdir(thedir):
filepath = thedir + file
for line in open(filepath):
if text in line:
subprocess.call(filepath, shell=True)
break