I'm trying to loop through some files in a directory. If the filename has two specific strings together, then I'm supposed to open and read those files for information. However, if none of the files have those two strings, I want to print an error message only once.
for filename in os.listdir(directory):
if filename.find("<string1>") != -1 and filename.find("<string2>") != -1:
#open file
else:
#print error message
I know doing this will print as many error messages as there are files in the directory (i.e. if there's 15 files with no matches, I'll get 15 error messages). But what I want is to only print an error message once after there aren't any matches in any of the N files in directory. I figured I could do something like this:
for filename in os.listdir(directory):
if filename.find("<string1>") != -1 and filename.find("<string2>") != -1:
#open file
else:
if filename[-1]: #if filename is last in directory
#print error message
But I've discovered this doesn't work. How would I get an error message to print only after the last filename has been read and doesn't match?
A simple solution would be to initialize some boolean flag before your for loop, e.g. found = false
If you find a file, set found = true. Then you can check the value of found after your for loop finishes and print the appropriate message based on its value.
Filter the list of files before the for-loop:
filenames = [fname for fname in os.listdir(directory)
if '<string1>' in fname and '<string2>' in fname]
if filenames:
for filename in filenames:
#open file
else:
#print error message
You can probably also use the glob module to get the filenames:
import glob
filenames = glob.glob(directory + '/*string1*string2*')
Another way is to use a variable to check if all the files have been processed. Checked and found it working in Python 2.7
import os
directory = "E:\\test\\"
files_count = len(os.listdir(directory))
files_processed = 0
for filename in os.listdir(directory):
if 'string1' in filename and 'string2' in filename:
#open file
print ("Opening file")
else:
files_processed = files_processed + 1
if (files_processed >= files_count):
print ("error message")
Not sure if this is extreme. But I'd make it a function and raise IOError.
Plus, i'd always use absolute path. Try the pathlib module too
import os
def get_files(directory):
for filename in os.listdir(directory):
if "string1" in filename and "string2" in filename:
yield filename
raise IOError("No such file")
for file in get_files('.'):
print(file)
# do stuff with file
Related
I wanted to supply python with a windows 'data path' that could be used to set up input processing. I googled this with no luck, and now figure I am on my own.
There appears to be many ways of reading in a file with python, and after some frustration with "\" and "/" and windows path names I found a way to get my data path set up. It is not a general approach but should serve me well.
Related Questions: Is this code ugly? Is this a nonstandard method? Are there elegant features in 3.6 that should be used?
### Title: Process an input file using a 'data path' for a user on windows
import sys
import os
print("OK, starting program...")
file_processed = False
for path, dirs, files in os.walk("/Users/Mike"):
if file_processed: break
for file in files:
if file_processed: break
if file == 'seriousdata.txt':
my_path = os.path.abspath(path)
my_dsn = os.path.join(my_path, file)
print("Opening txt file " + my_dsn + " for input.")
with open(my_dsn) as da_lines:
textlines = (line.rstrip('\r\n') for line in da_lines)
for line in textlines:
print(line)
file_processed = True
if file_processed:
pass
else:
print("File not found")
print("OK, program execution has ended.")
sys.exit() # END SAMPLE CODE SNIPPET
From looking at your code, I'm assuming that you want to start at one directory, and move through each child directory, printing out the matching filename's contents if it is found.
If so, then this is way to do this with recursion:
import os
def recursive_list(path, filename):
files = os.listdir(path)
for name in files:
try:
p = os.path.join(path, name)
if os.path.isdir(p):
recursive_list(p, filename)
else:
if name == filename:
with open(p, "r") as f:
print(f.read())
except PermissionError:
pass
return
recursive_list("/home/jebby/Desktop","some_file.txt")
This will start out listing files in path. For every file that is found, if that file is a directory, then the function itself is called (Starting at the path to that folder). If filename matches the name of any file in the current directory, it will be printed (if the user has permissions for that file).
Otherwise, if you only want to read the filename from a known directory without walking down the directory tree:
import os
data_path = "/home/jebby/Desktop"
file_you_want = "filename.txt"
with open(os.path.join(data_path, file_you_want), "r") as f:
content = f.read()
print(content)
The main question would be : Do you know the location of the file?
Jebby has an answer to crawl through the directories.
Here is a solution without using "import os"
dir_fullpath = "c:/project_folder/data"
dir_path = "data"
filename = "file.txt"
try:
f = open(dir_path + "/" + filename, 'r')
# print("open " +dir_path + "\" + filename)
# data=[]
for line in f:
print (line.rstrip())
# data.append(line.rstrip())
f.close()
except IOError:
print("Fail to open" + dir_path + "\" + filename)
The purpose of this code is:
Read a csv file which contains a column for a list of file names
here is the csv file:
https://drive.google.com/open?id=0B5bJvxM9TZkhVGI5dkdLVzAyNTA
Then check a specific folder to check if the files exist or not
If its found a file is not in the list delete it
here is the code:
import pandas as pd
import os.path
data = pd.read_csv('data.csv')
names = data['title']
path = "C:\\Users\\Sayed\\Desktop\\Economic Data"
for file in os.listdir(path):
os.path.exists(file)
print(file)
file = os.path.join(path, file)
fileName = os.path.splitext(file)
if fileName not in names:
print('error')
os.remove(file)
I modified the first code, and this is the new code and I got no error but the simply delete all the files in the directory
os.chdir does not return anything, so assigning the result to path means that path has None, which causes the error.
Since you're using pandas, here's a little trick to speed this up using pd.Series.isin.
root = "C:\Users\Sayed\Desktop\Economic Data"
files = os.listdir(root)
for f in data.loc[~data['title'].isin(files), 'title'].tolist():
try:
os.remove(os.path.join(root, f))
except OSError:
pass
Added a try-except check in accordance with EAFP (since I'm not doing an os.path.exists check here). Alternatively, you could add a filter based on existence using pd.Series.apply:
m = ~data['title'].isin(files) & data['title'].apply(os.path.exists)
for f in data.loc[m, 'title'].tolist():
os.remove(os.path.join(root, f))
Your path is the return value of the os.chdir() call. Which is obviously None.
You want to set path to the string representing the path ... leave the chdir out.
Basically, the problem I'm have is trying to open multiple files in a for loop. The filename has this format:
filename = 'mms1_fgm_srvy_l2_20160104_v4.18.0.cdf'
With '20160104' being the date, which I know how to update in the loop. The problem is that the '18' attached at the end isn't constant for every file, and I don't know how it changes, unlike the dates. I was wondering is there is a way to update the number, and check if the file exists in my directory.
As always, any help would be greatly appreciated. Thanks.
You can use the glob.glob() function with a suitable filename pattern to get a list of files (that exist) which match the pattern.
For example:
import glob
pattern = 'mms1_fgm_srvy_l2_*_v4.*.0.cdf'
for filename in glob.glob(pattern):
with open(filename) as file:
process(file)
import os
BASE_NAME = 'mms1_fgm_srvy_l2_20160104_v4.{}.0'
EXT = '.cdf'
attempts = int(input('Check file up to: '))
for num in range(attempts):
file_name = BASE_NAME.format(num) + EXT
if os.path.isfile(file_name):
# open file here
print("Opened File")
else:
print("File does not exist")
Checks if the file exists and if it does you can load it and save it how ever you want else it will print the the file doesn't exist
I've created a simple script to rename my media files that have lots of weird periods and stuff in them that I have obtained and want to organize further. My script kinda works, and I will be editing it to edit the filenames further but my os.rename line throws this error:
[Windows Error: Error 2: The system cannot find the file specified.]
import os
for filename in os.listdir(directory):
fcount = filename.count('.') - 1 #to keep the period for the file extension
newname = filename.replace('.', ' ', fcount)
os.rename(filename, newname)
Does anyone know why this might be? I have a feeling that it doesn't like me trying to rename the file without including the file path?
try
os.rename(filename, directory + '/' + newname);
Triton Man has already answered your question. If his answer doesn't work I would try using absolute paths instead of relative paths.
I've done something similar before, but in order to keep any name clashes from happening I temporarily moved all the files to a subfolder. The entire process happened so fast that in Windows Explorer I never saw the subfolder get created.
Anyhow if you're interested in looking at my script It's shown below. You run the script on the command line and you should pass in as a command-line argument the directory of the jpg files you want renamed.
Here's a script I used to rename .jpg files to multiples of 10. It might be useful to look at.
'''renames pictures to multiples of ten'''
import sys, os
debug=False
try:
path = sys.argv[1]
except IndexError:
path = os.getcwd()
def toint(string):
'''changes a string to a numerical representation
string must only characters with an ordianal value between 0 and 899'''
string = str(string)
ret=''
for i in string:
ret += str(ord(i)+100) #we add 101 to make all the numbers 3 digits making it easy to seperate the numbers back out when we need to undo this operation
assert len(ret) == 3 * len(string), 'recieved an invalid character. Characters must have a ordinal value between 0-899'
return int(ret)
def compare_key(file):
file = file.lower().replace('.jpg', '').replace('dscf', '')
try:
return int(file)
except ValueError:
return toint(file)
#files are temporarily placed in a folder
#to prevent clashing filenames
i = 0
files = os.listdir(path)
files = (f for f in files if f.lower().endswith('.jpg'))
files = sorted(files, key=compare_key)
for file in files:
i += 10
if debug: print('renaming %s to %s.jpg' % (file, i))
os.renames(file, 'renaming/%s.jpg' % i)
for root, __, files in os.walk(path + '/renaming'):
for file in files:
if debug: print('moving %s to %s' % (root+'/'+file, path+'/'+file))
os.renames(root+'/'+file, path+'/'+file)
Edit: I got rid of all the jpg fluff. You could use this code to rename your files. Just change the rename_file function to get rid of the extra dots. I haven't tested this code so there is a possibility that it might not work.
import sys, os
path = sys.argv[1]
def rename_file(file):
return file
#files are temporarily placed in a folder
#to prevent clashing filenames
files = os.listdir(path)
for file in files:
os.renames(file, 'renaming/' + rename_file(file))
for root, __, files in os.walk(path + '/renaming'):
for file in files:
os.renames(root+'/'+file, path+'/'+file)
Looks like I just needed to set the default directory and it worked just fine.
folder = r"blah\blah\blah"
os.chdir(folder)
for filename in os.listdir(folder):
fcount = filename.count('.') - 1
newname = filename.replace('.', ' ', fcount)
os.rename(filename, newname)
How can I loop this code so that it asks the user to search for another file if the file has not been found?
import os, sys
from stat import *
from os.path import join
lookfor = input("\nPlease enter file name you want to search? \n")
def search(directory):
for files in os.listdir(directory):
fileItem = os.path.join(directory, files)
fileItemStatInfo = os.stat(fileItem)
if S_ISDIR(fileItemStatInfo.st_mode):
search(fileItem)
elif S_ISREG(fileItemStatInfo.st_mode):
print("Searching", fileItem)
if lookfor in files:
print("\nThe File Has Been Found: %s" % join(directory, lookfor))
break
First, you need to make search return success or failure:
def search(directory):
for files in os.listdir(directory):
fileItem = os.path.join(directory, files)
fileItemStatInfo = os.stat(fileItem)
if S_ISDIR(fileItemStatInfo.st_mode):
return search(fileItem)
elif S_ISREG(fileItemStatInfo.st_mode):
print("Searching", fileItem)
if lookfor in files:
print("\nThe File Has Been Found: %s" % join(directory, lookfor))
return True
(Note that if we fall off the end of the for loop, we'll fall off the end of the function, which means we return None. Therefore, the function an only return True, with is truthy, or None, which is not.)
Now, you just loop until it returns something true:
while True:
lookfor=input("\nPlease enter file name you want to search? \n")
if search(lookfor):
break
print('Could not find that file, sorry. Try again.')
All that said, I don't think search does what you want it to—and you can simplify it tremendously, too.
First, I'm not sure whether you're trying to use lookfor as a global variable, or as a closure within the search function… but either way, you probably shouldn't be doing that. Pass it as an argument.
Also, having a variable named files that holds each filename, instead of a collection of them, is very confusing.
If you want to walk a directory tree recursively, use os.walk instead of trying to implement it yourself.
And you should probably be printing out the found filename, not the looked-for fragment.
Putting that together:
def search(lookfor, directory):
for dirpath, dirnames, filenames in os.walk(directory):
for filename in filenames:
if lookfor in filename:
print("\nThe File Has Been Found: %s" % join(dirpath, filename))
return True
Of course in most real-life code, you'll probably want to return the pathname, so the rest of your code can use it (e.g., to open the file), instead of just printing it out:
def search(lookfor, directory):
for dirpath, dirnames, filenames in os.walk(directory):
for filename in filenames:
if lookfor in filename:
return join(dirpath, filename)
while True:
lookfor=input("\nPlease enter file name you want to search? \n")
path = search(lookfor)
if path:
break
print('Could not find that file, sorry. Try again.')
with open(path) as f:
pass # now we can actually use the file we searched so hard for