I am using python to merge a number of CSV files.
I use this code to find the CSV files:
with open('C:\TODAY.csv', 'w') as f_obj:
rows = []
files = os.listdir('C:\RAW\\')
I then iterate through files and make a list of the rows using:
for f in files:
if fnmatch.fnmatch(f, '*.csv') and not fnmatch.fnmatch(f, 'TODAY.CSV'):
print f
rows.append(open(f).readlines())
I added the print f for the sake of debugging.
What happens is this:
I get an error saying results1.csv does not exist when trying to do the rows.append function however the print f does print results1.csv and therefore it must exist, and python knows it exists because it can print the filename.
So why, if python can print the filename, does python say on the next line that it doesn't exist?
The traceback is:
Traceback (most recent call last):
File "Exc.py", line 22, in <module>
rows.append(open(f).readlines())
IOError: [Errno 2] No such file or directory: 'result1.csv
os.listdir gives a list of file names, not of file paths. The file names can be used to open a file only if your program's current directory is already the directory you're listing.
Use os.path.join to join the directory path with a file name to give a file path:
rows.append(open(os.path.join('C:\RAW\\', f)).readlines())
Ref https://docs.python.org/2/library/fnmatch.html
fnmatch only tells you if the given filename matches the given pattern, NOT whether a file with that name actually exists.
Try this:
for f in files:
if fnmatch.fnmatch(f, '*.csv') and not fnmatch.fnmatch(f, 'TODAY.CSV'):
print f
try:
rows.append(open(f).readlines())
except IOError:
print "File didn't exist, skipping"
Related
I have a folder with 5 .txt files:
100240042.txt
102042044.txt
016904962.txt
410940329.txt
430594264.txt
One contains only different types of fruit (e.g. apple, banana, orange, ect.). However, the others all contain 'chicken'. These must be deleted, leaving only the fruit list left.
So far, I've tried 4 different solutions
Attempt 0
import os
for filename in os.listdir(r'C:\Users\Steve\AppData\Local\Programs\Python\Python37-32\Fruits'):
f = open(filename)
for line in filename:
if 'chicken' in line:
found=true
os.remove(filename)
f.close()
Attempt 1
import os
for file in os.listdir(r'C:\Users\Steve\AppData\Local\Programs\Python\Python37-32\Fruits'):
open(file, 'r')
f.read()
find('chicken')
os.remove()
f.close()
Attempt 2
import os
for file in os.listdir(r'C:\Users\Steve\AppData\Local\Programs\Python\Python37-32\Fruits'):
open(file, 'r')
f.read()
find('chicken')
os.remove()
f.close()
Attempt 3
import os
for file in os.listdir(r'C:\Users\Steve\AppData\Local\Programs\Python\Python37-32\Fruits'):
if 'chicken' in open(file).read():
os.remove(file)
f.close()
I think this is a relatively simple problem, but I keep getting the following errors:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '100240042.txt'
I see other problems but let me just address what you asked:
os.remove(filename)
This is executed at the current directory. Usually the directory you run your program. But if you try to run command rm filename on your shell, you will also see errors because the file is actually on another directory. What you want to do is this:
open(os.path.join(r'C:\Users\Steve\AppData\Local\Programs\Python\Python37-32\Fruits', filename))
and also:
os.remove(os.path.join(r'C:\Users\Steve\AppData\Local\Programs\Python\Python37-32\Fruits', filename))
So your code should look like the following:
DIR = r'C:\Users\Steve\AppData\Local\Programs\Python\Python37-32\Fruits'
for filename in os.listdir(DIR):
found = False
with open(os.path.join(DIR, filename)) as f:
for line in f:
if 'chicken' in line:
found = True
break
if found:
os.remove(os.path.join(DIR, filename))
You have to construct the full path of the file. If you see the error message,
FileNotFoundError: [Errno 2] No such file or directory: '100240042.txt'
It is trying to open the file relative to the path of the script. In short, its looking for this file in the same directory as your script.
For getting absolute path, do something like
os.path.abspath("myFile.txt")
This question already has answers here:
How to identify whether a file is normal file or directory
(7 answers)
Closed 5 years ago.
I found some example code for ClamAV. And it works fine, but it only scans a single file. Here's the code:
import pyclamav
import os
tmpfile = '/home/user/test.txt'
f = open(tmpfile, 'rb')
infected, name = pyclamav.scanfile(tmpfile)
if infected:
print "File infected with %s Deleting file." %name
os.unlink(file)
else:
print "File is clean!"
I'm trying to scan an entire directory, here's my attempt:
import pyclamav
import os
directory = '/home/user/'
for filename in os.listdir(directory):
f = open(filename, 'rb')
infected, name = pyclamav.scanfile(filename)
if infected:
print "File infected with %s ... Deleting file." %name
os.unlink(filename)
else:
print " %s is clean!" %filename
However, I'm getting the following error:
Traceback (most recent call last):
File "anti.py", line 7, in <module>
f = open(filename, 'rb')
IOError: [Errno 21] Is a directory: 'Public'
I'm pretty new to Python, and I've read several similar questions and they do something like what I did, I think.
os.listdir("DIRECTORY") returns list of all files/dir in the DIRECTORY . It is just file names not absolute paths. So, if You are executing this program from a different directory it's bound to fail.
If you are sure that everything in the directory is a file, no sub directories. You can try following,
def get_abs_names(path):
for file_name in os.listdir(path):
yield os.path.join(path, file_name)
Then ,
for file_name in get_abs_names("/home/user/"):
#Your code goes here.
The following code will go over all your directory file by file. Your error happens because you try to open a directory as if it is a file instead of entering the dir and opening the files inside
for subdir, dirs, files in os.walk(path): # walks through whole directory
for file in files:
filepath = os.path.join(subdir, file) # path to the file
#your code here
I am getting some error while writing contents to csv file in python
import sys
reload(sys)
sys.setdefaultencoding('utf8')
import csv
a = [['1/1/2013', '1/7/2013'], ['1/8/2013', '1/14/2013'], ['1/15/2013', '1/21/2013'], ['1/22/2013', '1/28/2013'], ['1/29/2013', '1/31/2013']]
f3 = open('test_'+str(a[0][0])+'_.csv', 'at')
writer = csv.writer(f3,delimiter = ',', lineterminator='\n',quoting=csv.QUOTE_ALL)
writer.writerow(a)
Error
Traceback (most recent call last):
File "test.py", line 10, in <module>
f3 = open('test_'+str(a[0][0])+'_.csv', 'at')
IOError: [Errno 2] No such file or directory: 'test_1/1/2013_.csv'
How to fix it and what is the error?
You have error message - just read it.
The file test_1/1/2013_.csv doesn't exist.
In the file name that you create - you use a[0][0] and in this case it result in 1/1/2013.
Probably this two signs '/' makes that you are looking for this file in bad directory.
Check where are this file (current directory - or in .test_1/1 directory.
It's probably due to the directory not existing - Python will create the file for you if it doesn't exist already, but it won't automatically create directories.
To ensure the path to a file exists you can combine os.makedirs and os.path.dirname.
file_name = 'test_'+str(a[0][0])+'_.csv'
# Get the directory the file resides in
directory = os.path.dirname(file_name)
# Create the directories
os.makedirs(directory)
# Open the file
f3 = open(file_name, 'at')
If the directories aren't desired you should replace the slashes in the dates with something else, perhaps a dash (-) instead.
file_name = 'test_' + str(a[0][0]).replace('/', '-') + '_.csv'
I am trying to list all the file ending with '.py'. The way I want it to be displayed is dirname + filename. I am using os.path.join() for this. I am getting an error when I put /Users as the starting point. But when I specify the directory where most of the '.py' files are like os.walk(r'/Users/name/Pythonfiles') I don't get the error.
This is the code I came up with:
for cdir, dir, files in os.walk(r'/Users'):
for file in files:
if file.endswith('.py'):
filename = os.path.join(cdir, file)
print filename
Error that I am getting:
Traceback (most recent call last):
File "/Users/name/PythonTutorials/finding_largest_file.py", line 9, in <module>
filename = os.path.join(dir, file)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 77, in join
elif path == '' or path.endswith('/'):
AttributeError: 'list' object has no attribute 'ends with'
And please suggest a better way to accomplish this task if possible.
You are using the wrong value in your os.path.join() call. The current directory being iterated over is the first value given by os.walk(), which you assigned to cdir.
You are using the second value returned by os.walk(), which is a list of directory names.
Use:
for cdir, directories, files in os.walk(r'/Users'):
for file in files:
if file.endswith('.py'):
filename = os.path.join(cdir, file)
print filename
I renamed dir to directories; a clearer name and also one that doesn't mask the built-in dir() function.
I have a number of plain-text config files (.dta) that are spread through 27 sub-directories. I am trying to parse some information from all of them into a common document that is easier to work with.
Thus far I have:
import linecache
import csv
import os
csvout = csv.writer(open("dtaCompile.csv","wb"))
directory = os.path.join("c:\\","DirectKey")
for root,dirs,files in os.walk(directory):
for file in files:
if file.endswith(".DTA"):
f=open(file,'r')
lines = f.readlines()
description = lines[1]
articleCode = lines[2]
OS = lines[25]
SMBIOS = lines[32]
pnpID = lines[34]
cmdLine = lines[28]
csvout.writerow([SMBIOS, description, articleCode, pnpID, OS, cmdLine])
f.close()
I'm getting the following error:
Traceback (most recent call last):
File "test.py", line 11, in <module>
f=open(file,'r')
IOError: [Errno 2] No such file or directory: '000003APP.DTA'
Instead of
f=open(file,'r')
Your probaby need
f=open(os.path.join(directory, root, file),'r')
file is just the name of the file, and doesn't say anything about the path to it. you have to use os.path.join with the various components to create the full path
if file.endswith(".DTA"):
file = os.path.join(directory, root, file)
Instead of:
f=open(file,'r')
Try:
f = open(os.path.join(directory, file), "r")
My guess is that the directory you're program is executing in is not the same as the directory you're walking.
Try printing:
os.getcwd()
to see.