I'm trying to open all csv files within a given folder and then "perform a calculation". I'm currently trying to use glob but it doesn't see any files in the folder. All the code after this seems to work with a single file path but I imagine im using glob wrong.
path = "C:/build/Files*.csv"
for fileName in glob.glob(path):
with open(fileName, 'r') as file, \
open('C:PycharmProjects/Result.csv', 'r') as result_file:
#perform caluclation
I'm not sure what your file path looks like, but perhaps are you missing a /?
path = "C:/build/Files/*.csv" seems like a file structure that would be more likely.
Either make the path a raw string:
path = r"C:/build/Files*.csv"
or use double backslash:
path = "C:\\build\\Files*.csv"
Related
I recently made a small program (.py) that takes data from another file (.txt) in the same folder.
Python file path: "C:\Users\User\Desktop\Folder\pythonfile.py"
Text file path: "C:\Users\User\Desktop\Folder\textfile.txt"
So I wrote: with open(r'C:\Users\User\Desktop\Folder\textfile.txt', encoding='utf8') as file
And it works, but now I want to replace this path with a relative path (because every time I move the folder I must change the path in the program) and I don't know how... or if it is possible...
I hope you can suggest something... (I would like it to be simple and I also forgot to say that I have windows 11)
Using os, you can do something like
import os
directory = os.path.dirname(__file__)
myFile = with open(os.path.join(directory, 'textfile.txt'), encoding='utf8') as file
If you pass a relative folder to the open function it will search for it in the loacl directory:
with open('textfile.txt', encoding='utf8') as f:
pass
This unfortunately will only work if you launch your script form its folder. If you want to be more generic and you want it to work regardless of which folder you run from you can do it as well. You can get the path for the python file being launched via the __file__ build-in variable. The pathlib module then provides some helpfull functions to get the parent directory. Putting it all together you could do:
from pathlib import Path
with open(Path(__file__).parent / 'textfile.txt', encoding='utf8') as f:
pass
import os
directory = os.path.dirname(os.path.abspath(__file__))
file_path = os.path.join(directory, 'textfile.txt')
with open(file_path, encoding='utf8') as file:
# and then the code here
I've got 2 folders, each with a different CSV file inside (both have the same format):
I've written some python code to search within the "C:/Users/Documents" directory for CSV files which begin with the word "File"
import glob, os
inputfile = []
for root, dirs, files in os.walk("C:/Users/Documents/"):
for datafile in files:
if datafile.startswith("File") and datafile.endswith(".csv"):
inputfile.append([os.path.join(root, datafile)])
print(inputfile)
That almost worked as it returns:
[['C:/Users/Documents/Test A\\File 1.csv'], ['C:/Users/Documents/Test B\\File 2.csv']]
Is there any way I can get it to return this instead (no sub list and shows / instead of \):
['C:/Users/Documents/Test A/File 1.csv', 'C:/Users/Documents/Test B/File 2.csv']
The idea is so I can then read both CSV files at once later, but I believe I need to get the list in the format above first.
okay, I will paste an option here.
I made use of os.path.abspath to get the the path before join.
Have a look and see if it works.
import os
filelist = []
for folder, subfolders, files in os.walk("C:/Users/Documents/"):
for datafile in files:
if datafile.startswith("File") and datafile.endswith(".csv"):
filePath = os.path.abspath(os.path.join(folder, datafile))
filelist.append(filePath)
filelist
Result:
['C:/Users/Documents/Test A/File 1.csv','C:/Users/Documents/Test B/File 2.csv']
I am trying to write a program in python that loops through data from various csv files within a folder. Right now I just want to see that the program can identify the files in a folder but I am unable to have my code print the file names in my folder. This is what I have so far, and I'm not sure what my problem is. Could it be the periods in the folder names in the file path?
import glob
path = "Users/Sarah/Documents/College/Lab/SEM EDS/1.28.20 CZTS hexane/*.csv"
for fname in glob.glob(path):
print fname
No error messages are popping up but nothing will print. Does anyone know what I'm doing wrong?
Are you on a Linux-base system ? If you're not, switch the / for \\.
Is the directory you're giving the full path, from the root folder ? You might need to
specify a FULL path (drive included).
If that still fails, silly but check there actually are files in there, as your code otherwise seems fine.
This code below worked for me, and listed csv files appropriately (see the C:\\ part, could be what you're missing).
import glob
path = "C:\\Users\\xhattam\\Downloads\\TEST_FOLDER\\*.csv"
for fname in glob.glob(path):
print(fname)
The following code gets a list of files in a folder and if they have csv in them it will print the file name.
import os
path = r"C:\temp"
filesfolders = os.listdir(path)
for file in filesfolders:
if ".csv" in file:
print (file)
Note the indentation in my code. You need to be careful not to mix tabs and spaces as theses are not the same to python.
Alternatively you could use os
import os
files_list = os.listdir(path)
out_list = []
for item in files_list:
if item[-4:] == ".csv":
out_list.append(item)
print(out_list)
Are you sure you are using the correct path?
Try moving the python script in the folder when the CSV files are, and then change it to this:
import glob
path = "./*.csv"
for fname in glob.glob(path):
print fname
as the title would imply I am looking to create a script that will allow me to print a list of file names in a directory to a CSV file.
I have a folder on my desktop that contains approx 150 pdf's. I'd like to be able to have the file names printed to a csv.
I am brand new to Python and may be jumping out of the frying pan and into the fire with this project.
Can anyone offer some insight to get me started?
First off you will want to start by grabbing all of the files in the directory, then simply by writing them to a file.
from os import listdir
from os.path import isfile, join
import csv
onlyfiles = [f for f in listdir("./") if isfile(join("./", f))]
with open('file_name.csv', 'w') as print_to:
writer = csv.writer(print_to)
writer.writerow(onlyfiles)
Please Note
"./" on line 5 is the directory you want to grab the files from.
Please replace 'file_name.csv' with the name of the file you want to right too.
The following will create a csv file with all *.pdf files:
from glob import glob
with open('/tmp/filelist.csv', 'w') as fout:
# write the csv header -- optional
fout.write("filename\n")
# write each filename with a newline characer
fout.writelines(['%s\n' % fn for fn in glob('/path/to/*.pdf')])
glob() is a nice shortcut to using listdir because it supports wildcards.
import os
csvpath = "csvfile.csv"
dirpath = "."
f = open("csvpath, "wb")
f.write(",".join(os.listdir(dirpath)))
f.close()
This may be improved to present filenames in way that you need, like for getting them back, or something. For instance, this most probably won't include unicode filenames in UTF-8 form but make some mess out of the encoding, but it is easy to fix all that.
If you have very big dir, with many files, you may have to wait some time for os.listdir() to get them all. This also can be fixed by using some other methods instead of os.listdir().
To differentiate between files and subdirectories see Michael's answer.
Also, using os.path.isfile() or os.path.isdir() you can recursively get all subdirectories if you wish.
Like this:
def getall (path):
files = []
for x in os.listdir(path):
x = os.path.join(path, x)
if os.path.isdir(x): files += getall(x)
else: files.append(x)
return files
I wish to read incoming XML files that have no specific name (e.g. date/time naming) to extract values and perform particular tasks.
I can step through the files but am having trouble opening them and reading them.
What I have that works is:-
import os
path = 'directory/'
listing = os.listdir(path)
for infile in listing:
print infile
But when I add the following to try and read the files it errors saying No such file or directory.
file = open(infile,'r')
Thank you.
You need to provide the path to file too:
file = open(os.path.join(path,infile),'r')
os.listdir provides the base names, not absolute paths. You'll need to do os.path.join(path, infile) instead of just infile (that may still be a relative path, which should be fine; if you needed an absolute path, you'd feed it through os.path.abspath).
As an alternative to joining the directory path and filename as in the other answers, you can use the glob module. This can also be handy when your directories might contain other (non-XML) files that you don't want to process:
import glob
for infile in glob.glob('directory/*.xml'):
print infile
You have to join the directory path and filename, using
os.path.join(path, infile)
Also use the path without the / :
path = 'directory'
Something like this (Not optimized, just a small change in your code):
import os
path = 'directory'
listing = os.listdir(path)
for infile in listing:
print infile
file_abs = os.path.join(path, infile)
file = open(file_abs,'r')