This question already has answers here:
Find all files in a directory with extension .txt in Python
(25 answers)
Closed 3 years ago.
I need to write a code in python that can scan for all files inside a folder containing determined extensions like .exe, .jpg, .pdf.
Just like the linux command "ls | grep *.pdf"
I've tried to use a list containing all extensions i need and used Regular Expressions to search for it inside the folder. But i don't know what to put inside re.search()
I don't want to use something like "os" library because this script needs to work on Linux and Windows.
#!/usr/bin/python
import re
file_types = [".exe", ".jpg", ".pdf", ".png", ".txt"]
for line in file_types:
# Do something like "ls | grep * + line"
namefile = re.search(line, i_dont_know_what_to_put_here)
print(namefile)
Update: Thank guys for help, i used glob library and it's works!
You can avoid the os module by using the glob module, which can filter files by regular expression (i.e. *.py)
from glob import glob
file_types = [".exe", ".jpg", ".pdf", ".png", ".txt"]
path = "path/to/files/*{}"
fnames = [ fname for fnames in [[fname for fname in glob( path.format( ext ))] for ext in file_types] for fname in fnames]
Hard to read but it's equivalent is:
from glob import glob
file_types = [".exe", ".jpg", ".pdf", ".png", ".txt"]
fnames = []
for ext in file_types:
for fname in glob( path.format( ext )):
fnames.append( fname )
EDIT: I'm not sure how this works cross platform as some other answers have considered.
EDIT2: glob may have unexpected side effects when used in windows. Getting Every File in a Windows Directory
Try os.listdir():
import os
file_types = ["exe", "jpg", "pdf", "png", "txt"]
files = [f for f in os.listdir('.') if os.path.isfile(f)]
# filter on file type
files = [f for f in files if f.split('.')[-1] in file_types]
In general the os and os.path module is going to be very useful to you here. You could use a regex, but unless performance is very important I wouldn't bother.
Adding to the other comments here, if you still wish to use re, the way you should use it is:
re.search(<string to search for(regex)>, <string to search IN>)
so in your case lets say you have filetype = ".pdf", your code will be:
re.search(".*\{}".format(filetype), filename)
where .* means "match any character 0 or more times", and the '\' along with the ".pdf" will mean "where the name contains .pdf" (the \ is an escape char so the dot won't be translated to regex). I believe you can also add a $ at the end of the regex to say "this is the end of the string".
And as mentioned here - os.listdir works perfectly fine for both Windows & Linux.
Hope that helps.
My suggestion (it will work on all OS - Windows, Linux and macOS):
import os
file_types = [".exe", ".jpg", ".pdf", ".png", ".txt"]
files = [entry.path for entry in os.scandir('.') if entry.is_file() and os.path.splitext(entry.name)[1] in file_types]
or (if you want just filenames instead of full paths):
files = [entry.name for entry in os.scandir('.') if entry.is_file() and os.path.splitext(entry.name)[1] in file_types]
Related
I am trying to write a program in python that loops through data from various csv files within a folder. Right now I just want to see that the program can identify the files in a folder but I am unable to have my code print the file names in my folder. This is what I have so far, and I'm not sure what my problem is. Could it be the periods in the folder names in the file path?
import glob
path = "Users/Sarah/Documents/College/Lab/SEM EDS/1.28.20 CZTS hexane/*.csv"
for fname in glob.glob(path):
print fname
No error messages are popping up but nothing will print. Does anyone know what I'm doing wrong?
Are you on a Linux-base system ? If you're not, switch the / for \\.
Is the directory you're giving the full path, from the root folder ? You might need to
specify a FULL path (drive included).
If that still fails, silly but check there actually are files in there, as your code otherwise seems fine.
This code below worked for me, and listed csv files appropriately (see the C:\\ part, could be what you're missing).
import glob
path = "C:\\Users\\xhattam\\Downloads\\TEST_FOLDER\\*.csv"
for fname in glob.glob(path):
print(fname)
The following code gets a list of files in a folder and if they have csv in them it will print the file name.
import os
path = r"C:\temp"
filesfolders = os.listdir(path)
for file in filesfolders:
if ".csv" in file:
print (file)
Note the indentation in my code. You need to be careful not to mix tabs and spaces as theses are not the same to python.
Alternatively you could use os
import os
files_list = os.listdir(path)
out_list = []
for item in files_list:
if item[-4:] == ".csv":
out_list.append(item)
print(out_list)
Are you sure you are using the correct path?
Try moving the python script in the folder when the CSV files are, and then change it to this:
import glob
path = "./*.csv"
for fname in glob.glob(path):
print fname
I am very new to python and just installed Eric6 I am wanting to search a folder (and all sub dirs) to print the filename of any file that has the extension of .pdf I have this as my syntax, but it errors saying
The debugged program raised the exception unhandled FileNotFoundError
"[WinError 3] The system can not find the path specified 'C:'"
File: C:\Users\pcuser\EricDocs\Test.py, Line: 6
And this is the syntax I want to execute:
import os
results = []
testdir = "C:\Test"
for folder in testdir:
for f in os.listdir(folder):
if f.endswith('.pdf'):
results.append(f)
print (results)
Use the glob module.
The glob module finds all the pathnames matching a specified pattern
import glob, os
parent_dir = 'path/to/dir'
for pdf_file in glob.glob(os.path.join(parent_dir, '*.pdf')):
print (pdf_file)
This will work on Windows and *nix platforms.
Just make sure that your path is fully escaped on windows, could be useful to use a raw string.
In your case, that would be:
import glob, os
parent_dir = r"C:\Test"
for pdf_file in glob.glob(os.path.join(parent_dir, '*.pdf')):
print (pdf_file)
For only a list of filenames (not full paths, as per your comment) you can do this one-liner:
results = [os.path.basename(f) for f in glob.glob(os.path.join(parent_dir, '*.pdf')]
Right now, you search each character string inside of testdir's variable.
so it's searching the folder for values "C", ":", "\", "T" etc. You'll want to also escape your escape character like "C:\...\...\"
You probably was to use os.listdir(testdir) instead.
Try running your Python script from C:. From the Command Prompt, you might wanna do this:
> cd C:\
> python C:\Users\pcuser\EricDocs\Test.py
As pointed out by Tony Babarino, use r"C:\Test" instead of "C:\Test" in your code.
There are a few problems in your code, take a look at how I've modified it below:
import os
results = []
testdir = "C:\\Test"
for f in os.listdir(testdir):
if f.endswith('.pdf'):
results.append(f)
print (results)
Note that I have escaped your path name, and removed your first if folder.... That wasn't getting the folders as you expected, but rather selecting a character of the path string one at a time.
You will need to modify the code to get it to look through all folders, this currently doesn't. Take a look at the glob module.
You will need to escape the backslash on windows and you can use os.walk to get all the pdf files.
for root,dirs,files in os.walk(testdir):
for f in files:
if f.endswith('.pdf'):
results.append(f)
print (results)
You are basically iterating through the string testdir with the first for loop then passing each character to os.listdir(folder) does not make any sense then, just remove that first for loop and use fnmatch method from fnmatch module:
import os
from fnmatch import fnmatch
ext = '*.pdf'
results = []
testdir = "C:\Test"
for f in os.listdir(testdir):
if fnmatch(f, ext):
results.append(f)
print (results)
Try testdir = r"C:\Test" instead of testdir = "C:\Test". In python You have to escape special characters like for example \. You can escape them also with symbol '\' so it would be "C:\\Test". By using r"C:\Test", You are telling python to use raw string.
Also for folder in testdir: line doesn't make sense because testdir is a string so You are basically trying to iterate over a string.
I had to mention the names of training images for my Yolo model,
Here's what i did to print names of all images which i kept for training YoloV3 Model
import os
for root, dirs, files in os.walk("."):
for filename in files:
print(filename)
It prints out all the file names from the current directory
I have a number of videos in a directory on my Mac that all have a specific string in the file name that I want to remove, but I want to keep the rest of the file name as it is. I'm running this python script from terminal.
I have this syntax but it doesn't seem to work. Is it practical to use the following? It seems to simple to be the best way to do this sort of thing which is why I don't think it works.
from os import rename, listdir
text = "Some text I want to remove from file name"
files = listdir("/Users/Admin/Desktop/Dir_of_videos/")
for x in files:
if text in files:
os.rename(files, files.replace(text, ""))
the problem is that you get incomplete paths when you are using listdir, basically, it returns only the files in the directory without the prepending path to the directory
this should do the job:
import os
in_dir = './test'
remove = 'hello'
paths = [os.path.join(in_dir,file) for file in os.listdir(in_dir) if remove in file]
for file in paths:
os.rename(file, file.replace(remove, ""))
Sometimes it is useful to substitute spaces with underscore. On linux machine, this works fine for me:
find /tmp/ -depth -name "* *" -execdir rename 's/ /_/g' "{}" \;
On windows, I'd like to do it using ipython. I think many people might have met this problem, how do you implement it in ipython?
Thank you!
Edit:
My apologize for the misunderstanding. Here is my script:
import os
def rm_space():
for filename in os.listdir("."):
if filename.find(" ") > 0:
newfilename = filename.replace(" ", "_")
os.rename(filename, newfilename)
This piece of code does substitute the spaces with underscore; however, there's a problem: How to substitute recursively?
I think this is a very common problem, that there might be already an idiomatic way to solve it, (just like the shell script above).
Code copy from stackoverflow and edit, works on my Mac.
import os
import sys
directory = sys.argv[1] # parse through file list in the current directory
for filename in os.listdir(directory): # parse through file list in the current directory
if filename.find(" ") > 0: # if an space is found
newfilename = filename.replace(" ","_") # convert underscores to space's
old_file_path = os.path.join(directory, filename)
new_file_path = os.path.join(directory, newfilename)
os.rename(old_file_path, new_file_path) # rename the file, note that arg[] of os.rename is path_of_file, that explains 2 lines of code above
How to substitute recursively?
Use OS.walk() rather than listdir
I wish to read incoming XML files that have no specific name (e.g. date/time naming) to extract values and perform particular tasks.
I can step through the files but am having trouble opening them and reading them.
What I have that works is:-
import os
path = 'directory/'
listing = os.listdir(path)
for infile in listing:
print infile
But when I add the following to try and read the files it errors saying No such file or directory.
file = open(infile,'r')
Thank you.
You need to provide the path to file too:
file = open(os.path.join(path,infile),'r')
os.listdir provides the base names, not absolute paths. You'll need to do os.path.join(path, infile) instead of just infile (that may still be a relative path, which should be fine; if you needed an absolute path, you'd feed it through os.path.abspath).
As an alternative to joining the directory path and filename as in the other answers, you can use the glob module. This can also be handy when your directories might contain other (non-XML) files that you don't want to process:
import glob
for infile in glob.glob('directory/*.xml'):
print infile
You have to join the directory path and filename, using
os.path.join(path, infile)
Also use the path without the / :
path = 'directory'
Something like this (Not optimized, just a small change in your code):
import os
path = 'directory'
listing = os.listdir(path)
for infile in listing:
print infile
file_abs = os.path.join(path, infile)
file = open(file_abs,'r')