Im a noob to python and I am trying to complete this simple task. I want to access multiple directories one by one that are all located inside one directory. I do not have the names of the multiple directories. I need to enter into the directory, combine some files, move back out of that directory, then into the next directory, combine some files, move back out of it, and so on........ I need to make sure I dont access the same directory more than once.
I looked around and tried various commands and nothing yet.
try using something like the following piece of code.:
import os, fnmatch
def find_files(directory, pattern):
for root, dirs, files in os.walk(directory):
for basename in files:
if fnmatch.fnmatch(basename, pattern):
filename = os.path.join(root, basename)
yield filename
use it something like this:
for filename in find_files('/home/', '*.html')
# do something
Sometimes I find glob to be useful:
from glob import glob
import os
nodes = glob('/tmp/*/*')
for node in nodes:
try:
print 'now in directory {}'.format(os.path.dirname(node))
with open(node, 'r') as f:
# Do something with f...
print len(f.read())
except IOError: # Because node may be a directory, which we cannot 'open'
continue
Related
I am new to python. I have successful written a script to search for something within a file using :
open(r"C:\file.txt) and re.search function and all works fine.
Is there a way to do the search function with all files within a folder? Because currently, I have to manually change the file name of my script by open(r"C:\file.txt),open(r"C:\file1.txt),open(r"C:\file2.txt)`, etc.
Thanks.
You can use os.walk to check all the files, as the following:
import os
for root, _, files in os.walk(path):
for filename in files:
with open(os.path.join(root, filename), 'r') as f:
#your code goes here
Explanation:
os.walk returns tuple of (root path, dir names, file names) in the folder, so you can iterate through filenames and open each file by using os.path.join(root, filename) which basically joins the root path with the file name so you can open the file.
Since you're a beginner, I'll give you a simple solution and walk through it.
Import the os module, and use the os.listdir function to create a list of everything in the directory. Then, iterate through the files using a for loop.
Example:
# Importing the os module
import os
# Give the directory you wish to iterate through
my_dir = <your directory - i.e. "C:\Users\bleh\Desktop\files">
# Using os.listdir to create a list of all of the files in dir
dir_list = os.listdir(my_dir)
# Use the for loop to iterate through the list you just created, and open the files
for f in dir_list:
# Whatever you want to do to all of the files
If you need help on the concepts, refer to the following:
for looops in p3: http://www.python-course.eu/python3_for_loop.php
os function Library (this has some cool stuff in it): https://docs.python.org/2/library/os.html
Good luck!
You can use the os.listdir(path) function:
import os
path = '/Users/ricardomartinez/repos/Salary-API'
# List for all files in a given PATH
file_list = os.listdir(path)
# If you want to filter by file type
file_list = [file for file in os.listdir(path) if os.path.splitext(file)[1] == '.py']
# Both cases yo can iterate over the list and apply the operations
# that you have
for file in file_list:
print(file)
#Operations that you want to do over files
What is the easiest way to copy files from multiple directories into just one directory using python? To be more clear, I have a tree that looks like this
+Home_Directory
++folder1
-csv1.csv
-csv2.csv
++folder2
-csv3.csv
-csv4.csv
and I want to put csv1,csv2,...etc all into some specified directory without the folder hierarchy.
+some_folder
-csv1.csv
-csv2.csv
-csv3.csv
-csv4.csv
Some solutions I have looked at:
Using shutil.copytree will not work because it will preserve the file structure which is not what I want.
The code I am playing with is very similar to what is posted in this question:
copy multiple files in python
the problem is that I do not know how to do this iteratively. Presumably it would just be another for loop on top of this but I am not familiar enough with the os and shutil libraries to know exactly what I am iterating over. Any help on this?
This is what I thought of. I am assuming you are only pulling csv files from 1 directory.
RootDir1 = r'*your directory*'
TargetFolder = r'*your target folder*'
for root, dirs, files in os.walk((os.path.normpath(RootDir1)), topdown=False):
for name in files:
if name.endswith('.csv'):
print "Found"
SourceFolder = os.path.join(root,name)
shutil.copy2(SourceFolder, TargetFolder) #copies csv to new folder
Edit: missing a ' at the end of RootDir1. You can also use this as a starting guide to make it work as desired.
import glob
import shutil
#import os
dst_dir = "E:/images"
print ('Named explicitly:')
for name in glob.glob('E:/ms/*/*/*'):
if name.endswith(".jpg") or name.endswith(".pdf") :
shutil.copy(name, dst_dir)
print ('\t', name)
You can use it to move all subfolders from the same to a different directory to wherever you want.
import shutil
import os
path=r'* Your Path*'
arr = os.listdir(path)
for i in range(len(arr)):
source_dir=path+'/'+arr[i]
target_dir = r'*Target path*'
file_names = os.listdir(source_dir)
for file_name in file_names:
shutil.move(os.path.join(source_dir, file_name), target_dir)
I want to make a Python script to quickly organize my files on my desktop into folders based on extension. Basically, how could I use a loop to take a file, do something to it, move on to the next file, and so on?
Everything you need is probably contained in the os library, more specifically in the os.path bit of it and the shutil one.
To explore a directory tree you can use os.walk and to move files around you can use shutil.move.
EDIT: a small script I hacked together to get you going:
import os
import shutil as sh
from collections import defaultdict
DESKTOP = '/home/mac/Desktop'
#This dictionary will contain: <extension>: <list_of_files> mappings
register = defaultdict(list)
#Populate the register
for dir_, dirs, fnames in os.walk('/home/mac/Desktop'):
for fname in fnames:
register[fname.rsplit('.', 1)[1]].append(os.path.join(dir_, fname))
#Iterate over the register, creating the directory and moving the files
#with that extension in it.
for dirname, files in register.iteritems():
dirname = os.path.join(DESKTOP, dirname)
if not os.path.exists(dirname):
os.makedirs(dirname)
for file_ in files:
sh.move(file_, dirname)
I would recommend os.walk from the module os and list of the filenames
Hi guys have a problem with renaming a set of files in separate folder in python. i have a folder structure like
images/p_0/aaa.jpg,bbb.jpg
images/p_1/aaa.jpg,bbb.jpg
images/p_2/aaa.jpg,bbb.jpg
I need to rename these jpg to like
images/p_0/A-1.jpg,B-1.jpg
images/p_1/A-2.jpg,B-2.jpg
images/p_2/A-3.jpg,B-3.jpg
and i am using the os.rename method. I just don't get how to cycle through the folders and find the image and rename it.
Use os.walk and os.rename.
import os
for root, dirs, files in os.walk(top, topdown=False):
for name in files:
#Rename your files and use os.path.join(root, name)
You can use os.listdir() to get all the files in some directory. You might also want to look through the rest of the os module documentation.
Take a look at os.listdir or os.walk, if you have a nested directory structure.
#!/usr/bin/env python
import os
# list all files
for filename in os.listdir('.'):
print filename # do something with filename ...
def is_image(filename):
""" Tiny filter function. """
return filename.endswith(".jpg") or filename.endswith(".png")
# only consider files, which pass the `is_image` filter
for filename in filter(is_image, os.listdir('.')):
print filename # do something with filename ...
In addition to os.walk(), glob.glob() might be useful.
I know how to delete single files, however I am lost in my implementation of how to delete all files in a directory of one type.
Say the directory is \myfolder
I want to delete all files that are .config files, but nothing to the other ones.
How would I do this?
Thanks Kindly
Use the glob module:
import os
from glob import glob
for f in glob ('myfolder/*.config'):
os.unlink (f)
I would do something like the following:
import os
files = os.listdir("myfolder")
for f in files:
if not os.path.isdir(f) and ".config" in f:
os.remove(f)
It lists the files in a directory and if it's not a directory and the filename has ".config" anywhere in it, delete it. You'll either need to be in the same directory as myfolder, or give it the full path to the directory. If you need to do this recursively, I would use the os.walk function.
Here ya go:
import os
# Return all files in dir, and all its subdirectories, ending in pattern
def gen_files(dir, pattern):
for dirname, subdirs, files in os.walk(dir):
for f in files:
if f.endswith(pattern):
yield os.path.join(dirname, f)
# Remove all files in the current dir matching *.config
for f in gen_files('.', '.config'):
os.remove(f)
Note also that gen_files can be easily rewritten to accept a tuple of patterns, since str.endswith accepts a tuple