What is the easiest way to copy files from multiple directories into just one directory using python? To be more clear, I have a tree that looks like this
+Home_Directory
++folder1
-csv1.csv
-csv2.csv
++folder2
-csv3.csv
-csv4.csv
and I want to put csv1,csv2,...etc all into some specified directory without the folder hierarchy.
+some_folder
-csv1.csv
-csv2.csv
-csv3.csv
-csv4.csv
Some solutions I have looked at:
Using shutil.copytree will not work because it will preserve the file structure which is not what I want.
The code I am playing with is very similar to what is posted in this question:
copy multiple files in python
the problem is that I do not know how to do this iteratively. Presumably it would just be another for loop on top of this but I am not familiar enough with the os and shutil libraries to know exactly what I am iterating over. Any help on this?
This is what I thought of. I am assuming you are only pulling csv files from 1 directory.
RootDir1 = r'*your directory*'
TargetFolder = r'*your target folder*'
for root, dirs, files in os.walk((os.path.normpath(RootDir1)), topdown=False):
for name in files:
if name.endswith('.csv'):
print "Found"
SourceFolder = os.path.join(root,name)
shutil.copy2(SourceFolder, TargetFolder) #copies csv to new folder
Edit: missing a ' at the end of RootDir1. You can also use this as a starting guide to make it work as desired.
import glob
import shutil
#import os
dst_dir = "E:/images"
print ('Named explicitly:')
for name in glob.glob('E:/ms/*/*/*'):
if name.endswith(".jpg") or name.endswith(".pdf") :
shutil.copy(name, dst_dir)
print ('\t', name)
You can use it to move all subfolders from the same to a different directory to wherever you want.
import shutil
import os
path=r'* Your Path*'
arr = os.listdir(path)
for i in range(len(arr)):
source_dir=path+'/'+arr[i]
target_dir = r'*Target path*'
file_names = os.listdir(source_dir)
for file_name in file_names:
shutil.move(os.path.join(source_dir, file_name), target_dir)
Related
I'm trying to write a python script to move all music files from my whole pc to one spcific folder.
They are scattered everywhere and I want to get them all in one place, so I don't want to copy but completely move them.
I was already able to make a list of all the files with this script:
import os
targetfiles = []
extensions = (".mp3", ".wav", ".flac")
for root, dirs, files in os.walk('/'):
for file in files:
if file.endswith(extensions):
targetfiles.append(os.path.join(root, file))
print(targetfiles)
This prints out a nice list of all the files but I'm stuck to now move them.
I did many diffent tries with different code and this was one of them:
import os
import shutil
targetfiles = []
extensions = (".mp3", ".wav", ".flac")
for root, dirs, files in os.walk('/'):
for file in files:
if file.endswith(extensions):
targetfiles.append(os.path.join(root, file))
new_path = 'C:/Users/Nicolaas/Music/All' + file
shutil.move(targetfiles, new_path)
But everything I try gives me an error:
TypeError: rename: src should be string, bytes or os.PathLike, not list
I think I've met my limit gathering this all as I'm only starting at Python but I would be very grateful if anyone could point me in the right direction!
You are trying to move a list of files to a new location, but the shutil.move function expects a single file as the first argument. To move all the files in the targetfiles list to the new location, you have to use a loop to move each file individually.
for file in targetfiles:
shutil.move(file, new_path)
Also if needed add a trailing slash to the new path 'C:/Users/Nicolaas/Music/All/'
On a sidenote are you sure that moving all files with those extentions is a good idea? I would suggest copying them or having a backup.
Edit:
You can use an if statement to exclude certain folders from being searched.
for root, dirs, files in os.walk('/'):
if any(folder in root for folder in excluded_folders):
continue
for file in files:
if file.endswith(extensions):
targetfiles.append(os.path.join(root, file))
Where excluded_folder is a list of the unwanted folders like: excluded_folders = ['Program Files', 'Windows']
I would suggest using glob for matching:
import glob
def match(extension, root_dir):
return glob.glob(f'**\\*.{extension}', root_dir=root_dir, recursive=True)
root_dirs = ['C:\\Path\\to\\Albums', 'C:\\Path\\to\\dir\\with\\music\\files']
excluded_folders = ['Bieber', 'Eminem']
extensions = ("mp3", "wav", "flac")
targetfiles = [f'{root_dir}\\{file_name}' for root_dir in root_dirs for extension in extensions for file_name in match(extension, root_dir) if not any(excluded_folder in file_name for excluded_folder in excluded_folders)]
Then you can move these files to new_path
I am trying to write a program in python that loops through data from various csv files within a folder. Right now I just want to see that the program can identify the files in a folder but I am unable to have my code print the file names in my folder. This is what I have so far, and I'm not sure what my problem is. Could it be the periods in the folder names in the file path?
import glob
path = "Users/Sarah/Documents/College/Lab/SEM EDS/1.28.20 CZTS hexane/*.csv"
for fname in glob.glob(path):
print fname
No error messages are popping up but nothing will print. Does anyone know what I'm doing wrong?
Are you on a Linux-base system ? If you're not, switch the / for \\.
Is the directory you're giving the full path, from the root folder ? You might need to
specify a FULL path (drive included).
If that still fails, silly but check there actually are files in there, as your code otherwise seems fine.
This code below worked for me, and listed csv files appropriately (see the C:\\ part, could be what you're missing).
import glob
path = "C:\\Users\\xhattam\\Downloads\\TEST_FOLDER\\*.csv"
for fname in glob.glob(path):
print(fname)
The following code gets a list of files in a folder and if they have csv in them it will print the file name.
import os
path = r"C:\temp"
filesfolders = os.listdir(path)
for file in filesfolders:
if ".csv" in file:
print (file)
Note the indentation in my code. You need to be careful not to mix tabs and spaces as theses are not the same to python.
Alternatively you could use os
import os
files_list = os.listdir(path)
out_list = []
for item in files_list:
if item[-4:] == ".csv":
out_list.append(item)
print(out_list)
Are you sure you are using the correct path?
Try moving the python script in the folder when the CSV files are, and then change it to this:
import glob
path = "./*.csv"
for fname in glob.glob(path):
print fname
Im a noob to python and I am trying to complete this simple task. I want to access multiple directories one by one that are all located inside one directory. I do not have the names of the multiple directories. I need to enter into the directory, combine some files, move back out of that directory, then into the next directory, combine some files, move back out of it, and so on........ I need to make sure I dont access the same directory more than once.
I looked around and tried various commands and nothing yet.
try using something like the following piece of code.:
import os, fnmatch
def find_files(directory, pattern):
for root, dirs, files in os.walk(directory):
for basename in files:
if fnmatch.fnmatch(basename, pattern):
filename = os.path.join(root, basename)
yield filename
use it something like this:
for filename in find_files('/home/', '*.html')
# do something
Sometimes I find glob to be useful:
from glob import glob
import os
nodes = glob('/tmp/*/*')
for node in nodes:
try:
print 'now in directory {}'.format(os.path.dirname(node))
with open(node, 'r') as f:
# Do something with f...
print len(f.read())
except IOError: # Because node may be a directory, which we cannot 'open'
continue
I want to make a Python script to quickly organize my files on my desktop into folders based on extension. Basically, how could I use a loop to take a file, do something to it, move on to the next file, and so on?
Everything you need is probably contained in the os library, more specifically in the os.path bit of it and the shutil one.
To explore a directory tree you can use os.walk and to move files around you can use shutil.move.
EDIT: a small script I hacked together to get you going:
import os
import shutil as sh
from collections import defaultdict
DESKTOP = '/home/mac/Desktop'
#This dictionary will contain: <extension>: <list_of_files> mappings
register = defaultdict(list)
#Populate the register
for dir_, dirs, fnames in os.walk('/home/mac/Desktop'):
for fname in fnames:
register[fname.rsplit('.', 1)[1]].append(os.path.join(dir_, fname))
#Iterate over the register, creating the directory and moving the files
#with that extension in it.
for dirname, files in register.iteritems():
dirname = os.path.join(DESKTOP, dirname)
if not os.path.exists(dirname):
os.makedirs(dirname)
for file_ in files:
sh.move(file_, dirname)
I would recommend os.walk from the module os and list of the filenames
Hi guys have a problem with renaming a set of files in separate folder in python. i have a folder structure like
images/p_0/aaa.jpg,bbb.jpg
images/p_1/aaa.jpg,bbb.jpg
images/p_2/aaa.jpg,bbb.jpg
I need to rename these jpg to like
images/p_0/A-1.jpg,B-1.jpg
images/p_1/A-2.jpg,B-2.jpg
images/p_2/A-3.jpg,B-3.jpg
and i am using the os.rename method. I just don't get how to cycle through the folders and find the image and rename it.
Use os.walk and os.rename.
import os
for root, dirs, files in os.walk(top, topdown=False):
for name in files:
#Rename your files and use os.path.join(root, name)
You can use os.listdir() to get all the files in some directory. You might also want to look through the rest of the os module documentation.
Take a look at os.listdir or os.walk, if you have a nested directory structure.
#!/usr/bin/env python
import os
# list all files
for filename in os.listdir('.'):
print filename # do something with filename ...
def is_image(filename):
""" Tiny filter function. """
return filename.endswith(".jpg") or filename.endswith(".png")
# only consider files, which pass the `is_image` filter
for filename in filter(is_image, os.listdir('.')):
print filename # do something with filename ...
In addition to os.walk(), glob.glob() might be useful.