Hi everyone I´m trying now to move some files from one folder to another. The files that i have are
results_1_1
results_2_1
results_3_1
results_1_2
results_2_2
results_3_2
What i´d like to get is to move files
results_1_1
results_2_1
results_3_1 to one folder let´s call it A
other
results_1_2
results_2_2
results_3_2 to another let´s call it B
I don´t want to do it manually because, in reality, I have much more files
I have created by python a new folder that I want to have
and so far about moving the files to another directory
import shutil
source_folder = r"C:/Users/...."
destination_folder= r"C:/Users/...."
files_to_move = ['results_1_3.dat']
for file in files_to_move:
source = source_folder + file
destination = destination_folder + file
shutil.move(source,destination)
print("Moved!")
But with this program I can only move one file at a time I tried writing results_.+3.dat', results*_3.dat' but I´m seeing an error all the time
You can use os.listdir to find all the files in your directory.
This code finds all the files which names end with '1'.
import os
files_to_move = [f for f in os.listdir() if f.split('.')[0][-1] == '1']
You can also add an extension check if necessary:
files_to_move = [f for f in os.listdir() if f.split('.')[0][-1] == '1' and f.split('.')[1] == 'dat']
You can list all files in your directory using os.listdir(source_folder)
If you have mixed files in your directory, you can select your results files with something like this
files_to_move = [x for x in os.listdir(source_folder) if all(j in x for j in ['results', '.dat'])]
I would then create a dictionary containing the destination folders associated with the numbers in your results file.
destination_A = r'/path/to/destinationA'
destination_B = r'/path/to/destinationB'
moving_dict = {
'1': destination_A,
'2': destination_B
}
You can then move your files based on the filename
for file in files_to_move:
destination_folder = moving_dict[file[:-4].split('_')[-1]]
source = source_folder + file
destination = destination_folder + file
shutil.move(source,destination)
Related
I've tried to write some code which will rename some files in a folder - essentially, they're listed as xxx_(a).bmp whereas they need to be xxx_a.bmp, where a runs from 1 to 2000.
I've used the inbuilt os.rename function to essentially swap them inside of a loop to get the right numbers, but this gives me FileNotFoundError [WinError2] the system cannot find the file specified Z:/AAA/BBB/xxx_(1).bmp' -> 'Z:/AAA/BBB/xxx_1.bmp'.
I've included the code I've written below if anyone could point me in the right direction. I've checked that I'm working in the right directory and it gives me the directory I'm expecting so I'm not sure why it can't find the files.
import os
n = 2000
folder = r"Z:/AAA/BBB/"
os.chdir(folder)
saved_path = os.getcwd()
print("CWD is" + saved_path)
for i in range(1,n):
old_file = os.path.join(folder, "xxx_(" + str(i) + ").bmp")
new_file = os.path.join(folder, "xxx_" +str(i)+ ".bmp")
os.rename(old_file, new_file)
print('renamed files')
The problem is os.rename doesn't create a new directory if the new name is a filename in a directory that does not currently exist.
In order to create the directory first, you can do the following in Python3:
os.makedirs(dirname, exist_ok=True)
In this case dirname can contain created or not-yet-created subdirectories.
As an alternative, one may use os.renames, which handles new and intermediate directories.
Try iterating files inside the directory and processing the files that meet your criteria.
from pathlib import Path
import re
folder = Path("Z:/AAA/BBB/")
for f in folder.iterdir():
if '(' in f.name:
new_name = f.stem.replace('(', '').replace(')', '')
# using regex
# new_name = re.sub('\(([^)]+)\)', r'\1', f.stem)
extension = f.suffix
new_path = f.with_name(new_name + extension)
f.rename(new_path)
I have two folders with images. Let the two folder names A and B. A contains 100 files and B has only 80 files. Both the files have the same name. I want to save only the 80 files from A which has the same correspondence to B in folder C.
Here is a part of my code. However, it is throwing error :
Required argument 'img' (pos 2) not found.
path1= '/home/vplab/Kitty/Saliency Dataset/PiCANet-Implementation/TrainSet/images'
path_mask= '/home/vplab/Kitty/Saliency Dataset/PiCANet-Implementation/TrainSet/masks'
save_path = '/home/vplab/Kitty/Saliency Dataset/PiCANet-Implementation/TrainSet/exp'
for file in os.listdir(path1):
for file1 in os.listdir(path_mask):
img_name = file[:-4]
mask_name =file1[:-4]
if img_name == mask_name:
cv2.imwrite(os.path.join(save_path,img_name))
Your issue here is that you are not passing a file object to cv2.imwrite(os.path.join(save_path,img_name)) when trying to perform the copy; that's what the error is telling you.
However, your current approach includes a nested for loop which will give poor performance. If you only want to know the files that the directories have in common, you can create a set of the file names in each directory and find the intersection. Then you just need to iterate through the common files and copy them over (as said in the comments, there's no need for cv2 here - they may be images but they're just regular files that can be copied).
import os
from shutil import copyfile
dir_1 = 'A'
dir_2 = 'B'
output_dir = 'C'
files_1 = os.listdir(dir_1)
files_2 = os.listdir(dir_2)
# Find the common files between both
common_files = set(files_1).intersection(files_2)
# Copy the common files over.
for file in common_files:
copyfile(os.path.join(dir_1, file),
os.path.join(output_dir, file))
If the reason that you are stripping the last characters from the files in os.listdir is because the files have the same name but different extensions, you only need to make two small modifications (where here I'm assuming the extension is .png that needs to be added back later):
files_1 = [item[:-4] for item in os.listdir(dir_1)]
files_2 = [item[:-4] for item in os.listdir(dir_2)]
And:
for file in common_files:
file = file + '.png' # Add the extension back on to the file name
copyfile(os.path.join(dir_1, file),
os.path.join(output_dir, file))
The any() method returns True if any element of an iterable is True. If not, any() returns False. shutil.copy - Copies the file src to the file or directory dst.
import os
import shutil
def read_file(folderName,folderPath):
''' Return list of files name '''
path = folderPath+folderName
return [file for file in os.listdir(path)]
def save_file(soureFolderName,destFolderName,folderPath,fileName):
''' Save file on destination folder'''
try:
source_path = folderPath+soureFolderName+"/"+fileName
dest_path = folderPath+destFolderName+"/"+fileName
shutil.copy(source_path, dest_path)
except Exception as e:
print(e)
base_path = '/home/vplab/Kitty/Saliency Dataset/PiCANet-Implementation/TrainSet/'
folder_images_files = read_file('images',base_path)
folder_masks_file = read_file('masks',base_path)
for file_1 in folder_images_files:
#Check folder A file is exists in folder B
if any(file_1 == file_2 for file_2 in folder_masks_file):
save_file("images","exp",base_path,file_1)
I am working with Python3 and am trying to change the names of files in multiple subdirectories to match the folder name. My directory looks like this:
path: C:\Users\Me\Project
In the path, I have the following folders: alldata, folderA, folderB, folderC
FolderA, folderB, and folderC, each contain a file called data.csv
I want to add the letter name of the folder (e.g., A, B, C) to the file (e.g., dataA.csv) inside the folder and then move all these renamed files to "alldata"
I really appreciate the help!
This one might be a little hardcoded, but is probably more understandable for people who are just starting out in Python:
import os
import shutil
# Enter the 'Project' folder
os.chdir('C:\\Users\\Me\\Project')
# Filter for folders that contain the word 'folder'
folders = [folder for folder in os.listdir() if 'folder' in folder]
for folder in folders:
# Get the last letter of every folder
suffix = folder[-1]
# Build source and destination path for the csv files
source = folder + '\\data.csv'
dest = 'alldata\\data' + suffix + '.csv'
shutil.move(source, dest)
The 'os' module in python gives you access to functions that deal with folders and files. For example, there are functions in the os module to move, copy, rename, delete folders and files. Try this for example:
import os
basePath = "C:\\Users\\Me\\Project\\"
# Rename and move the data.csv file in folderA to dataA.csv in the alldata folder
os.rename(basePath + "folderA\\data.csv", basePath + "alldata\\dataA.csv")
# Rename and move the data.csv file in folderB to dataB.csv in the alldata folder
os.rename(basePath + "folderB\\data.csv", basePath + "alldata\\dataB.csv")
# Rename and move the data.csv file in folderC to dataC.csv in the alldata folder
os.rename(basePath + "folderC\\data.csv", basePath + "alldata\\dataC.csv")
# Make sure that they moved as intended
filesInAllDataFolder = os.listdir(basePath + "alldata\\")
print(filesInAllDataFolder)
The os module is super handy and I guarantee you'll use it a lot, so play with it!
This works for me:
import os
def scan_dir(folder):
for name in os.listdir(folder):
path = os.path.join(folder, name)
if os.path.isfile(path):
if 'data' in path:
dir_name = path.split('/')[-2]
new_name_path = path[:-3]+dir_name+'.csv'
new_name_path = new_name_path.split('/')
new_name_path[-2] = 'alldata'
new_name_path = "/".join(new_name_path)
os.rename(path, new_name_path)
else:
scan_dir(path)
directory = 'C:\Users\Me\Project'
scan_dir(directory)
I have 400 folders. Within each folder, there are 3 additional folders I have to drill into and then in the last folder I have 22 different files. Each of these 400 folders have the same named files. Ultimately, I want to create 22 folders, one for each file name and within each of the 22 folders I want only files that have the same name. Since a folder cannot have two files with the same name, adding the final 6 characters of the file path to the file name solves the issue.
Using the image below as an example of the files:
I need an ACO folder with 400 files all of which are labeled i.e., "ACO050168", "ACO150074", "ACO150128"
I need an BDUAL folder with 400 files all of which are labeled BDUAL i.e., "BDUAL050168", "BDUAL150074", "BDUAL150128"
etc..
Here are file paths for example, notice that the final folder and the 4th from the last folder are unique
'C:\\Users\\MBobak\\Documents\\Loop\\Raw Claims Data 050168\\D#CJR\\Y2018M02\\P050168',
'C:\\Users\\MBobak\\Documents\\Loop\\Raw Claims Data 150074\\D#CJR\\Y2018M02\\P150074',
'C:\\Users\\MBobak\\Documents\\Loop\\Raw Claims Data 150128\\D#CJR\\Y2018M02\\P150128'
The 22 files within each folder are structured as .csv but do not have the .csv extension. They are just files. I attached an image to show what it looks like.
I know there are ways to do this in the command-line, but I can't find a specific example of this question since there are drills downs from each initial folder.
I attempted to do this in python, and I wrote a small script in python to find the final directories that hold the 22 files:
all_files_dir = r'C:\Users\MBobak\Documents\Loop'
folders = os.listdir(all_files_dir)
ccn_list = [x[-6:] for x in folders]
ccn_file_dir = [r'C:\Users\MBobak\Documents\Loop\Raw Claims Data ' + x + '\D#CJR\Y2018M02\P' + x for x in ccn_list]
But now I am having trouble copying/moving these into new folders.
So - not enough reputation to comment and ask for more info. To me it sounds like you have similar filenames in the bottom of the structure that you want copied to the same target folder?
Ie you want all ACO files to end up in a folder named ACO??? You can only have one file with that name in a folder.
So this first version will actually overwrite any similar named file in the target folder. Can you clarify what you really need? Ie an example of the target folder contents and filenames.
# folder.py
import os
import sys
import platform
import shutil
walk_dir = sys.argv[1]
if platform.system() == 'Windows':
delimiter = '\\'
else:
delimiter = '/'
print('walk_dir = ' + walk_dir)
print('walk_dir (absolute) = ' + os.path.abspath(walk_dir))
for root, subdirs, files in os.walk(walk_dir):
print('--\ncurrent = ' + root)
for filename in files:
file_path = os.path.join(root, filename)
print('\t- file %s (full path: %s)' % (filename, file_path))
target_name = root[root.rfind(delimiter)+2:len(root)]
if not os.path.isdir(filename):
os.mkdir(filename)
target = filename + delimiter + filename + target_name
shutil.copy2(file_path,target)
Call it like this:
C:\\Users\\MBobak\\Documents\\python folder.py Loop
If you can give me more information I may be able to give you a better answer.
So after some helpful suggestions and intense googling, I came up with this below which does what I need. This script will go into each folder and rename each file as it copies it into another folder, which is titled the original file name. The final result has 22 folders (as described in the question) with 400 files each. The code below includes the snippet from the question stem.
import os, shutil
dest = r'C:\Users\MBobak\Documents\Target_Folder'
all_folders_dir = r'C:\Users\MBobak\Documents\Staging_Folder'
files_22 = r'C:\Users\MBobak\Documents\Staging_Folder\Raw Claims Data 050069\D#CJR\Y2018M02\P050069' ## pulling list of all the named files
folders = os.listdir(all_folders_dir)
files = os.listdir(files_22)
ccn_list = [x[-6:] for x in folders]
ccn_file_dir = [r'C:\Users\MBobak\Documents\Raw Claims Data ' + x + '\D#CJR\Y2018M02\P' + x for x in ccn_list]
for file in files:
for hosp_folder, ccn in zip(ccn_file_dir, ccn_list):
if not os.path.exists(dest+'\\'+file):
os.mkdir(dest+'\\'+file)
shutil.copy(hosp_folder+"\\"+file ,dest +'\\'+ file +'\\'+ file + '_' + ccn)
I have 3 folders. 1)Content 2) Shared 3)MoveHere
I have 50 txt files in "Content" folder and 300 text files in "Shared" folder.
here, names of 50 txt files in "Content" folder matches with names of files in "Shared" folder.
What we need here is, move those matched files from "Shared" folder to MoveHere folder.
I tried below code, but it did not work.
#!/usr/bin/env python
import os
import shutil
# Get current working directory
CD = os.getcwd()
SUD = '/D:/TestScript/shareduser/'
DEST = '/D:/TestScript/MoveHere/'
# Get a list of files in the current working directory
for file_CD in os.listdir(CD):
file_name_in_Content = os.path.basename(os.path.splitext(file_CD)[0])
#print('CWD '+ file_name_in_Content)
for file_SUD in os.listdir(SUD):
file_name_in_SharedUser = os.path.basename(os.path.splitext(file_SUD)[0])
if file_name_in_Content == file_name_in_SharedUser:
SRC_FULL_PATH = SUD + "/" + file_SUD
DEST_FULL_PATH = DEST + "/" + file_SUD
shutil.move(SRC_PATH, DEST_PATH)
print ("\nDone")
I appreciate all your help.