We Keep Coding

How to move files based on their names with python? [duplicate]

This question already has answers here:
Moving all files from one directory to another using Python
(11 answers)
Closed 6 months ago.
I have a folder with a lot of tutorial links, so I wanted to create a script that reads the file name and for instance, if the file has in its name the word "VBA" or "Excel" it would create the folder Excel and send to it. The same would happen with files containing the word "python".
The code is running, but nothing happens and the files still in the same directory. Does anyone have an idea of what I'm doing wrong?
Here is what I have in the folder, all files are links to youtube tutorials or websites:
Please see my code below:
import os
import shutil
os.chdir(r"C:\Users\RMBORGE\Desktop\Useful stuff")
path = r"C:\Users\RMBORGE\Desktop\Useful stuff\Excel"
for f in os.listdir():
if "vba" in f:
shutil.move(os.chdir,path)

Try this
import os
import shutil
path_to_files = 'some path'
files = os.listdir(path_to_files)
for f in files:
if 'Excel' in f:
created_folder = os.path.join(path_to_files, 'Excel')
filepath = os.path.join(path_to_files, f)
os.makedirs(created_folder, exist_ok=True)
shutil.move(filepath, created_folder)
NB: You can add more if statements for different keywords like Excel

Use pathlib mkdir for creating the folders. Prepare the folders/keywords you want sort in the list 'folders'. And then what is important is skipping the folders because os.listdir() gives you the folders aswell (and there is an error if you want to move a folder into itself)
import os
import shutil
import pathlib
folders = ["vba", "Excel"]
path = "/home/vojta/Desktop/THESIS/"
for f in os.listdir():
if not os.path.isdir(os.path.join(path, f)): # skip folders
for fol in folders:
if fol in f:
pathlib.Path(os.path.join(path, fol)).mkdir(parents=True, exist_ok=True)
fol_path = os.path.join(path, fol)
shutil.move(os.path.join(path, f), os.path.join(fol_path, f))

looping through multiple folders to clear items in each folder

I have a folder that contains a lot of subfolders, with images saved as png in each folder :
For example :
emotion\angry
emotion\disgusted
emotion\fearful
emotion\happy
I can remove the images in one of the folder using the below code :
folder_path = (r'C:\Users\emotion\angry')
test = os.listdir(folder_path)
for images in test:
if images.endswith(".png"):
os.remove(os.path.join(folder_path, images))
How do I create a loop to loop through each subfolder in emotion/?
As I do not want to manually write out all the code to clear all the folders...

You can list files using glob patterns and delete them using your normal os.remove.
import os
import glob
fileList = glob.glob('C:\Users\emotion\*\*.png')
for filePath in fileList:
try:
os.remove(filePath)
except:
print("Error while deleting file : ", filePath)

def all_dirs(directory):
# Returning list with all directories in selected directory
dir_list = []
for dir in [x[0] for x in os.walk(directory)]:
dir_list.append(dir)
return dir_list
This solution should work for every OS.

Python: Loop to open multiple folders and files in python

I am new to python and currently work on data analysis.
I am trying to open multiple folders in a loop and read all files in folders.
Ex. working directory contains 10 folders needed to open and each folder contains 10 files.
My code for open each folder with .txt file;
file_open = glob.glob("home/....../folder1/*.txt")
I want to open folder 1 and read all files, then go to folder 2 and read all files... until folder 10 and read all files.
Can anyone help me how to write loop to open folder, included library needed to be used?
I have my background in R, for example, in R I could write loop to open folders and files use code below.
folder_open <- dir("......./main/")
for (n in 1 to length of (folder_open)){
file_open <-dir(paste0("......./main/",folder_open[n]))
for (k in 1 to length of (file_open){
file_open<-readLines(paste0("...../main/",folder_open[n],"/",file_open[k]))
//Finally I can read all folders and files.
}
}

This recursive method will scan all directories within a given directory and then print the names of the txt files. I kindly invite you to take it forward.
import os
def scan_folder(parent):
# iterate over all the files in directory 'parent'
for file_name in os.listdir(parent):
if file_name.endswith(".txt"):
# if it's a txt file, print its name (or do whatever you want)
print(file_name)
else:
current_path = "".join((parent, "/", file_name))
if os.path.isdir(current_path):
# if we're checking a sub-directory, recursively call this method
scan_folder(current_path)
scan_folder("/example/path") # Insert parent direcotry's path

Given the following folder/file tree:
C:.
├───folder1
│ file1.txt
│ file2.txt
│ file3.csv
│
└───folder2
file4.txt
file5.txt
file6.csv
The following code will recursively locate all .txt files in the tree:
import os
import fnmatch
for path,dirs,files in os.walk('.'):
for file in files:
if fnmatch.fnmatch(file,'*.txt'):
fullname = os.path.join(path,file)
print(fullname)
Output:
.\folder1\file1.txt
.\folder1\file2.txt
.\folder2\file4.txt
.\folder2\file5.txt

Your glob() pattern is almost correct. Try one of these:
file_open = glob.glob("home/....../*/*.txt")
file_open = glob.glob("home/....../folder*/*.txt")
The first one will examine all of the text files in any first-level subdirectory of home/......, whatever that is. The second will limit itself to subdirectories named like "folder1", "folder2", etc.
I don't speak R, but this might translate your code:
for filename in glob.glob("......../main/*/*.txt"):
with open(filename) as file_handle:
for line in file_handle:
# perform data on each line of text

I think nice way to do that would be to use os.walk. That will generate tree and you can then iterate through that tree.
import os
directory = './'
for d in os.walk(directory):
print(d)

This code will look for all directories inside a directory, printing out the names of all files found there:
#--------*---------*---------*---------*---------*---------*---------*---------*
# Desc: print filenames one level down from starting folder
#--------*---------*---------*---------*---------*---------*---------*---------*
import os, fnmatch, sys
def find_dirs(directory, pattern):
for item in os.listdir(directory):
if os.path.isdir(os.path.join(directory, item)):
if fnmatch.fnmatch(item, pattern):
filename = os.path.join(directory, item)
yield filename
def find_files(directory, pattern):
for item in os.listdir(directory):
if os.path.isfile(os.path.join(directory, item)):
if fnmatch.fnmatch(item, pattern):
filename = os.path.join(directory, item)
yield filename
#--------*---------*---------*---------*---------*---------*---------*---------#
while True:# M A I N L I N E #
#--------*---------*---------*---------*---------*---------*---------*---------#
# # Set directory
os.chdir("C:\\Users\\Mike\\\Desktop")
for filedir in find_dirs('.', '*'):
print ('Got directory:', filedir)
for filename in find_files(filedir, '*'):
print (filename)
sys.exit() # END PROGRAM

pathlib is a good choose
from pathlib import Path
# or use: glob('**/*.txt')
for txt_path in [_ for _ in Path('demo/test_dir').rglob('*.txt') if _.is_file()]:
print(txt_path.absolute())

Move pairs of files (.txt & .xml) into their corresponding folder using Python

I have been working this challenge for about a day or so. I've looked at multiple questions and answers asked on SO and tried to 'MacGyver' the code used for my purpose, but still having issues.
I have a directory (lets call it "src\") with hundreds of files (.txt and .xml). Each .txt file has an associated .xml file (let's call it a pair). Example:
src\text-001.txt
src\text-001.xml
src\text-002.txt
src\text-002.xml
src\text-003.txt
src\text-003.xml
Here's an example of how I would like it to turn out so each pair of files are placed into a single unique folder:
src\text-001\text-001.txt
src\text-001\text-001.xml
src\text-002\text-002.txt
src\text-002\text-002.xml
src\text-003\text-003.txt
src\text-003\text-003.xml
What I'd like to do is create an associated folder for each pair and then move each pair of files into its respective folder using Python. I've already tried working from code I found (thanks to a post from Nov '12 by Sethdd, but am having trouble figuring out how to use the move function to grab pairs of files. Here's where I'm at:
import os
import shutil
srcpath = "PATH_TO_SOURCE"
srcfiles = os.listdir(srcpath)
destpath = "PATH_TO_DEST"
# grabs the name of the file before extension and uses as the dest folder name
destdirs = list(set([filename[0:9] for filename in srcfiles]))
def create(dirname, destpath):
full_path = os.path.join(destpath, dirname)
os.mkdir(full_path)
return full_path
def move(filename, dirpath):
shutil.move(os.path.join(srcpath, filename)
,dirpath)
# create destination directories and store their names along with full paths
targets = [
(folder, create(folder, destpath)) for folder in destdirs
]
for dirname, full_path in targets:
for filename in srcfile:
if dirname == filename[0:9]:
move(filename, full_path)
I feel like it should be easy, but Python isn't something I work with everyday and it's been a while since my scripting days... Any help would be greatly appreciated!
Thanks,
WK2EcoD

Use the glob module to interate all of the 'txt' files. From that you can parse and create the folders and copy the files.

The process should be as simple as it appears to you as a human.
for file_name in os.listdir(srcpath):
dir = file_name[:9]
# if dir doesn't exist, create it
# move file_name to dir
You're doing a lot of intermediate work that seems to be confusing you.
Also, insert some simple print statements to track data flow and execution flow. It appears that you have no tracing output so far.

You can do it with os module. For every file in directory check if associated folder exists, create if needed and then move the file. See the code below:
import os
SRC = 'path-to-src'
for fname in os.listdir(SRC):
filename, file_extension = os.path.splitext(fname)
if file_extension not in ['xml', 'txt']:
continue
folder_path = os.path.join(SRC, filename)
if not os.path.exists(folder_path):
os.mkdir(folderpath)
os.rename(
os.path.join(SRC, fname),
os.path.join(folder_path, fname)
)

My approach would be:
Find the pairs that I want to move (do nothing with files without a pair)
Create a directory for every pair
Move the pair to the directory
#! /usr/bin/env python
# -*- coding: utf-8 -*-
import os, shutil
import re
def getPairs(files):
pairs = []
file_re = re.compile(r'^(.*)\.(.*)$')
for f in files:
match = file_re.match(f)
if match:
(name, ext) = match.groups()
if ext == 'txt' and name + '.xml' in files:
pairs.append(name)
return pairs
def movePairsToDir(pairs):
for name in pairs:
os.mkdir(name)
shutil.move(name+'.txt', name)
shutil.move(name+'.xml', name)
files = os.listdir()
pairs = getPairs(files)
movePairsToDir(pairs)
NOTE: This script works when called inside the directory with the pairs.

Loop over multiple folders from list with glob.glob

How can I loop over a defined list of folders and all of the individual files inside each of those folders?
I'm trying to have it copy all the months in each year folder. But when I run it nothing happens..
import shutil
import glob
P4_destdir = ('Z:/Source P4')
yearlist = ['2014','2015','2016']
for year in yearlist:
for file in glob.glob(r'{0}/*.csv'.format(yearlist)):
print (file)
shutil.copy2(file,P4_destdir)

I think the problem might be that you require a / in you source path:
import shutil
import glob
P4_destdir = ('Z:/Source P4/')
yearlist = ['2014','2015','2016'] # assuming these files are in the same directory as your code.
for year in yearlist:
for file in glob.glob(r'{0}/*.csv'.format(yearlist)):
print (file)
shutil.copy2(file,P4_destdir)
Another thing that might be a problem is if the destination file does not yet exist. You can create it using os.mkdir:
import os
dest = os.path.isdir('Z:/Source P4/') # Tests if file exists
if not dest:
os.mkdir('Z:/Source P4/')