We Keep Coding

Generating instances of duplicate files

I'm trying to edit a script I've previously written to generate .lab files (or basically txt files with a .lab extension) as part of a project I'm currently working on. Specifically, what I'm currently working on is to edit the script such that it will be able to handle duplicate filenames. Example: if a file named filename already exists, instead of appending or overwriting the existing file, it would instead create a new file named filename_1.
The script is written as shown below:
for line in reader:
utterance = line[4]
path = line[1].split("/")
folder_name = path[len(path) - 2]
file_name = path[len(path) - 1].split(".")[0]
duplicate_num = 0
# Generate the .lab file
try:
if os.path.isfile(truncated_audio_dir + "/" + folder_name + "/" + file_name + "_cut.lab"):
new_file_name = file_name + "_" + str(duplicate_num)
while os.path.isfile(truncated_audio_dir + "/" + folder_name + "/" + new_file_name + "_cut.lab"):
duplicate_num += 1
new_file_name = file_name + "_" + str(duplicate_num)
print("New File Name: ", new_file_name)
outfile = open(truncated_audio_dir + "/" + folder_name + "/" + file_name + "_cut.lab", "a")
outfile.write(utterance)
outfile.close()
elif not os.path.isfile(truncated_audio_dir + "/" + folder_name + "/" + file_name + "_cut.lab"):
print("Is File")
outfile = open(truncated_audio_dir + "/" + folder_name + "/" + file_name + "_cut.lab", "a")
outfile.write(utterance)
outfile.close()
except FileNotFoundError:
print(truncated_audio_dir + "/" + folder_name + "/" + file_name + "_cut.lab" + " not found")
continue
The issue is that when I try and run the script, the issues I'd been experiencing beforehand still seem to persist. Particularly, the case I'd written to handle duplicates does not seem to trigger at all, instead the program keeps throwing FileNotFoundError exceptions (which I'd originally written to handle the case if there was a directory that didn't exist). I'm suspecting that the FileNotFoundError exception handling I'd originally written is causing the issue, but maybe there may be something else that I may not be aware of. Any help would be gladly appreciated.
(the above code is a majority of the script, but not the complete script; I imported sys, csv, and os and reader refers to a csv that I am reading from)

A simple python script to 'search text files for whole words' - with GUI

I am currently building a small program that allows searching for phrases in actors' dialog, using transcribed text files from video clips. I run into a few issues as described below...
Create user input:
# Get the SEARCH WINDOW
root = tk.Tk()
root.withdraw()
root.option_add('*background', '#111111')
root.option_add('*Entry*background', '#999999')
searchPhrase = sd.askstring(
"PhraseFinder v0.1 | filmwerk.nyc 2021 ", "Type keyword, or entire phrase, to search...", parent=root,)>
This seems to work fine. User input stored in searchPhrase...
Take user input from above (searchPhrase) and search a directory containing 800 text files ('whole word' search only - 'ignore case').
# Do THE SEARCH, based on user input
import glob
import os
rootDir = '/Volumes/audio/TRANSCRIBE/OUT'
os.chdir( rootDir )
for files in glob.glob( "*.txt" ):
with open(files) as f:
contents = f.read()
if (re.search(r'\b'+ re.escape(searchPhrase) + r'\b', contents, re.IGNORECASE)):
print( f )
This outputs:
<_io.TextIOWrapper name='FW_A01_2020-12-01_1856_C0004.txt' mode='r' encoding='US-ASCII'>
<_io.TextIOWrapper name='FW_A01_2020-12-01_1900_C0007.txt' mode='r' encoding='US-ASCII'>
The search result is correct, but the output format is not what I expected. So I need to rename stuff here. Unless there's a better way to get (print) the results? Currently, this gets output by print( f ).
The only thing I need from this output is to grab the actual file name:
FW_A01_2020-12-01_1856_C0004.txt and FW_A01_2020-12-01_1900_C0007.txt.
Then I need to rename & add the full path and finally store those search results files (clip list) in a continuous list, formatted like this:
> '/Volumes/RAID/Data/Media/TWO_CHAIRS/footage/FW_A01_2020-12-01_1806_C0001/FW_A01_2020-12-01_1806_C0001_000000.dng', '/Volumes/RAID/Data/Media/TWO_CHAIRS/footage/FW_A01_2020-12-01_1806_C0001/FW_A01_2020-12-01_1806_C0001_000000.dng',
Rename the 'search result' filenames (and add the full path), then store them in a variable. Since I don't know (yet) how to pipe in my actual search results into this function, I'll get the rootDir instead to perform the 'rename' as a test.
for currentFile in listofFiles:
listofFiles = listdir(rootDir)
for currentFile in listofFiles:
sourceFile = rootDir + "/" + currentFile
mainNameEnd = currentFile.find('.')
newFileName = currentFile[:mainNameEnd] + '_000000.dng'
dirLoc = currentFile[:mainNameEnd]
fullPathName = "'" + mediaDir + project.GetName() + "/" + "footage" + "/" + dirLoc + "/" + newFileName + "'" + "," + " "
print("Converting path name: " + fullPathName)
This outputs:
Converting path name: '/Volumes/RAID/Data/Media/TWO_CHAIRS/footage/FW_A01_2020-12-01_1806_C0001/FW_A01_2020-12-01_1806_C0001_000000.dng',
Converting path name: '/Volumes/RAID/Data/Media/TWO_CHAIRS/footage/FW_A01_2020-12-01_1812_C0003/FW_A01_2020-12-01_1812_C0003_000000.dng',
Converting path name: '/Volumes/RAID/Data/Media/TWO_CHAIRS/footage/FW_A01_2020-12-01_1856_C0004/FW_A01_2020-12-01_1856_C0004_000000.dng',
Great, exactly the output format I need. However, this only works with files found in rootDir. What I really need is to grab the 'search result' clip list and rename those files the same way. Also, the clip list needs to be a continuous line as shown earlier.
Once that's working I'll use the reformated clip list in the function below. This will then import the clips into an external app.
# Import clips from Search Result
# We insert the search_result_clip_list, separated by comma.
clips = resolve.GetMediaStorage().AddItemsToMediaPool(search_result_clip_list) # <-- clip list goes here
print(search_result_clip_list)
In a nutshell, I can't figure out how to take my search results, create a list, and finally use that list in the function above.
Would someone know how to implement this properly?
python 3.6.8 | MacOS 10.13.2 | Davinci Resolve 15

Real file name is in variable files and you should simply use
print(files)
In f you have file-object which reads data from file - not file name - and eventually you could use
print( f.name )
but I would prefer first version.
EDIT:
If you want to keep all filenames which match regex then you should use list.
Before loop create searchResult = [] and inside loop use searchResult.append( files )
searchResult = []
for files in glob.glob( "*.txt" ):
# ... code ...
if (re.search(r'\b'+ re.escape(searchPhrase) + r'\b', contents, re.IGNORECASE)):
print( files )
searchResult.append( files )

To get all the names in the same list:
You can use an empty list and add items to it in each loop like this:
my_names_list = []
for currentFile in listofFiles:
sourceFile = rootDir + "/" + currentFile
mainNameEnd = currentFile.find('.')
newFileName = currentFile[:mainNameEnd] + '_000000.dng'
dirLoc = currentFile[:mainNameEnd]
fullPathName = "'" + mediaDir + project.GetName() + "/" + "footage" + "/" + dirLoc + "/" + newFileName + "'" + "," + " "
print("Converting path name: " + fullPathName)
my_names_list.append(fullPathName)
You will get a list with all the names as its items.
Respect of this: However, this only works with files found in rootDir I don't really get what you want, try to be more specific.

Python: Check if files exist and copy only the missing files

I'm a begynder in python and trying to make a script that does the following:
Check number of files, if they exist in the destFile
If they all exist, exit the script (don't do anything)
If some files are missing, copy only the missing files from the srcFile to the destFile
The script that I have made is working, but the issue that I would like your help with is to make my script only copies the file/files missing and not as my script is doing now, which copies from file 1 (test1.txt) to the file missing. Example if test4.txt & test5.txt files are missing in destFile, my script will copy from test1.txt to test5.txt, in stead of only copying the two missing files test4.txt & test5.txt.
import os, shutil
from datetime import datetime
count = 0
error = "ERROR! file is missing! (files have been copied)"
sttime = datetime.now().strftime('%d/%m/%Y - %H:%M:%S - ')
os.chdir("C:\log")
log = "log.txt"
srcFile = [r"C:\srcFile\test1.txt",
r"C:\srcFile\test2.txt",
r"C:\srcFile\test3.txt",
r"C:\srcFile\test4.txt",
r"C:\srcFile\test5.txt"]
destFile = [r"C:\destFile\test1.txt",
r"C:\destFile\test2.txt",
r"C:\destFile\test3.txt",
r"C:\destFile\test4.txt",
r"C:\destFile\test5.txt"]
for file in destFile:
if not os.path.exists(file):
for file_sr in srcFile:
if not os.path.exists(file):
shutil.copy(file_sr, 'C:\destFile')
count +=1
with open(log, 'a') as logfile:
logfile.write(sttime + error + " " + str(count) + " => " + file + '\n')

The problem is that you're iterating over all of the source files whenever you detect a missing destination file: for file_sr in srcFile:. Instead, you can copy just the missing file by keeping track of the position (in the array) of the missing destination file:
for position, file in enumerate(destFile):
if not os.path.exists(file):
file_sr = srcFile[position]
if not os.path.exists(file):
shutil.copy(file_sr, 'C:\destFile')

Using your code, you can do:
import os, shutil
from datetime import datetime
count = 0
error = "ERROR! file is missing! (files have been copied)"
sttime = datetime.now().strftime('%d/%m/%Y - %H:%M:%S - ')
os.chdir("C:\log")
log = "log.txt"
srcFile = [r"C:\srcFile\test1.txt",
r"C:\srcFile\test2.txt",
r"C:\srcFile\test3.txt",
r"C:\srcFile\test4.txt",
r"C:\srcFile\test5.txt"]
destFile = [r"C:\destFile\test1.txt",
r"C:\destFile\test2.txt",
r"C:\destFile\test3.txt",
r"C:\destFile\test4.txt",
r"C:\destFile\test5.txt"]
for file in destFile:
if not os.path.exists(file):
src_file = destFile.replace("destFile","srcFile")
shutil.copy(src_file, file)
count +=1
with open(log, 'a') as logfile:
logfile.write(sttime + error + " " + str(count) + " => " + file + '\n')

Thank you for your help guys. Exactly my problem was that I was iterating over all of the source files whenever I detected a missing destination file. The following logic from mackorone is doing what I was looking for.
for position, file in enumerate(destFile):
if not os.path.exists(file):
file_sr = srcFile[position]
shutil.copy(file_sr, 'C:\destFile')

I have updated the script, so now this script compares two folders, source folder and destination folder. If destination folder is missing files from the source folder, it will be copied. The script is working fine.
import os
import shutil
from datetime import datetime
sttime = datetime.now().strftime('%d/%m/%Y - %H:%M:%S - ')
error = "ERROR! file is missing! (files have been copied)"
des_path = 'C:\des_folder'
sr_path = 'C:\sr_folder'
des_folder = os.listdir(des_path)
sr_folder = os.listdir(sr_path)
count = 0
os.chdir("C:\log")
log = "log.txt"
def compare_folder(folder1,folder2):
files_in_sr_folder = set(sr_folder) - set(des_folder)
return files_in_sr_folder
files_missing = compare_folder(sr_folder,des_folder)
if len(files_missing) != 0:
for file in files_missing:
full_path_files = os.path.join(sr_path,file)
shutil.copy(full_path_files,des_path)
count +=1
with open(log, 'a') as logfile:
logfile.write(sttime + error + " " + str(count) + " => " + file + '\n')
else:
exit

How to add characters to filename using glob in Python?

The following is a snippet of code. The script takes the input files from a "Test" folder, runs a function and then outputs the files with the same name in the "Results" folder (i.e. "Example_Layer.shp"). How could I set it so that the output file would instead read "Example_Layer(A).shp"?
#Set paths
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"
def run():
#Set definitions
input = path_res + "/" + "input.shp"
output = path_res + "/" + fname
#Set current path to path_dir and search for only .shp files then run function
os.chdir(path_dir)
for fname in glob.glob("*.shp"):
run_function, input, output
run()

You currently calculate the output variable once (which IMO wouldn't work since you don't have any fname defined yet).
Move the statement where you compute the output variable within the for loop, like below:
#Set paths
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"
def run():
#Set definitions
input = path_res + "/" + "input.shp"
#Set current path to path_dir and search for only .shp files then run function
os.chdir(path_dir)
for fname in glob.glob("*.shp"):
output = path_res + "/" + fname
run_function, input, output
run()

To answer your question:
How could I set it so that the output file would instead read "Example_Layer(A).shp"
You can use shutil.copy to copy the file to the new directory, adding a "(A)" to each file name using os.path.join to join the path and new filename:
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"
import os
import shutil
def run():
os.chdir(path_dir)
for fname in glob.glob("*.shp"):
name,ex = fname.rsplit(".",1) # split on "." to rejoin later adding a ("A")
# use shutil.copy to copy the file after adding ("A")
shutil.copy(fname,os.path.join(path_res,"{}{}{}".format(name,"(A)",ex)))
# to move and rename in one step
#shutil.move(fname,os.path.join(path_res,"{}{}{}".format(name,"(A)",ex)))

Python copy files to a new directory and rename if file name already exists

I've already read this thread but when I implement it into my code it only works for a few iterations.
I'm using python to iterate through a directory (lets call it move directory) to copy mainly pdf files (matching a unique ID) to another directory (base directory) to the matching folder (with the corresponding unique ID). I started using shutil.copy but if there are duplicates it overwrites the existing file.
I'd like to be able to search the corresponding folder to see if the file already exists, and iteratively name it if more than one occurs.
e.g.
copy file 1234.pdf to folder in base directory 1234.
if 1234.pdf exists to name it 1234_1.pdf,
if another pdf is copied as 1234.pdf then it would be 1234_2.pdf.
Here is my code:
import arcpy
import os
import re
import sys
import traceback
import collections
import shutil
movdir = r"C:\Scans"
basedir = r"C:\Links"
try:
#Walk through all files in the directory that contains the files to copy
for root, dirs, files in os.walk(movdir):
for filename in files:
#find the name location and name of files
path = os.path.join(root, filename)
print path
#file name and extension
ARN, extension = os.path.splitext(filename)
print ARN
#Location of the corresponding folder in the new directory
link = os.path.join(basedir,ARN)
# if the folder already exists in new directory
if os.path.exists(link):
#this is the file location in the new directory
file = os.path.join(basedir, ARN, ARN)
linkfn = os.path.join(basedir, ARN, filename)
if os.path.exists(linkfn):
i = 0
#if this file already exists in the folder
print "Path exists already"
while os.path.exists(file + "_" + str(i) + extension):
i+=1
print "Already 2x exists..."
print "Renaming"
shutil.copy(path, file + "_" + str(i) + extension)
else:
shutil.copy(path, link)
print ARN + " " + "Copied"
else:
print ARN + " " + "Not Found"

Sometimes it is just easier to start over... I apologize if there is any typo, I haven't had the time to test it thoroughly.
movdir = r"C:\Scans"
basedir = r"C:\Links"
# Walk through all files in the directory that contains the files to copy
for root, dirs, files in os.walk(movdir):
for filename in files:
# I use absolute path, case you want to move several dirs.
old_name = os.path.join( os.path.abspath(root), filename )
# Separate base from extension
base, extension = os.path.splitext(filename)
# Initial new name
new_name = os.path.join(basedir, base, filename)
# If folder basedir/base does not exist... You don't want to create it?
if not os.path.exists(os.path.join(basedir, base)):
print os.path.join(basedir,base), "not found"
continue # Next filename
elif not os.path.exists(new_name): # folder exists, file does not
shutil.copy(old_name, new_name)
else: # folder exists, file exists as well
ii = 1
while True:
new_name = os.path.join(basedir,base, base + "_" + str(ii) + extension)
if not os.path.exists(new_name):
shutil.copy(old_name, new_name)
print "Copied", old_name, "as", new_name
break
ii += 1

I always use the time-stamp - so its not possible, that the file exists already:
import os
import shutil
import datetime
now = str(datetime.datetime.now())[:19]
now = now.replace(":","_")
src_dir="C:\\Users\\Asus\\Desktop\\Versand Verwaltung\\Versand.xlsx"
dst_dir="C:\\Users\\Asus\\Desktop\\Versand Verwaltung\\Versand_"+str(now)+".xlsx"
shutil.copy(src_dir,dst_dir)

For me shutil.copy is the best:
import shutil
#make a copy of the invoice to work with
src="invoice.pdf"
dst="copied_invoice.pdf"
shutil.copy(src,dst)
You can change the path of the files as you want.

I would say you have an indentation problem, at least as you wrote it here:
while not os.path.exists(file + "_" + str(i) + extension):
i+=1
print "Already 2x exists..."
print "Renaming"
shutil.copy(path, file + "_" + str(i) + extension)
should be:
while os.path.exists(file + "_" + str(i) + extension):
i+=1
print "Already 2x exists..."
print "Renaming"
shutil.copy(path, file + "_" + str(i) + extension)
Check this out, please!

import os
import shutil
import glob
src = r"C:\Source"
dest = r"C:\Destination"
par = "*"
i=1
d = []
for file in glob.glob(os.path.join(src,par)):
f = str(file).split('\\')[-1]
for n in glob.glob(os.path.join(dest,par)):
d.append(str(n).split('\\')[-1])
if f not in d:
print("copied",f," to ",dest)
shutil.copy(file,dest)
else:
f1 = str(f).split(".")
f1 = f1[0]+"_"+str(i)+"."+f1[1]
while f1 in d:
f1 = str(f).split(".")
f1 = f1[0]+"_"+str(i)+"."+f1[1]
print("{} already exists in {}".format(f1,dest))
i =i + 1
shutil.copy(file,os.path.join(dest,f1))
print("renamed and copied ",f1 ,"to",dest)
i = 1