We Keep Coding

Python shutil.copy Permission Error Windows

I know this question is asked and answered many places on this site but none of the resolutions work for me. Basically, I need shutil.copy to work the same way as shutil.move works (and it does) but when I do .copy I get the Error 13 Permissions error on the source file.
I can't do shutil.copytree because I need not the content of the first file folder to be copied but the file folder itself (with the contents inside). I've tried checking the permissions, modifying, nothing works.
I've also tried using the full path including the folder names (no variables) and I still get the same error.
This is in Windows 7 and 8.1
Here is the code:
import os
import csv
import re
from os import path
from distutils.dir_util import copy_tree
import shutil
# Open the csv file
f = open("Consumers.csv")
csv_f = csv.reader(f)
#Loop throught the csv file
for eachrow in csv_f:
#loop through the folders in the directory
for foldname in os.listdir():
#If the folder name is the same as a field in the first column of the csv file
if (foldname == eachrow[0]):
#Name the second column filed name "bucket."
#This is also the name of a folder in the same directory
bucket = eachrow[1]
#copy the first folder (and contents) into the second folder
shutil.copy (foldname, bucket)
And the error:
PermissionError Traceback (most recent call last)
<ipython-input-36-61e009418603> in <module>
25
26 #copy the first folder (and contents) into the second folder
---> 27 shutil.copy (foldname, bucket)
28
29
~\Anaconda3\lib\shutil.py in copy(src, dst, follow_symlinks)
243 if os.path.isdir(dst):
244 dst = os.path.join(dst, os.path.basename(src))
--> 245 copyfile(src, dst, follow_symlinks=follow_symlinks)
246 copymode(src, dst, follow_symlinks=follow_symlinks)
247 return dst
~\Anaconda3\lib\shutil.py in copyfile(src, dst, follow_symlinks)
118 os.symlink(os.readlink(src), dst)
119 else:
--> 120 with open(src, 'rb') as fsrc:
121 with open(dst, 'wb') as fdst:
122 copyfileobj(fsrc, fdst)
PermissionError: [Errno 13] Permission denied: 'Last1_First1_11111'
Any help would be greatly appreciated!

I've done something similar once. So maybe this helps, if I understand you correctly.
You can modify the behaviour of copytree via the ignore= option. ignore= takes a callable that in turn takes the currently (i.e. during the recursion) visited path and a list of that path's content as arguments. It has to return an iterable that copytree then ignores.
I've used the following imports (you can adjust that):
import shutil
import os
Take this as ignore function:
def ignore(path, content_list):
return [
content
for content in content_list
if os.path.isdir(os.path.join(path, content))
]
It packs all subdirectories of the visited path in the ignore list. The effect is that copytree stops its recursion after the first directory and the files in it!
Instead of
shutil.copy (foldname, bucket)
use
shutil.copytree(foldname, bucket, ignore=ignore)
It worked as designed for me. But I'm a bit unsure how your path structure exactly looks like. It would be good if foldname and bucket were absolute paths.
To ensure that I would add the following to the imports (the Path class makes working with paths extremely easy):
from pathlib import Path
Then supplement the loop through the folder with
#loop through the folders in the directory
for foldname in os.listdir():
fold_path = Path(foldname).resolve()
and use (instead of the version above)
shutil.copytree(fold_path, (fold_path.parent / bucket),
ignore=ignore,
dirs_exist_ok=True)
That's my understanding of what you are trying to achieve. (It's always good to illustrate questions with small examples to ensure that your intend is really clear.)
EDIT: Complete program (without comments and imports that aren't needed in this part):
import csv
import os
import shutil
from pathlib import Path
def ignore(path, content_list):
return [
content
for content in content_list
if os.path.isdir(os.path.join(path, content))
]
f = open("Consumers.csv")
csv_f = csv.reader(f)
for eachrow in csv_f:
for foldname in os.listdir():
fold_path = Path(foldname).resolve()
if foldname == eachrow[0]:
bucket = eachrow[1]
shutil.copytree(fold_path,
(fold_path.parent / bucket),
ignore=ignore,
dirs_exist_ok=True)

FileNotFoundError: [Errno 2] No such file or directory: When trying to move or copy file to another directory

I am reading data from an excel spreadsheet(a list of textfiles) and using each row to retrieve the actual text file from a directory using Pycharm Community edition 2020 v1.3. I've set up a small loop to test whether the algorithm is working. When I try to print out the results it works perfectly, but when I attempt to move or copy the corresponding files to another directory I get the following output.
Traceback (most recent call last):
File "C:/Users/dalea/PycharmProjects/untitled/d1.py", line 46, in <module>
shutil.copy(file, MDA_For_Parsing) # Move it to MDA parsing directory
File "C:\Users\dalea\AppData\Local\Programs\Python\Python37\lib\shutil.py", line 248, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "C:\Users\dalea\AppData\Local\Programs\Python\Python37\lib\shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '19940804_10-K_edgar_data_354952_0000891618-94-000152_1.txt'
Process finished with exit code 1
The search seems to be working as required, it's the copy or move statement which is the problem. Can anyone explain why? My code is as follows.
import re
import glob
import os,sys
import csv
import shutil
import pandas as pd
import fnmatch
import string
import xlrd
from os import listdir
from os.path import isfile
MDA_Path = 'D:/1994_QTR3' # contains Loughram and MacDonald 10-K files
MDA_Target_List = r'D:/PhD_Data/Wenrui_Filing_list' # stores wenruis data
# MDA_For_Parsing = 'D:/Required_MDA_1994_QTR3' # will hold all 10-Ks from wenrui's spreadsheet once detected
MDA_For_Parsing = 'D:/Test'
# open the CSV file and extract the column containing the location of the text file(s)
datas = pd.read_excel(r'D:/PhD_Data/Wenrui_Filing_list/1994-2017filingslist_Wenrui_13Jul2020.xlsx')
df = pd.DataFrame(datas, columns = ['FILE_NAME']) # extract the data contained in FILE_NAME column
df['FILE_NAME'] = df['FILE_NAME'].str[26:] # remove the first 26 characters which contain the edgar drive info
df['FILE_NAME'] = df['FILE_NAME'].str.strip() # remove all leading and trailing
file_length = len(df) # count number of files in Wenrui's list (will need this later to loop through all occurrences)
dirs = os.listdir(MDA_Path)
for x in range(6): # if the L&M file exists in Wenrui's data set
for file in dirs:
# if file == df['FILE_NAME'][x]:
if df['FILE_NAME'][x] in file:
print(file)
shutil.copy(file, MDA_For_Parsing) # Move it to MDA parsing directory
Any help would be appreciated. Thanks in advance.

os.listdir(MDA_Path) only returns the base file names. The names do not include the folder.
Ti fix you issue, include the path when you do the copy:
shutil.copy(MDA_Path + '/' + file, MDA_For_Parsing)

How to fix 'NotADirectoryError' in Python on Mac

I'm trying to walk through all the files on my computer and create a list of files larger than 100MB. However, when walking through the files, I keep getting a 'NotADirectoryError.' The way I understand my code, it should be looking base files in the second for loop, so I don't know why the 'NotADirectory' Error keeps getting raised.
I'm on a Mac using Python3.7.4. I've tried adding an exception for 'NotADirectoryError', but it just gave me a 'FileNotFound Error,' so I figured adding exception upon exception wasn't the best way to solve this.
I think I'm misunderstanding how either os.walk or os.path.getsize work, but after reviewing the documentation for both I'm still confused about what I'm doing wrong.
big_dict= {}
folder = os.path.abspath('/Users/')
#walking thru all the folders on my computer
for folders, subfolders, filenames in os.walk(folder):
#for each file that's bigger than 100 MB, add it to the 'big_dict'
for filename in filenames:
filename = os.path.join(folder, filename) +'/'
file_size = os.path.getsize(filename) -- this line always gives the error
#convert size to MB, add to big_dict as value with filename key
if file_size > 1000000:
big_dict[filename] = str(file_size/10000) + ' MB'
Traceback (most recent call last):
File "/Users/jonbauer/size_check.py", line 17, in <module>
file_size = os.path.getsize(filename)
File "/Applications/Thonny.app/Contents/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/../../../../../../../Python.framework/Versions/3.7/lib/python3.7/genericpath.py", line 50, in getsize
return os.stat(filename).st_size
NotADirectoryError: [Errno 20] Not a directory: '/Users/.localized/'
This is the error message I get when running the code. As I said before, I'm trying to walk through all my files and add larger files to a list, but I keep encountering this error.

big_dict= {}
folder = os.path.abspath('/Users')
for root, _, filenames in os.walk(folder):
for filename in filenames:
path = os.path.join(root, filename)
if os.path.islink(path):
continue
file_size_mb = os.path.getsize(path) >> 20
if file_size_mb > 100:
file_size_mb_str = '{} MB'.format(file_size_mb)
print(path, file_size_mb_str)
big_dict[path] = file_size_mb_str
There are several problems in your code:
filename = os.path.join(folder, filename) +'/' is incorrect, because folder is the base folder ('/Users/' in your case), not the actual folder you're walking.
You need to skip links, because getsize() won't work on them
file_size > 1000000: os.path.getsize() returns bytes

Copy Files in directory with specific date

I want to copy files that have specific date. I can filter out the date. Copying makes problems.
import os
from os import walk
import time
from datetime import date, timedelta
import zipfile
import io
import shutil
src = 'K:\\Userfiles'
dest = 'L:\\Userfiles'
date1 = date.today() - timedelta(2)
for root, dirs, files in os.walk(src):
for file in files:
if ( 'zip' in file):
x = file[-18:-8]
d = date1.strftime('%Y-%m-%d')
if x == d:
shutil.copyfile(file, dest)
ERROR is: FileNotFoundError: [Errno 2] No such file or directory.
Traceback (most recent call last):
File "C:/Python37/datetime_finder.py", line 28, in shutil.copyfile(file, 'K:\Userfiles\Ucar\UNZIP')
File "C:\Python37\lib\shutil.py", line 120, in copyfile with open(src, 'rb') as fsrc: FileNotFoundError: [Errno 2] No such file or directory: 'getContents_2019-01-27.csv.zip

Taken from https://docs.python.org/3/library/shutil.html#shutil.copyfile
shutil.copyfile(src, dst, *, follow_symlinks=True)
Copy the contents (no metadata) of the file named src to a file named dst and return dst. src and dst are path names given as strings. dst must be the complete target file name; look at shutil.copy() for a copy that accepts a target directory path.

If I am not mistaken, you are missing setting dest value inside your inner for loop, so shutil.copyfile fails as '' (empty string) does not make sense as second argument. As side note if you want to copy only .zip files it is better to use:
if file.endswith('zip'):
instead of
if ('zip' in file):
which is also True for example for 'my_list_of_zip_files.txt', also keep in mind case-sensitiveness, so it might be better to use following if
if file.lower().endswith('zip'):

You are getting the error in this line shutil.copyfile(file, dest)
Mentioning the full path should fix the problem.
Ex:
shutil.copyfile(os.path.join(root, file), ".")

IOError: [Errno 13] Permission denied: Bulk renaming script

This is one of my first python scripts designed to go into a directory and rename all the files of a certain extension to the same thing. For instance, go into a music directory and change the name of all the album artwork files to something like folder.jpg so that they're easily recognized and displayed by music playing software like foobar2000.
I'm currently getting the error in the title. I've tried:
os.chmod(pathname, 0777)
and
os.chmod(pathname, stat.S_IWOTH)
to allow other entities to edit the directory I'm looking at, but the error remains. I'm not too familiar with Windows's permissions system or with Python so this is mystifying me. Any idea where I'm going wrong?
#Changes file names in every subdirectory of the target directory
import os
import sys
import shutil
#Get target path
pathname = raw_input('Enter path for music directory (ex. C:\\Music): ')
try:
os.chdir(pathname)
except:
print('Failed. Invalid directory?')
sys.exit()
#Get variables for renaming
fn = raw_input('Enter desired file name for all converted files: ')
ft = raw_input('Enter the file extension you want the program to look for (ex. .jpg): ')
outname = fn + ft
#Create path tree
for path, subdirs, files in os.walk (pathname):
#Search tree for files with defined extention
for name in files:
if name.lower().endswith(ft):
#Rename the files
src = os.path.join(path, name)
print (src)
dst = os.path.join(path, outname)
print dst
shutil.move(src, dst)
print('Complete')
A second more benign issue is that when I use a print statement to check on what files are being edited, it seems that before the first .jpg is attempted, the program processes and renamed a file called d:\test\folder.jpg which doesn't exist. This seems to succeed and the program fails in the last loop the second time through, when an existing file is processed. The program runs like this:
>>> ================================ RESTART ================================
>>>
Enter path for music directory (ex. C:\Music): d:\test
Enter desired file name for all converted files: folder
Enter the file extension you want the program to look for (ex. .jpg): .jpg
d:\test\folder.jpg
d:\test\folder.jpg
d:\test\Anathema\Alternative 4\AA.jpg
d:\test\Anathema\Alternative 4\folder.jpg
Traceback (most recent call last):
File "D:\Documents\Programming\Python scripts\Actually useful\bulk_filename_changer.py", line 29, in <module>
shutil.move(src, dst)
File "C:\Python27\lib\shutil.py", line 301, in move
copy2(src, real_dst)
File "C:\Python27\lib\shutil.py", line 130, in copy2
copyfile(src, dst)
File "C:\Python27\lib\shutil.py", line 83, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: 'd:\\test\\Anathema\\Alternative 4\\folder.jpg'
>>>
The glitch seems to be benign because it doesn't cause an error, but it might be related to an error I have in the code that leads to the IOError I get when the first "real" file is processed. Beats me...

I am assuming that d: is not a CD drive or something not writable. If so then have you checked the properties of your files? Are any of them read only? There would also need to be more than one of the file type in the folder to cause this, I believe.