I want to copy files that have specific date. I can filter out the date. Copying makes problems.
import os
from os import walk
import time
from datetime import date, timedelta
import zipfile
import io
import shutil
src = 'K:\\Userfiles'
dest = 'L:\\Userfiles'
date1 = date.today() - timedelta(2)
for root, dirs, files in os.walk(src):
for file in files:
if ( 'zip' in file):
x = file[-18:-8]
d = date1.strftime('%Y-%m-%d')
if x == d:
shutil.copyfile(file, dest)
ERROR is: FileNotFoundError: [Errno 2] No such file or directory.
Traceback (most recent call last):
File "C:/Python37/datetime_finder.py", line 28, in shutil.copyfile(file, 'K:\Userfiles\Ucar\UNZIP')
File "C:\Python37\lib\shutil.py", line 120, in copyfile with open(src, 'rb') as fsrc: FileNotFoundError: [Errno 2] No such file or directory: 'getContents_2019-01-27.csv.zip
Taken from https://docs.python.org/3/library/shutil.html#shutil.copyfile
shutil.copyfile(src, dst, *, follow_symlinks=True)
Copy the contents (no metadata) of the file named src to a file named dst and return dst. src and dst are path names given as strings. dst must be the complete target file name; look at shutil.copy() for a copy that accepts a target directory path.
If I am not mistaken, you are missing setting dest value inside your inner for loop, so shutil.copyfile fails as '' (empty string) does not make sense as second argument. As side note if you want to copy only .zip files it is better to use:
if file.endswith('zip'):
instead of
if ('zip' in file):
which is also True for example for 'my_list_of_zip_files.txt', also keep in mind case-sensitiveness, so it might be better to use following if
if file.lower().endswith('zip'):
You are getting the error in this line shutil.copyfile(file, dest)
Mentioning the full path should fix the problem.
Ex:
shutil.copyfile(os.path.join(root, file), ".")
Related
I have two folders: In, Out - it is not system folder on disk D: - Windows 7. Out contain "myfile.txt" I run the following command in python:
>>> shutil.copyfile( r"d:\Out\myfile.txt", r"D:\In" )
Traceback (most recent call last):
File "<pyshell#39>", line 1, in <module>
shutil.copyfile( r"d:\Out\myfile.txt", r"D:\In" )
File "C:\Python27\lib\shutil.py", line 82, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: 'D:\\In'
What's the problem?
Read the docs:
shutil.copyfile(src, dst)
Copy the contents (no metadata) of the file named src to a file
named dst. dst must be the complete target file name; look at copy()
for a copy that accepts a target directory path.
use
shutil.copy instead of shutil.copyfile
example:
shutil.copy(PathOf_SourceFileName.extension,TargetFolderPath)
Use shutil.copy2 instead of shutil.copyfile
import shutil
shutil.copy2('/src/dir/file.ext','/dst/dir/newname.ext') # file copy to another file
shutil.copy2('/src/file.ext', '/dst/dir') # file copy to diff directory
I solved this problem, you should be the complete target file name for destination
destination = pathdirectory + filename.*
I use this code fir copy wav file with shutil :
# open file with QFileDialog
browse_file = QFileDialog.getOpenFileName(None, 'Open file', 'c:', "wav files (*.wav)")
# get file name
base = os.path.basename(browse_file[0])
os.path.splitext(base)
print(os.path.splitext(base)[1])
# make destination path with file name
destination= "test/" + os.path.splitext(base)[0] + os.path.splitext(base)[1]
shutil.copyfile(browse_file[0], destination)
First of all, make sure that your files aren't locked by Windows, some applications, like MS Office, locks the oppened files.
I got erro 13 when i was is trying to rename a long file list in a directory, but Python was trying to rename some folders that was at the same path of my files. So, if you are not using shutil library, check if it is a directory or file!
import os
path="abc.txt"
if os.path.isfile(path):
#do yor copy here
print("\nIt is a normal file")
Or
if os.path.isdir(path):
print("It is a directory!")
else:
#do yor copy here
print("It is a file!")
Visual Studio 2019
Solution : Administrator provided full Access to this folder "C:\ProgramData\Docker"
it is working.
ERROR: File IO error seen copying files to volume: edgehubdev. Errno: 13, Error Permission denied : [Errno 13] Permission denied: 'C:\ProgramData\Docker\volumes\edgehubdev\_data\edge-chain-ca.cert.pem'
[ERROR]: Failed to run 'iotedgehubdev start -d "C:\Users\radhe.sah\source\repos\testing\AzureIotEdgeApp1\config\deployment.windows-amd64.json" -v' with error: WARNING! Using --password via the CLI is insecure. Use --password-stdin.
ERROR: File IO error seen copying files to volume: edgehubdev. Errno: 13, Error Permission denied : [Errno 13] Permission denied: 'C:\ProgramData\Docker\volumes\edgehubdev\_data\edge-chain-ca.cert.pem'
use
> from shutil import copyfile
>
> copyfile(src, dst)
for src and dst use:
srcname = os.path.join(src, name)
dstname = os.path.join(dst, name)
This works for me:
import os
import shutil
import random
dir = r'E:/up/2000_img'
output_dir = r'E:/train_test_split/out_dir'
files = [file for file in os.listdir(dir) if os.path.isfile(os.path.join(dir, file))]
if len(files) < 200:
# for file in files:
# shutil.copyfile(os.path.join(dir, file), dst)
pass
else:
# Amount of random files you'd like to select
random_amount = 10
for x in range(random_amount):
if len(files) == 0:
break
else:
file = random.choice(files)
shutil.copyfile(os.path.join(dir, file), os.path.join(output_dir, file))
Make sure you aren't in (locked) any of the the files you're trying to use shutil.copy in.
This should assist in solving your problem
I avoid this error by doing this:
Import lib 'pdb' and insert 'pdb.set_trace()' before 'shutil.copyfile', it would just like this:
import pdb
...
print(dst)
pdb.set_trace()
shutil.copyfile(src,dst)
run the python file in a terminal, it will execute to the line 'pdb.set_trace()', and now the 'dst' file will print out.
copy the 'src' file by myself, and substitute and remove the 'dst' file which has been created by the above code.
Then input 'c' and click the 'Enter' key in the terminal to execute the following code.
well the questionis old, for new viewer of Python 3.6
use
shutil.copyfile( "D:\Out\myfile.txt", "D:\In" )
instead of
shutil.copyfile( r"d:\Out\myfile.txt", r"D:\In" )
r argument is passed for reading file not for copying
I know this question is asked and answered many places on this site but none of the resolutions work for me. Basically, I need shutil.copy to work the same way as shutil.move works (and it does) but when I do .copy I get the Error 13 Permissions error on the source file.
I can't do shutil.copytree because I need not the content of the first file folder to be copied but the file folder itself (with the contents inside). I've tried checking the permissions, modifying, nothing works.
I've also tried using the full path including the folder names (no variables) and I still get the same error.
This is in Windows 7 and 8.1
Here is the code:
import os
import csv
import re
from os import path
from distutils.dir_util import copy_tree
import shutil
# Open the csv file
f = open("Consumers.csv")
csv_f = csv.reader(f)
#Loop throught the csv file
for eachrow in csv_f:
#loop through the folders in the directory
for foldname in os.listdir():
#If the folder name is the same as a field in the first column of the csv file
if (foldname == eachrow[0]):
#Name the second column filed name "bucket."
#This is also the name of a folder in the same directory
bucket = eachrow[1]
#copy the first folder (and contents) into the second folder
shutil.copy (foldname, bucket)
And the error:
PermissionError Traceback (most recent call last)
<ipython-input-36-61e009418603> in <module>
25
26 #copy the first folder (and contents) into the second folder
---> 27 shutil.copy (foldname, bucket)
28
29
~\Anaconda3\lib\shutil.py in copy(src, dst, follow_symlinks)
243 if os.path.isdir(dst):
244 dst = os.path.join(dst, os.path.basename(src))
--> 245 copyfile(src, dst, follow_symlinks=follow_symlinks)
246 copymode(src, dst, follow_symlinks=follow_symlinks)
247 return dst
~\Anaconda3\lib\shutil.py in copyfile(src, dst, follow_symlinks)
118 os.symlink(os.readlink(src), dst)
119 else:
--> 120 with open(src, 'rb') as fsrc:
121 with open(dst, 'wb') as fdst:
122 copyfileobj(fsrc, fdst)
PermissionError: [Errno 13] Permission denied: 'Last1_First1_11111'
Any help would be greatly appreciated!
I've done something similar once. So maybe this helps, if I understand you correctly.
You can modify the behaviour of copytree via the ignore= option. ignore= takes a callable that in turn takes the currently (i.e. during the recursion) visited path and a list of that path's content as arguments. It has to return an iterable that copytree then ignores.
I've used the following imports (you can adjust that):
import shutil
import os
Take this as ignore function:
def ignore(path, content_list):
return [
content
for content in content_list
if os.path.isdir(os.path.join(path, content))
]
It packs all subdirectories of the visited path in the ignore list. The effect is that copytree stops its recursion after the first directory and the files in it!
Instead of
shutil.copy (foldname, bucket)
use
shutil.copytree(foldname, bucket, ignore=ignore)
It worked as designed for me. But I'm a bit unsure how your path structure exactly looks like. It would be good if foldname and bucket were absolute paths.
To ensure that I would add the following to the imports (the Path class makes working with paths extremely easy):
from pathlib import Path
Then supplement the loop through the folder with
#loop through the folders in the directory
for foldname in os.listdir():
fold_path = Path(foldname).resolve()
and use (instead of the version above)
shutil.copytree(fold_path, (fold_path.parent / bucket),
ignore=ignore,
dirs_exist_ok=True)
That's my understanding of what you are trying to achieve. (It's always good to illustrate questions with small examples to ensure that your intend is really clear.)
EDIT: Complete program (without comments and imports that aren't needed in this part):
import csv
import os
import shutil
from pathlib import Path
def ignore(path, content_list):
return [
content
for content in content_list
if os.path.isdir(os.path.join(path, content))
]
f = open("Consumers.csv")
csv_f = csv.reader(f)
for eachrow in csv_f:
for foldname in os.listdir():
fold_path = Path(foldname).resolve()
if foldname == eachrow[0]:
bucket = eachrow[1]
shutil.copytree(fold_path,
(fold_path.parent / bucket),
ignore=ignore,
dirs_exist_ok=True)
I am reading data from an excel spreadsheet(a list of textfiles) and using each row to retrieve the actual text file from a directory using Pycharm Community edition 2020 v1.3. I've set up a small loop to test whether the algorithm is working. When I try to print out the results it works perfectly, but when I attempt to move or copy the corresponding files to another directory I get the following output.
Traceback (most recent call last):
File "C:/Users/dalea/PycharmProjects/untitled/d1.py", line 46, in <module>
shutil.copy(file, MDA_For_Parsing) # Move it to MDA parsing directory
File "C:\Users\dalea\AppData\Local\Programs\Python\Python37\lib\shutil.py", line 248, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "C:\Users\dalea\AppData\Local\Programs\Python\Python37\lib\shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '19940804_10-K_edgar_data_354952_0000891618-94-000152_1.txt'
Process finished with exit code 1
The search seems to be working as required, it's the copy or move statement which is the problem. Can anyone explain why? My code is as follows.
import re
import glob
import os,sys
import csv
import shutil
import pandas as pd
import fnmatch
import string
import xlrd
from os import listdir
from os.path import isfile
MDA_Path = 'D:/1994_QTR3' # contains Loughram and MacDonald 10-K files
MDA_Target_List = r'D:/PhD_Data/Wenrui_Filing_list' # stores wenruis data
# MDA_For_Parsing = 'D:/Required_MDA_1994_QTR3' # will hold all 10-Ks from wenrui's spreadsheet once detected
MDA_For_Parsing = 'D:/Test'
# open the CSV file and extract the column containing the location of the text file(s)
datas = pd.read_excel(r'D:/PhD_Data/Wenrui_Filing_list/1994-2017filingslist_Wenrui_13Jul2020.xlsx')
df = pd.DataFrame(datas, columns = ['FILE_NAME']) # extract the data contained in FILE_NAME column
df['FILE_NAME'] = df['FILE_NAME'].str[26:] # remove the first 26 characters which contain the edgar drive info
df['FILE_NAME'] = df['FILE_NAME'].str.strip() # remove all leading and trailing
file_length = len(df) # count number of files in Wenrui's list (will need this later to loop through all occurrences)
dirs = os.listdir(MDA_Path)
for x in range(6): # if the L&M file exists in Wenrui's data set
for file in dirs:
# if file == df['FILE_NAME'][x]:
if df['FILE_NAME'][x] in file:
print(file)
shutil.copy(file, MDA_For_Parsing) # Move it to MDA parsing directory
Any help would be appreciated. Thanks in advance.
os.listdir(MDA_Path) only returns the base file names. The names do not include the folder.
Ti fix you issue, include the path when you do the copy:
shutil.copy(MDA_Path + '/' + file, MDA_For_Parsing)
I am trying to open the file from folder and read it but it's not locating it. I am using Python3
Here is my code:
import os
import glob
prefix_path = "C:/Users/mpotd/Documents/GitHub/Python-Sample-
codes/Mayur_Python_code/Question/wx_data/"
target_path = open('MissingPrcpData.txt', 'w')
file_array = [os.path.abspath(f) for f in os.listdir(prefix_path) if
f.endswith('.txt')]
file_array.sort() # file is sorted list
for f_obj in range(len(file_array)):
file = os.path.abspath(file_array[f_obj])
join_file = os.path.join(prefix_path, file) #whole file path
for filename in file_array:
log = open(filename, 'r')#<---- Error is here
Error: FileNotFoundError: [Errno 2] No such file or directory: 'USC00110072.txt'
You are not giving the full path to a file to the open(), just its name - a relative path.
Non-absolute paths specify locations in relation to current working directory (CWD, see os.getcwd).
You would have to either os.path.join() correct directory path to it, or os.chdir() to the directory that the files reside in.
Also, remember that os.path.abspath() can't deduce the full path to a file just by it's name. It will only prefix its input with the path of the current working directory, if the given path is relative.
Looks like you are forgetting to modify the the file_array list. To fix this, change the first loop to this:
file_array = [os.path.join(prefix_path, name) for name in file_array]
Let me reiterate.
This line in your code:
file_array = [os.path.abspath(f) for f in os.listdir(prefix_path) if f.endswith('.txt')]
is wrong. It will not give you a list with correct absolute paths. What you should've done is:
import os
import glob
prefix_path = ("C:/Users/mpotd/Documents/GitHub/Python-Sample-"
"codes/Mayur_Python_code/Question/wx_data/")
target_path = open('MissingPrcpData.txt', 'w')
file_array = [f for f in os.listdir(prefix_path) if f.endswith('.txt')]
file_array.sort() # file is sorted list
file_array = [os.path.join(prefix_path, name) for name in file_array]
for filename in file_array:
log = open(filename, 'r')
You are using relative path where you should be using an absolute one. It's a good idea to use os.path to work with file paths. Easy fix for your code is:
prefix = os.path.abspath(prefix_path)
file_list = [os.path.join(prefix, f) for f in os.listdir(prefix) if f.endswith('.txt')]
Note that there are some other issues with your code:
In python you can do for thing in things. You did for thing in range(len(things)) it's much less readable and unnecessary.
You should use context managers when you open a file. Read more here.
I'm trying to build a little renaming program to help save me time in the future.
Basically it will go through directories I point it too and rename files if they meet certain criteria.
I have written what I need but I have a bug in the very beginning that I can't figure out.
Here is the code:
import os
import fnmatch
for file in os.listdir("""/Users/Desktop/TESTME"""):
if fnmatch.fnmatch(file,'MISC*'):
os.rename(file, file[4:12] + '-13-Misc.jpg')
When I try to run it I am getting this:
Traceback (most recent call last):
File "/Users/Documents/Try.py", line 6, in <module>
os.rename(file, file[4:12] + '-13-Misc.jpg')
OSError: [Errno 2] No such file or directory
I also tried this:
if fnmatch.fnmatch(file,'MISC*'):
fun = file[4:12] + '-13-Misc.jpg'
os.rename(file, fun)
But I get the same thing.
It's not recognizing the file as a file.
Am I going about this the wrong way?
You'll need to include the full path to the filenames you are trying to rename:
import os
import fnmatch
directory = "/Users/Desktop/TESTME"
for file in os.listdir(directory):
if fnmatch.fnmatch(file, 'MISC*'):
path = os.path.join(directory, file)
target = os.path.join(directory, file[4:12] + '-13-Misc.jpg'
os.rename(path, target)
The os.path.join function intelligently joins path elements into a whole, using the correct directory separator for your platform.
The function os.listdir() only returns the file names of the files in the given directory, not their full paths. You can use os.path.join(directory, file_name) to reconstruct the full path of the file.
You could also do this in bash:
cd /Users/Desktop/TESTME/
for f in MISC*; do mv "$f" "${f:4:8}-13-Misc.jpg"; done