I am trying to copy a source code tree using the below code and running into an error,am not sure why I am getting this?error says \\Ref\builds/out exists but it doesnt exist,"out" is the directory the source location that the script is trying to copy to destination,any other ways to do copy if shutil is not suited for this type of copy?
//local/mnt/workspace/04.01_HY11/out
\\Ref\builds/out
copying
Traceback (most recent call last):
File "test.py", line 21, in <module>
main()
File "test.py", line 18, in main
copytree(src,dst)
File "test.py", line 11, in copytree
shutil.copytree(s, d)
File "/pkg/qct/software/python/2.5.2/.amd64_linux26/lib/python2.5/shutil.py", line 110, in copytree
os.makedirs(dst)
File "/pkg/qct/software/python/2.5.2/.amd64_linux26/lib/python2.5/os.py", line 171, in makedirs
mkdir(name, mode)
OSError: [Errno 17] File exists: '\\\\Ref\\builds/out'
Python code
import os,shutil
def copytree(src, dst, symlinks=False, ignore=None):
for item in os.listdir(src):
s = os.path.join(src, item)
print s
d = os.path.join(dst, item)
print d
if os.path.isdir(s):
print "copying"
shutil.copytree(s, d, symlinks, ignore)
else:
shutil.copy2(s, d)
def main ():
src="//local/mnt/workspace/04.01_HY11"
dst="\\\\Ref\\builds"
copytree(src,dst)
if __name__ == '__main__':
main()
The documentation for shutil.copytree clearly says:
The destination directory, named by dst, must not already exist; it will be created as well as missing parent directories.
But \\ref\builds\out already exists—you can see from the exception's stack trace that it's trying to mkdir that path, but that's failing with an error indicating that that path already exists (which can happen when the path exists as either a regular file or a directory).
You need to copy to path that doesn't already exist, by either choosing a different path, or by deleting the existing tree at that location first. The latter can be done with shutil.rmtree.
Related
I'm trying to write a code that moves files in my download folder to other specified folders but I keep getting errors. Here's my code.
import os
import shutil
series = []
for i in os.listdir('C:\\Users\\Mike\\Downloads\\Video'):
if ('.mp4') in i:
series.append(i)
for j in series:
if 'Thrones' in j:
shutil.move(j,'C:\\Users\\Mike\\Desktop\\')
I keep getting this error
Traceback (most recent call last):
File "C:/Users/Mike/Downloads/Video/Arrange.py", line 70, in <module>
Series(series)
File "C:/Users/Mike/Downloads/Video/Arrange.py", line 48, in Series
shutil.move(serie, 'C:\\Users\\Mike\\Desktop\\Movies\\Series\\Lost\\s2\\')
File "C:\Users\Mike\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 536, in move
raise Error("Destination path '%s' already exists" % real_dst)
shutil.Error: Destination path 'C:\Users\Mike\Desktop\Movies\Series\Lost\s2\lost - s02e08 (o2tvseries.com).mp4' already exists
>>>
but the file actually moves. How do I move the files without getting this error each time?
The error you get is Windows platform specific. You are using shutil.move which uses os.rename under the hood. From docs:
On Windows, if dst already exists, OSError will be raised even if it is a file
You could check if the file exists in the destination before you move it and depending what you want to achieve:
1) don't overwrite the destination, just remove file from source
2) remove the file from source first and overwrite the destination
Below you can find implementation of solution 2)
import os
for name in series:
if 'Thrones' in name:
if not os.path.isfile(name):
shutil.move(name, 'C:\\Users\\Mike\\Desktop\\')
else:
os.remove(name)
I am trying to use shutil.copytree to copy a directory onto multiple other directories. I can't get it to work. I'm pretty sure I just need to implement ignore_errors=True, but I cant get it to work. How should I go about implementing 'ignore_errors=True' into
for CopyHere in DeleteThis:
for CopyThis in FilestoCopy:
shutil.copytree(CopyThis, CopyHere)
print('Files have been copied')
My code is as follows:
import shutil
import time
DeleteThis = ['E:', 'F:']
FilestoCopy = ['C:\\Users\\2402neha\\Desktop\\Hehe']
for Directory_to_delete in DeleteThis:
shutil.rmtree(Directory_to_delete, ignore_errors=True)
print('Directories have been wiped')
time.sleep(2)
for CopyHere in DeleteThis:
for CopyThis in FilestoCopy:
shutil.copytree(CopyThis, CopyHere)
print('Files have been copied')
Here are the error messages I get:
Traceback (most recent call last):
File "C:\Users\2402neha\OneDrive\Python\Dis Cleaner\Copy paste test.py", line 17, in <module>
shutil.copytree(CopyThis, CopyHere)
File "C:\Users\2402neha\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 309, in copytree
os.makedirs(dst)
File "C:\Users\2402neha\AppData\Local\Programs\Python\Python35\lib\os.py", line 241, in makedirs
mkdir(name, mode)
PermissionError: [WinError 5] Ingen tilgang: 'E:'
Your destination is E:
The destination directory needs to not exist.
From the documentation for shutil.copytree:
shutil.copytree(src, dst, symlinks=False, ignore=None)
Recursively copy an entire directory tree rooted at src. The destination directory, named by dst, must not already exist; it will be created as well as missing parent directories.
You probably want the directory name you're copying and to join it with the destination:
directory = os.path.basename(CopyThis)
destination = os.path.join(CopyHere, directory)
shutil.copytree(CopyThis, destination)
I'm trying to code a little script that watches a defined directory with a while-loop. Every file or directory that is in this directory is compressed to RAR and moved to another directory after the process is completed.
My problem: everytime I copy a file or folder to this directory, the script doesn't wait and startes the process the second it sees a new file or folder. But when the files or folders are bigger than a few kilobytes the loop breaks with a permission error.
Since I'm a Python beginner I don't know which module to use. Is there a checking module to see if the file or folder that the tool wants to process is used by another process? Or am I going in the wrong direction?
Edit: added the code for directory-only listening:
watchDir = "L:\\PythonTest\\testfolder\\"
finishedDir = "L:\\PythonTest\\finishedfolders\\"
rarfilesDir = "L:\\PythonTest\\rarfiles\\"
rarExe = "L:\\PythonTest\\rar.exe"
rarExtension = ".rar"
rarCommand = "a"
while True:
dirList = [name for name in os.listdir(watchDir) if os.path.isdir(os.path.join(watchDir,name))]
for entryName in dirList:
if not os.path.exists((os.path.join(finishedDir,entryName))):
sourcePath = os.path.join(watchDir,entryName)
entryNameStripped = entryName.replace(" ", "")
os.chdir(watchDir)
archiveName = rarfilesDir+entryNameStripped+rarExtension
subprocesscall = [rarExe, rarCommand, archiveName, entryName]
subprocess.call(subprocesscall, shell=True)
shutil.move(sourcePath,finishedDir)
When I run the script and try to add a file of several GB (named #filename# in the following lines) these errors occur:
Creating archive L:\PythonTest\rarfiles\#filename#.rar
Cannot open #filename#
The process cannot access the file, since it's used by another process.
Adding #filename# OK
WARNING: Cannot open 1 file
Done
Traceback (most recent call last):
File "C:\Python34\lib\shutil.py", line 522, in move
os.rename(src, real_dst)
PermissionError: [WinError 5] Access denied: #filepath#
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "L:/Python Test/test.py", line 35, in <module>
shutil.move(sourcePath,finishedDir)
File "C:\Python34\lib\shutil.py", line 531, in move
copytree(src, real_dst, symlinks=True)
File "C:\Python34\lib\shutil.py", line 342, in copytree
raise Error(errors)
shutil.Error: #filepath#
instead of using os.listdir, you can use os.walk, os.walk yields 3 tuple dirpath(path of directory,filenames(all files in that dirpath),dirnames(all the sub directories in dirpath)
for x,y,z in os.walk('path-of-directory'):
do you stuff with x,y,z the three tuples
I've used the following code to remove a tree on a USB device however I'm receiving an OSError:
I also ran the code with sudo python.
import shutil
import os
src = "/media/device/my_folder"
if os.path.exists(dst):
shutil.rmtree(dst)
I've just used shutil.copytree(src, dst) in another script to write the files to the device in the first place. However the USB device was removed during the copy, this is probably causing the issue I'm having as all other files except the one that was half copied have been removed okay.
I'm getting the following traceback:
Traceback (most recent call last):
File "writetousb/tests/deleteTest.py", line 32, in <module>
shutil.rmtree(src)
File "/usr/lib/python2.7/shutil.py", line 252, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "/usr/lib/python2.7/shutil.py", line 250, in rmtree
os.remove(fullname)
OSError: [Errno 30] Read-only file system: '/media/device/21823/21916.jpg'
So I'm guessing I'll need to change the permissions of the folder and it's files before I remove them?
If I use chmod to set the permissions correctly before I try to use shutil.rmtree then it should work. I'm going to test this and provide an update when I know it works.
I can confirm the solution works.
import shutil
import os
src = "/media/device/my_folder"
if os.path.exists(dst):
os.chmod(dst, 0o777)
for root,dirs,_ in os.walk(dst):
for d in dirs :
os.chmod(os.path.join(root,d) , 0o777)
shutil.rmtree(dst)
I'm having trouble figuring out how to move all .log and .txt files in a certain folder and it's subdirectories to a new folder. I understand how to move one file with shutil. But, I tried to use a loop, unsuccessfully, to move all. Can someone help me with this? Thanks ....
import os, os.path
import re
def print_tgzLogs (arg, dir, files):
for file in files:
path = os.path.join (dir, file)
path = os.path.normcase (path)
defaultFolder = "Log_Text_Files"
if not defaultFolder.endswith(':') and not os.path.exists('c:\\Extracted\Log_Text_Files'):
os.mkdir('C:\\Extracted\\Log_Text_Files')
if re.search(r".*\.txt$", path) or re.search(r".*\.log$", path):
os.rename(path, 'C:\\Extracted\\Log_Text_Files')
print path
os.path.walk('C:\\Extracted\\storage', print_tgzLogs, 0)
Below is the trace back error:
Traceback (most recent call last):
File "C:\SQA_log\scan.py", line 20, in <module>
os.path.walk('C:\\Extracted\\storage', print_tgzLogs, 0)
File "C:\Python27\lib\ntpath.py", line 263, in walk
walk(name, func, arg)
File "C:\Python27\lib\ntpath.py", line 263, in walk
walk(name, func, arg)
File "C:\Python27\lib\ntpath.py", line 263, in walk
walk(name, func, arg)
File "C:\Python27\lib\ntpath.py", line 259, in walk
func(arg, top, names)
File "C:\SQA_log\scan.py", line 16, in print_tgzLogs
os.rename(path, 'C:\\Extracted\\Log_Text_Files')
WindowsError: [Error 183] Cannot create a file when that file already exists
According to the traceback, the log-files are already existing. The Python docs to the os.rename say:
On Windows, if dst already exists, OSError will be raised [...].
Now you can either:
delete the files manually or
delete the files automatically using os.remove(path)
If you want the files to be automatically deleted, the code would look like this (notice that I replaced your regular expression with the python endswith as suggested by utdemir):
import os, os.path
def print_tgzLogs (arg, dir, files):
for file in files:
path = os.path.join (dir, file)
path = os.path.normcase (path)
defaultFolder = "Log_Text_Files"
if not defaultFolder.endswith(':') and not os.path.exists('c:\\Extracted\Log_Text_Files'):
os.mkdir('C:\\Extracted\\Log_Text_Files')
if path.endswith(".txt") or path.endswith(".log"):
if os.path.exists('C:\\Extracted\\Log_Text_Files\\%s' % file):
os.remove('C:\\Extracted\\Log_Text_Files\\%s' % file)
os.rename(path, 'C:\\Extracted\\Log_Text_Files\\%s' % file)
print path
os.path.walk('C:\\Extracted\\storage', print_tgzLogs, 0)
It looks like are trying to use
os.rename(path, 'C:\\Extracted\\Log_Text_Files')
to move the file path into the directory C:\Extracted\Log_Text_Files, but rename doesn't work like this: it's going to try to make a new file named C:\Extracted\Log_Text_Files. You probably want something more like this:
os.rename(path, os.path.join('C:\\Extracted\\Log_Text_Files',os.path.basename(path))