I'm trying to create a shell script that will copy files from one computer (employee's old computer) to another (employee's new computer). I have it to the point where I can copy files over, thanks to the lovely people here, but I'm running into a problem - if I'm going from, say, this directory that has 2 files:
C:\Users\specificuser\Documents\Test Folder
....to this directory...
C:\Users\specificuser\Desktop
...I see the files show up on the Desktop, but the folder those files were in (Test Folder) isn't created.
Here is the copy function I'm using:
#copy function
def dir_copy(srcpath, dstpath):
#if the destination path doesn't exist, create it
if not os.path.exists(dstpath):
os.makedir(dstpath)
#tag each file to the source path to create the file path
for file in os.listdir(srcpath):
srcfile = os.path.join(srcpath, file)
dstfile = os.path.join(dstpath, file)
#if the source file path is a directory, copy the directory
if os.path.isdir(srcfile):
dir_copy(srcfile, dstfile)
else: #if the source file path is just a file, copy the file
shutil.copyfile(srcfile, dstfile)
I know I need to create the directory on the destination, I'm just not quite sure how to do it.
Edit: I found that I had a type (os.makedir instead of os.mkdir). I tested it, and it creates directories like it's supposed to. HOWEVER I'd like it to create the directory one level up from where it's starting. For example, in Test Folder there is Sub Test Folder. It has created Sub Test Folder but won't create Test Folder because Test Folder is not part of the dstpath. Does that make sense?
You might want to look at shutil.copytree(). It performs the recursive copy functionality, including directories, that you're looking for. So, for a basic recursive copy, you could just run:
shutil.copytree(srcpath, dstpath)
However, to accomplish your goal of copying the source directory to the destination directory, creating the source directory inside of the destination directory in the process, you could use something like this:
import os
import shutil
def dir_copy(srcpath, dstdir):
dirname = os.path.basename(srcpath)
dstpath = os.path.join(dstdir, dirname)
shutil.copytree(srcpath, dstpath)
Note that your srcpath must not contain a slash at the end for this to work. Also, the result of joining the destination directory and the source directory name must not already exist, or copytree will fail.
This is a common problem with file copy... do you intend to just copy the contents of the folder or do you want the folder itself copied. Copy utilities typically have a flag for this and you can too. I use os.makedirs so that any intermediate directories are created also.
#copy function
def dir_copy(srcpath, dstpath, include_directory=False):
if include_directory:
dstpath = os.path.join(dstpath, os.path.basename(srcpath))
os.makedirs(dstpath, exist_ok=True)
#tag each file to the source path to create the file path
for file in os.listdir(srcpath):
srcfile = os.path.join(srcpath, file)
dstfile = os.path.join(dstpath, file)
#if the source file path is a directory, copy the directory
if os.path.isdir(srcfile):
dir_copy(srcfile, dstfile)
else: #if the source file path is just a file, copy the file
shutil.copyfile(srcfile, dstfile)
import shutil
import os
def dir_copy(srcpath, dstpath):
try:
shutil.copytree(srcpath, dstpath)
except shutil.Error as e:
print('Directory not copied. Error: %s' % e)
except OSError as e:
print('Directory not copied. Error: %s' % e)
dir_copy('/home/sergey/test1', '/home/sergey/test2')
I use this script to backup (copy) my working folder. It will skip large files, keep folder structure (hierarchy) and create destination folders if they don't exist.
import os
import shutil
for root, dirs, files in os.walk(the_folder_copy_from):
for name in files:
if os.path.getsize(os.path.join(root, name))<10*1024*1024:
target=os.path.join("backup", os.path.relpath(os.path.join(root, name),start=the_folder_copy_from))
print(target)
os.makedirs(os.path.dirname(target),exist_ok=True)
shutil.copy(src=os.path.join(root, name),dst=target)
print("Done")
Related
I have a python script that must check first if folder exist in giving path then if exist delete the existing one then copy the new one from the source. if does't exist just copy the new folder.
the problem is that in the source path if i have folder the system will check if the folder exist in the destination and delete and copy if exist.
what i want is to just check for one folder name "test" and then if does't exist copy the new folder if exist delete and copy.
code:
import os
import shutil
from os import path
import datetime
src = "I:/"
src2 = "I:/test"
dst = "C:/Users/LT GM/Desktop/test/"
dst2 = "C:/Users/LT GM/Documents/"
dst3 = "C:/Users/LT GM/Documents/Visual Studio 2017"
def main():
copy()
def copy():
#go through the src files and folders
for root,dirs,files in os.walk(src):
try:
for folder in dirs:
#return folder name
full_folder_name = os.path.join(src, folder)
print("folder : ",full_folder_name)
#check if folder exits
if os.path.exists(dst):
print("folder exist")
#delete folder
shutil.rmtree(dst)
print("the deleted folder is :{0}".format(dst))
#copy the folder as it is (folder with the files)
copieddst = shutil.copytree(src2,dst)
print("copy of the folder is done :{0}".format(copieddst))
else:
print("folder does Not Exist")
#copy the folder as it is (folder with the files)
copieddst = shutil.copytree(src2,dst)
print("copy of the folder is done :{0}".format(copieddst))
except Exception as e:
print(e)
try:
for name in files:
full_file_name = os.path.join(src, name)
print("files: ",full_file_name)
#check for pdf extension
if name.endswith("pdf"):
#copy files
shutil.copy(full_file_name, dst2)
#check for doc & docx extension
elif name.endswith("docx") or name.endswith("doc"):
#copy files
shutil.copy(full_file_name, dst3)
print("word files done")
print("pdf files done")
except Exception as e:
print(e)
if __name__=="__main__":
main()
why do you even check? Just rmtree the destination and ignore the error if it doesn't exist. You're not getting any significant saving by first checking, you're just making your code more complex.
why do you delete src for every folder you're copying? Just delete it once before the loop
also you should probably copy the sub-folders of src in sub-folders of dst rather than dump everything in src
os.walk will recursively walks through all the directories under the root (and thus their directories, and theirs, ...), that really doesn't seem to be what you want here,
your path management is weird as hell, why do you have two different sources and three destinations used completely inconsistently?
i want a python script that do these three tasks:
check if the path contains word file copied to specific destination
check if the path contains pdf files copied to specific destination
check if the path contains directory and copy the hole folder to
specific destination.
for this reason i am using the os.walk() to list the dirs and files of the path
and i am using shutil library to copy files and dirs.
code
import os
from distutils.dir_util import copy_tree
import shutil
from os import path
import datetime
def main():
src = "C:/Users/LT GM/Desktop/Python_files/"
dst2 = "C:/Users/LT GM/Desktop/"
for root,dirs,files in os.walk(src):
for name in files:
print("files: ",os.path.join(root,name))
for name in dirs:
copieddst = copy_tree(src,dst2)
print("directory: ",os.path.join(root,name))
print(" coppied directory :{0}".format(copieddst) )
# make a duplicate of an existing file
if path.exists(src):
# get the path to the file in the current directory
print("****")
src = path.realpath("pandas.pdf")
#seperate the path from the filter
head, tail = path.split(src)
print("path:" +head)
print("file:" +tail)
dst =str(datetime.date.today()) + tail
# nowuse the shell to make a copy of the file
shutil.copy(src, dst)
if __name__=="__main__":
main()
the problem is that i can copy the files or the content of the directory. not the hole directory and how to check if its pdf or doc files?
If you want to copy directory rather than file, then use shutil.copytree. In usage it is similiar to shutil.copy2, that is:
import shutil
shutil.copytree('mydir', 'mydircopy')
Note that by default dirs_exist_ok is False meaning that destination should not exist, when shutil.copytree is launched.
I built a script in Python to copy any files from a list of folders to a destination folder already made.
source = ['c:/test/source/', ]
destination = 'c:/test/destination/'
def copy(source, destination):
import os, shutil
try:
for folder in source:
files = os.listdir(folder)
for file in files:
current_file = os.path.join(folder, file)
shutil.copy(os.path.join(folder, file), destination)
except:
pass
The problem with this script is that it didn't copy the sub folders. Any suggestion to fix it ?
Thanks
I think you need to use shutil.copytree
shutil.copytree(os.path.join(folder, file), destination)
but shutil.copytree won't overwrite if folder exist,
if you want to overwrite all, use distutils.dir_util.copy_tree
from distutils import dir_util
dir_util.copy_tree(os.path.join(folder, file), destination)
I am copy folder and all sub folders inside the main folder using shutil copytree
import shutil
import sys
import os
import re
SOURCE_FOLDER = sys.argv[1]
DESTINATION_FOLDER = sys.argv[2]
def copyDirectory(SOURCE_FOLDER, DESTINATION_FOLDER):
try:
print SOURCE_FOLDER
print DESTINATION_FOLDER
shutil.copytree(SOURCE_FOLDER, DESTINATION_FOLDER)
# Directories are the same
#except:
# print "Not copied"
except shutil.Error as e:
print('Directory not copied. Error: %s' % e)
# Any error saying that the directory doesn't exist
except OSError as e:
print('Directory not copied. Error: %s' % e)
copyDirectory(SOURCE_FOLDER,DESTINATION_FOLDER)
The problem is if the directory exists it throws error
Directory not copied. Error: [Errno 17] File exists: 'destination'
What i want is if directory already exists it want to check all the sub directories and if sub directory also exists it should check all the files in it and it should skip the existing files and copy the new files in that sub directory,If sub direscotry not exists then it should copy that sub directory
Note: Sub directories might be nested(Sub directory of sub directory).
But the above script is not working what should i add to that script?
shutil.copytree isn't written to skip existing destination files and directories. From the docs
The destination directory must not already exist.
You will need to write your own solution. The existing copytree code is a good start.
In order to check if directory is already exists you can use: os.path.exists(directory)
if not os.path.exists(DESTINATION_FOLDER):
shutil.copytree(SOURCE_FOLDER, DESTINATION_FOLDER)
If the dest directory already exists you can get run your functions on the sub-directories of the src-dir.
You can get a list of all src-dir sub-directories using the following function which get directory name as input, and return a list of sub-directories
def SubDirPath (d):
return filter(os.path.isdir, [os.path.join(d,f) for f in os.listdir(d)])
using this list of directories you can execute your function again, on each instance of the directory.
For each directory which exists in both : src and dst - you'll need to check for every file in src-dir if the file also exists in the dst-dir.
Best Regards,
Yaron
With python3, you can use shutil.copytree with an option to ignore existing dirs error:
shutil.copytree(SOURCE_FOLDER, DESTINATION_FOLDER, dirs_exist_ok=True)
I need to update an existing directory with files that are provided in a Patch directory.
This is what I'm starting with. All commented out by me and then I try to build each line.
# $SourceDirectory = Patch folder that has files in any number of sub folders
# $DestDirectory = Application folder that has the files that need patching
# $UnMatchedFilesFolder = A Folder where SourceFiles go that don't have a match in $DestDirectory
# import os.path
# import os.listdir
#
# Create list1 of files from $SourceDirectory
# For each file (excluding directory names) in List1 (including subfolders), search for it in $DestDirectory and its subfolders;
# If you find the file by the same name, then create a backup of that file with .old;
# move $DestDirectoryPathAndFile to $DestDirectoryPathAndFile.old;
# print "Creating backup of file";
# After the backup is made, then copy the file from the $SourceDirectory to the;
# exact same location where it was found in the $DestDirectory. ;
# Else;
# move file to UnmatchedFilesDirectory.;
# If the number of files in $UnMatchedFilesDirectory =/ 0;
# Create list3 from $UnmatchedFilesDirectory
# print "The following files in $UnMatchedFilesDirectory will need to be installed individually";
# Print "Automated Patching completed.";
# Print "Script completed";
As mentioned in the previous post, I am skeptical of the course you are following based on the information given. Based on the document given, there are far better sites/tutorials available for free to help you learn Python/programming. That said, Stack Overflow is a friendly place, and so I hope to provide you with information which will help you on your way:
import os
source_dir =r"D:\temp"
dest_dir=r"D:\temp2"
for root, dirs, files in os.walk(source_dir):
# os.walk 'root' steps through subdirectories as we iterate
# this allows us to join 'root' and 'file' without missing any sub-directories
for file in files:
exist_path = os.path.join(root, file)
# expected_file represents the fullpath of a file we are looking to create/replace
expected_file = exist_path.replace(source_dir, dest_dir)
current = os.path.join(root, file)
if os.path.exists(expected_file):
print "The file %s exists, os.rename with '.old' before copying %s" % (current, exist_path)
# .. note:: we should rename to .bkp here, then we would correctly copy the file below without conflict
print "Now %s doesn't exist, we are free to copy %s" % (expected_file, exist_path)