We Keep Coding

how to copy files and directory from source to destination using python

i want a python script that do these three tasks:
check if the path contains word file copied to specific destination
check if the path contains pdf files copied to specific destination
check if the path contains directory and copy the hole folder to
specific destination.
for this reason i am using the os.walk() to list the dirs and files of the path
and i am using shutil library to copy files and dirs.
code
import os
from distutils.dir_util import copy_tree
import shutil
from os import path
import datetime
def main():
src = "C:/Users/LT GM/Desktop/Python_files/"
dst2 = "C:/Users/LT GM/Desktop/"
for root,dirs,files in os.walk(src):
for name in files:
print("files: ",os.path.join(root,name))
for name in dirs:
copieddst = copy_tree(src,dst2)
print("directory: ",os.path.join(root,name))
print(" coppied directory :{0}".format(copieddst) )
# make a duplicate of an existing file
if path.exists(src):
# get the path to the file in the current directory
print("****")
src = path.realpath("pandas.pdf")
#seperate the path from the filter
head, tail = path.split(src)
print("path:" +head)
print("file:" +tail)
dst =str(datetime.date.today()) + tail
# nowuse the shell to make a copy of the file
shutil.copy(src, dst)
if __name__=="__main__":
main()
the problem is that i can copy the files or the content of the directory. not the hole directory and how to check if its pdf or doc files?

If you want to copy directory rather than file, then use shutil.copytree. In usage it is similiar to shutil.copy2, that is:
import shutil
shutil.copytree('mydir', 'mydircopy')
Note that by default dirs_exist_ok is False meaning that destination should not exist, when shutil.copytree is launched.

Python: folder creation when copying files

I'm trying to create a shell script that will copy files from one computer (employee's old computer) to another (employee's new computer). I have it to the point where I can copy files over, thanks to the lovely people here, but I'm running into a problem - if I'm going from, say, this directory that has 2 files:
C:\Users\specificuser\Documents\Test Folder
....to this directory...
C:\Users\specificuser\Desktop
...I see the files show up on the Desktop, but the folder those files were in (Test Folder) isn't created.
Here is the copy function I'm using:
#copy function
def dir_copy(srcpath, dstpath):
#if the destination path doesn't exist, create it
if not os.path.exists(dstpath):
os.makedir(dstpath)
#tag each file to the source path to create the file path
for file in os.listdir(srcpath):
srcfile = os.path.join(srcpath, file)
dstfile = os.path.join(dstpath, file)
#if the source file path is a directory, copy the directory
if os.path.isdir(srcfile):
dir_copy(srcfile, dstfile)
else: #if the source file path is just a file, copy the file
shutil.copyfile(srcfile, dstfile)
I know I need to create the directory on the destination, I'm just not quite sure how to do it.
Edit: I found that I had a type (os.makedir instead of os.mkdir). I tested it, and it creates directories like it's supposed to. HOWEVER I'd like it to create the directory one level up from where it's starting. For example, in Test Folder there is Sub Test Folder. It has created Sub Test Folder but won't create Test Folder because Test Folder is not part of the dstpath. Does that make sense?

You might want to look at shutil.copytree(). It performs the recursive copy functionality, including directories, that you're looking for. So, for a basic recursive copy, you could just run:
shutil.copytree(srcpath, dstpath)
However, to accomplish your goal of copying the source directory to the destination directory, creating the source directory inside of the destination directory in the process, you could use something like this:
import os
import shutil
def dir_copy(srcpath, dstdir):
dirname = os.path.basename(srcpath)
dstpath = os.path.join(dstdir, dirname)
shutil.copytree(srcpath, dstpath)
Note that your srcpath must not contain a slash at the end for this to work. Also, the result of joining the destination directory and the source directory name must not already exist, or copytree will fail.

This is a common problem with file copy... do you intend to just copy the contents of the folder or do you want the folder itself copied. Copy utilities typically have a flag for this and you can too. I use os.makedirs so that any intermediate directories are created also.
#copy function
def dir_copy(srcpath, dstpath, include_directory=False):
if include_directory:
dstpath = os.path.join(dstpath, os.path.basename(srcpath))
os.makedirs(dstpath, exist_ok=True)
#tag each file to the source path to create the file path
for file in os.listdir(srcpath):
srcfile = os.path.join(srcpath, file)
dstfile = os.path.join(dstpath, file)
#if the source file path is a directory, copy the directory
if os.path.isdir(srcfile):
dir_copy(srcfile, dstfile)
else: #if the source file path is just a file, copy the file
shutil.copyfile(srcfile, dstfile)

import shutil
import os
def dir_copy(srcpath, dstpath):
try:
shutil.copytree(srcpath, dstpath)
except shutil.Error as e:
print('Directory not copied. Error: %s' % e)
except OSError as e:
print('Directory not copied. Error: %s' % e)
dir_copy('/home/sergey/test1', '/home/sergey/test2')

I use this script to backup (copy) my working folder. It will skip large files, keep folder structure (hierarchy) and create destination folders if they don't exist.
import os
import shutil
for root, dirs, files in os.walk(the_folder_copy_from):
for name in files:
if os.path.getsize(os.path.join(root, name))<10*1024*1024:
target=os.path.join("backup", os.path.relpath(os.path.join(root, name),start=the_folder_copy_from))
print(target)
os.makedirs(os.path.dirname(target),exist_ok=True)
shutil.copy(src=os.path.join(root, name),dst=target)
print("Done")

how to make a tar file backup of a directory in python

What I'm trying to do: I'm trying to make a recursive .tar file backup of the directory this python script is run in.
What I currently have:
import os
import zipfile
import datetime
import tarfile
datetime = str( datetime.datetime.now() )
def zipdir(path, zip):
for root, dirs, files in os.walk(path):
for file in files:
zip.write(os.path.join(root, file))
backupdir = raw_input('Which directory should we backup to? \n')
if backupdir :
try:
zipf = zipfile.ZipFile('DrupalInstanceBackup'+datetime+'.zip', mode='w')
zipdir('/Users/localuser/Downloads/backup', zipf)
except Exception as e:
print e
finally:
zipf.close()
What it currently does:
It makes a .zip backup, but when extracted it doesn't show any files.
What Im trying to do:
Can someone help me make this script recursively backup a directory and create a .tar archive of the directory and its files in a recursive manner?
Thank you

The good news is that tarfile supports recursively adding members any work.
with tarfile.open(archive_name + '.tar.gz', mode='w:gz') as archive:
archive.add('/Users/localuser/Downloads/backup', recursive=True)
recursive=True is the default, so you can omit it.

How do i copy files with shutil in python

im trying to copy a .txt file from one dest to another. This code is running but the file isnt copying. What am i doing wrong?
import shutil
import os
src = "/c:/users/mick/temp"
src2 = "c:/users/mick"
dst = "/c:/users/mick/newfolder1/"
for files in src and src2:
if files.endswith(".txt"):
shutil.copy(files, dst)

Your for loop isn't actually searching through the files of each of your sources. In addition your for loop isn't looping through each of the sources but rather the letters in src and src2 that are present in both. This adjustment should handle what you need.
import os
src = ["c:/users/mick/temp", "c:/users/mick"]
dst = "c:/users/mick/newfolder1/"
for source in src:
for src_file in os.listdir(source):
if src_file.endswith(".txt"):
old_path = os.path.join(source, src_file)
new_path = os.path.join(dst, src_file)
os.rename(old_path, new_path)
You shouldn't need shutil for this situation as it is simply a more powerful os.rename that attempts to handle different scenarios a little better (to my knowledge). However if "newfolder1" isn't already existant than you will want to replace os.rename() with os.renames() as this attempts to create the directories inbetween.

shutils is useful for copying, but to move a file use
os.rename(oldpath, newpath)

I know the question is little older. But Here are the simple way to achieve your task. You may change the file extension as you wish. For copying you may use copy() & for moving yo may change it to move(). Hope this helps.
#To import the modules
import os
import shutil
#File path list atleast source & destination folders must be present not everything.
#*******************************WARNING**********************************************
#Make sure these folders names are already created while using the 'move() / copy()'*
#************************************************************************************
filePath1 = 'C:\\Users\\manimani\\Downloads' #Downloads folder
filePath2 = 'F:\\Projects\\Python\\Examples1' #Downloads folder
filePath3 = 'C:\\Users\\manimani\\python' #Python program files folder
filePath4 = 'F:\\Backup' #Backup folder
filePath5 = 'F:\\PDF_Downloads' #PDF files folder
filePath6 = 'C:\\Users\\manimani\\Videos' #Videos folder
filePath7 = 'F:\\WordFile_Downloads' #WordFiles folder
filePath8 = 'F:\\ExeFiles_Downloads' #ExeFiles folder
filePath9 = 'F:\\Image_Downloads' #JPEG_Files folder
#To change the directory from where the files to look // ***Source Directory***
os.chdir(filePath8)
#To get the list of files in the specific source directory
myFiles = os.listdir()
#To move all the '.docx' files to a different location // ***Destination Directory***
for filename in myFiles:
if filename.endswith('.docx'):
print('Moving the file : ' + filename)
shutil.move(filename, filePath4) #Destination directory name
print('All the files are moved as directed. Thanks')

import os, shutil
src1 = "c:/users/mick/temp/"
src2 = "c:/users/mick/"
dist = "c:/users/mick/newfolder"
if not os.path.isdir(dist) : os.mkdir(dist)
for src in [src1, src2] :
for f in os.listdir(src) :
if f.endswith(".txt") :
#print(src) # For testing purposes
shutil.copy(src, dist)
os.remove(src)
I think the error was that you were trying to copy a directory as a file, but you forgot to go through the files in the directory. You were just doing shutil.copy("c:/users/mick/temp/", c:/users/mick/newfolder/").

Python - Move and overwrite files and folders

I have a directory, 'Dst Directory', which has files and folders in it and I have 'src Directory' which also has files and folders in it. What I want to do is move the contents of 'src Directory' to 'Dst Directory' and overwrite anyfiles that exist with the same name. So for example 'Src Directory\file.txt' needs to be moved to 'Dst Directory\' and overwrite the existing file.txt. The same applies for some folders, moving a folder and merging the contents with the same folder in 'dst directory'
I'm currently using shutil.move to move the contents of src to dst but it won't do it if the files already exist and it won't merge folders; it'll just put the folder inside the existing folder.
Update: To make things a bit clearer, what I'm doing is unzipping an archive to the Dst Directory and then moving the contents of Src Directory there and rezipping, effectively updating files in the zip archive. This will be repeated for adding new files or new versions of files etc which is why it needs to overwrite and merge
Solved: I solved my problem by using distutils.dir_util.copy_tree(src, dst), this copies the folders and files from src directory to dst directory and overwrites/merges where neccesary.

This will go through the source directory, create any directories that do not already exist in destination directory, and move files from source to the destination directory:
import os
import shutil
root_src_dir = 'Src Directory\\'
root_dst_dir = 'Dst Directory\\'
for src_dir, dirs, files in os.walk(root_src_dir):
dst_dir = src_dir.replace(root_src_dir, root_dst_dir, 1)
if not os.path.exists(dst_dir):
os.makedirs(dst_dir)
for file_ in files:
src_file = os.path.join(src_dir, file_)
dst_file = os.path.join(dst_dir, file_)
if os.path.exists(dst_file):
# in case of the src and dst are the same file
if os.path.samefile(src_file, dst_file):
continue
os.remove(dst_file)
shutil.move(src_file, dst_dir)
Any pre-existing files will be removed first (via os.remove) before being replace by the corresponding source file. Any files or directories that already exist in the destination but not in the source will remain untouched.

Use copy() instead, which is willing to overwrite destination files. If you then want the first tree to go away, just rmtree() it separately once you are done iterating over it.
http://docs.python.org/library/shutil.html#shutil.copy
http://docs.python.org/library/shutil.html#shutil.rmtree
Update:
Do an os.walk() over the source tree. For each directory, check if it exists on the destination side, and os.makedirs() it if it is missing. For each file, simply shutil.copy() and the file will be created or overwritten, whichever is appropriate.

Since none of the above worked for me, so I wrote my own recursive function. Call Function copyTree(dir1, dir2) to merge directories. Run on multi-platforms Linux and Windows.
def forceMergeFlatDir(srcDir, dstDir):
if not os.path.exists(dstDir):
os.makedirs(dstDir)
for item in os.listdir(srcDir):
srcFile = os.path.join(srcDir, item)
dstFile = os.path.join(dstDir, item)
forceCopyFile(srcFile, dstFile)
def forceCopyFile (sfile, dfile):
if os.path.isfile(sfile):
shutil.copy2(sfile, dfile)
def isAFlatDir(sDir):
for item in os.listdir(sDir):
sItem = os.path.join(sDir, item)
if os.path.isdir(sItem):
return False
return True
def copyTree(src, dst):
for item in os.listdir(src):
s = os.path.join(src, item)
d = os.path.join(dst, item)
if os.path.isfile(s):
if not os.path.exists(dst):
os.makedirs(dst)
forceCopyFile(s,d)
if os.path.isdir(s):
isRecursive = not isAFlatDir(s)
if isRecursive:
copyTree(s, d)
else:
forceMergeFlatDir(s, d)

You can use this to copy directory overwriting existing files:
import shutil
shutil.copytree("src", "dst", dirs_exist_ok=True)
dirs_exist_ok argument was added in Python 3.8.
See docs: https://docs.python.org/3/library/shutil.html#shutil.copytree

If you also need to overwrite files with read only flag use this:
def copyDirTree(root_src_dir,root_dst_dir):
"""
Copy directory tree. Overwrites also read only files.
:param root_src_dir: source directory
:param root_dst_dir: destination directory
"""
for src_dir, dirs, files in os.walk(root_src_dir):
dst_dir = src_dir.replace(root_src_dir, root_dst_dir, 1)
if not os.path.exists(dst_dir):
os.makedirs(dst_dir)
for file_ in files:
src_file = os.path.join(src_dir, file_)
dst_file = os.path.join(dst_dir, file_)
if os.path.exists(dst_file):
try:
os.remove(dst_file)
except PermissionError as exc:
os.chmod(dst_file, stat.S_IWUSR)
os.remove(dst_file)
shutil.copy(src_file, dst_dir)

Have a look at: os.remove to remove existing files.

I had a similar problem. I wanted to move files and folder structures and overwrite existing files, but not delete anything which is in the destination folder structure.
I solved it by using os.walk(), recursively calling my function and using shutil.move() on files which I wanted to overwrite and folders which did not exist.
It works like shutil.move(), but with the benefit that existing files are only overwritten, but not deleted.
import os
import shutil
def moverecursively(source_folder, destination_folder):
basename = os.path.basename(source_folder)
dest_dir = os.path.join(destination_folder, basename)
if not os.path.exists(dest_dir):
shutil.move(source_folder, destination_folder)
else:
dst_path = os.path.join(destination_folder, basename)
for root, dirs, files in os.walk(source_folder):
for item in files:
src_path = os.path.join(root, item)
if os.path.exists(dst_file):
os.remove(dst_file)
shutil.move(src_path, dst_path)
for item in dirs:
src_path = os.path.join(root, item)
moverecursively(src_path, dst_path)