I want to move a lot of files. the path to these files is stored in a list. I want to keep the whole directory structure but want to move them to a different folder.
So for example the files are
D:\test\test1\test1.txt
D:\test\test1\test2.txt
I want to move them to C:\ from D:\ and keep the directory structure. How should I go about doing it?
this is the code i have, it is not working
import os, fnmatch
import shutil
f=open('test_logs.txt','r') #logs where filenames are stored with filenames as first entry
for line in f:
filename=line.split()
output_file="C:" + filename[0].lstrip("D:")
shutil.move(filename[0],output_file)
I read the file name fine and I can generate the destination filename fine but when I run it, it gives me an error saying "No such file or directory" (and gives the path of the output filename).
I think you want something like this:
import sys
import os
import shutil
# terminology:
# path = full path to a file, i.e. directory + file name
# directory = directory, possibly starting with a drive
# file name = the last component of the path
sourcedrive = 'D:'
destdrive = 'C:'
log_list_file = open('test_logs.txt', 'r')
for line in log_list_file:
sourcepath = line.split()[0] # XXX is this correct?
if sourcepath.startswith(sourcedrive):
destpath = sourcepath.replace(sourcedrive, destdrive, 1)
else:
print >>sys.stderr, 'Skipping %s: Not on %s' % (sourcepath, sourcedrive)
continue
destdir = os.path.dirname(destpath)
if not os.path.isdir(destdir):
try:
os.makedirs(destdir)
except (OSError, IOError, Error) as e:
print >>sys.stderr, 'Error making %s: %s' % (destdir, e)
continue
try:
shutil.move(sourcepath, destpath)
except (OSError, IOError, Error) as e:
print >>sys.stderr, 'Error moving %s to %s: %s' % (sourcepath, destpath, e)
Do you also want to remove the source directory if it's empty?
Update: Ah, ok, I see the problem -- shutil.move won't copy to a nonexistent directory. To do what you're trying to do, you have to create the new directory tree first. Since it's a bit safer to use a built-in move function than to roll your own copy-and-delete procedure, you could do this:
with open('test_logs.txt','r') as f:
files_to_copy = [line.split()[0] for line in f]
paths_to_copy = set(os.path.split(filename)[0] for filename in files_to_copy)
def ignore_files(path, names, ptc=paths_to_copy):
return [name for name in names if os.path.join(path, name) not in ptc]
shutil.copytree(src, dst, ignore=ignore_files)
for filename in files_to_copy:
output_file="C:" + filename.lstrip("D:")
shutil.move(filename, output_file)
Let me know if that doesn't work
Original Post: If you want to move only some of the files, your best bet is to use shutil.copytree's ignore keyword. Assuming your list of files includes full paths and directories (i.e. ['D:\test\test1\test1.txt', 'D:\test\test1\test2.txt', 'D:\test\test1']), create an ignore_files function and use it like this:
files_to_copy = ['D:\test\test1\test1.txt', 'D:\test\test1\test2.txt', 'D:\test\test1']
def ignore_files(path, names, ftc=files_to_copy):
return [name for name in names if os.path.join(path, name) not in ftc]
shutil.copytree(src, dst, ignore=ignore_files)
Then you can just delete the files in files_to_copy:
for f in files_to_copy:
try:
os.remove(f)
except OSError: # can't remove() a directory, so pass
pass
I tested this -- make certain that you include the paths you want to copy as well as the files in files_to_copy -- otherwise, this will delete files without copying them.
Related
I am aiming to create a function that does the following:
Declare a path with a file, not just a folder. e.g. 'C:/Users/Lampard/Desktop/Folder1/File.py'
Create a folder in same folder as the declared file path - Calling it 'Archive'
Cut the file and paste it into the new folder just created.
If the folder 'Archive' already exists - then simply cut and paste the file into there
I have spent approx. 15-20min going through these:
https://www.programiz.com/python-programming/directory
Join all except last x in list
https://docs.python.org/3/library/pathlib.html#operators
And here is what I got to:
import os
from pathlib import Path, PurePath
from shutil import copy
#This path will change every time - just trying to get function right first
path = 'C:/Users/Lampard/Desktop/Folder1/File.py'
#Used to allow suffix function
p = PurePath(path)
#Check if directory is a file not a folder
if not p.suffix:
print("Not an extension")
#If it is a file
else:
#Create new folder before last file
#Change working directory
split = path.split('/')
new_directory = '/'.join(split[:-1])
apply_new_directory = os.chdir(new_directory)
#If folder does not exist create it
try:
os.mkdir('Archive')#Create new folder
#If not, continue process to copy file and paste it into Archive
except FileExistsError:
copy(path, new_directory + '/Archive/' + split[-1])
Is this code okay? - does anyone know a simpler method?
Locate folder/file in path
print [name for name in os.listdir(".") if os.path.isdir(name)]
Create path
import os
# define the name of the directory to be created
path = "/tmp/year"
try:
os.mkdir(path)
except OSError:
print ("Creation of the directory %s failed" % path)
else:
print ("Successfully created the directory %s " % path)
To move and cut files you can use this library
As you're already using pathlib, there's no need to use shutil:
from pathlib import Path
path = 'C:/Users/Lampard/Desktop/Folder1/File.py' # or whatever
p = Path(path)
target = Path(p.with_name('Archive')) # replace the filename with 'Archive'
target.mkdir() # create target directory
p.rename(target.joinpath(p.name)) # move the file to the target directory
Feel free to add appriopriate try…except statements to handle any errors.
Update: you might find this version more readable:
target = p.parent / 'Archive'
target.mkdir()
p.rename(target / p.name)
This is an example of overloading / operator.
I need to copy a file to a subdirectory with its name changed to name_timestamp(by appending the time stamp).
I'm using a COPY method on a csv file and once that copy is done, I need to move the CSV file to a subdirectory and rename it to CSV_timestamp.
Below is the sample code. Can anyone help me or suggest me on how to do this?
import os, shutil, time
if not os.path.exists(dirName):
os.mkdir(dirName)
print("Directory " , dirName , " Created ")
else:
print("Directory " , dirName , " already exists")
def copyFile(src, dest):
try:
shutil.copy(src, dest)
# eg. src and dest are the same file
except shutil.Error as e:
print('Error: %s' % e)
# eg. source or destination doesn't exist
except IOError as e:
print('Error: %s' % e.strerror)
os.rename("path/to/current/age.csv", "path/to/new/destination/for/age.csv")
shutil.move("path/to/current/age.csv", "path/to/new/destination/for/age.csv")
Sticking to your approach (with a function that checks whether the directory exists etc.), you could do the following:
import os
import shutil
from time import gmtime, strftime
def copyFile(old_path, new_directory):
# check if the directory already exists
if not os.path.exists(new_directory):
os.mkdir(new_directory)
print(f"Directory {new_directory} Created.")
else:
print(f"Directory {new_directory} already exists.")
# create new path from new_directory, the filename and the timestamp
new_path = new_directory + old_path.split("/")[len(old_path)-1].split(".")[0] + strftime("%Y_%m_%d", gmtime()) + ".csv"
# copy the file to the new path
try:
shutil.copy(old_path, new_path)
# eg. src and dest are the same file
except shutil.Error as e:
print(f"Error: {e}")
# eg. source or destination doesn't exist
except IOError as e:
print(f"Error: {e.strerror}")
old_path = '/path/to/directory/file.csv'
new_directory = '/path/to/new/directory/'
copyFile(old_path, new_directory)
Please note that
As of Python 3.6, you can use f-strings to include variables in strings more easily.
It wasn't clear to me what kind of timestamp you are looking for.
This approach will give you a stamp in the format of YYYY_MM_DD
of the current day, but you could easily change that, see the
documentation on the time package.
After using shutil.copy, you don't need shutil.move any more, as
the first already copied your file and saved it to the destination
path.
It should be as simple as setting your destination path to whatever directory you want it in when you finish.
For example, say your file is in users/foo/bar/myfile.csv (this would be your src path). Suppose you want that file's copy in users/mydocuments/bar/foo/mynewfile.csv (this would be your dest path).
All you'd need to do is:
import shutil
import os
src = 'users/foo/bar/myfile.csv'
tstamp = os.path.getmtime(path)
dest = 'users/mydocuments/bar/foo/mynewfile' + tstamp + '.csv'
shutil.move(src,dest)
you can use os.rename() insteed
import time
timestamp_name=int(time.time())
os.rename('path/to/file/name.text','path/to/file/'+timestamp_name+'.txt)
I wrote a small python program which processes all the files in a directory. I want to restrict that to include only JSON files in that directory. For example, the line fname in fileList: in the code snipped below should only enumerate files with the extension *.json
#Set the directory you want to start from
rootDir = '/home/jas_parts'
for dirName, subdirList, fileList in os.walk(rootDir):
print('Found directory: %s' % dirName)
for fname in fileList:
print('\t%s' % fname)
fname='jas_parts/'+fname
with open(fname, 'r+') as f:
json_data = json.load(f)
event = json_data['Total']
print(event)
Since your file name is string you can use the str.endswith method to check if it is json file.
if fname.endswith('.json'):
#do_something()
Just filter the names that you are interested in.
if fname[-5:] == '.json':
(of course, you can also use os.path.splitext, or re, doesn't really matter how you get to the extension)
Here's a general solution to the question: "How do I do X to all files with names matching some pattern under directory Y?"
#!python
from __future__ import print_function
import fnmatch, os, os.path
def files_under(directory, pattern):
'''Yield all files matching pattern under some directory
'''
for p, dnames, fnames in os.walk(directory):
for match in fnmatch.filter(fnames, pattern):
yield(os.path.join(p, match))
if __name__ == '__main__':
import sys
if len(sys.argv) < 3:
print('Must supply path and (quoted) pattern', file=sys.stderr)
sys.exit(1)
try:
for each in files_under(sys.argv[1], sys.argv[2]):
print(each)
except EnvironmentError, e:
print ('Error trying to walk tree: %s ' % e, file=sys.stderr)
sys.exit(2)
The function is files_under() and the rest is just a very simplistic wrapper around it to print the matching results.
It's also easy to extend this to handle multiple patterns and even, with a little extra work, to ensure that files with names matching multiple patterns are only yielded once each. But I'll leave these enhancements as an exercise to the student.
I am trying to find some files, create a folder and move the files in there.
def test():
try:
logfile = "C:\\Users\\alkis\\Desktop\\testouter\\test"
result_dir = os.path.join(logfile, "testzip")
print result_dir
os.makedirs(result_dir)
os.chmod(result_dir, stat.S_IWRITE)
kpath = logfile + "\\*.jpg"
print kpath
files = glob.glob(kpath)
for file in files:
filename = os.path.splitext(file)[0]
print filename
os.chmod(filename, stat.S_IWRITE)
shutil.move(filename, result_dir)
except Exception, e:
#shutil.rmtree(result_dir)
print e.__doc__ + "\r\n"
print e.message
return
The error I am getting is: MS-Windows OS call failed
I check the permissions on my files and they are not read only.
You are listing each file, removing the extension, then trying to move that filename.
The extension is part of the filename, don't remove it. Windows Exlorer hides the extension only when displaying files.
You also don't need to call os.chmod() on the filename; just skip that step:
for file in files:
filename = os.path.splitext(file)[0]
print filename
shutil.move(filename, result_dir)
I'm trying to delete old SVN files from directory tree. shutil.rmtree and os.unlink raise WindowsErrors, because the script doesn't have permissions to delete them. How can I get around that?
Here is the script:
# Delete all files of a certain type from a direcotry
import os
import shutil
dir = "c:\\"
verbosity = 0;
def printCleanMsg(dir_path):
if verbosity:
print "Cleaning %s\n" % dir_path
def cleandir(dir_path):
printCleanMsg(dir_path)
toDelete = []
dirwalk = os.walk(dir_path)
for root, dirs, files in dirwalk:
printCleanMsg(root)
toDelete.extend([root + os.sep + dir for dir in dirs if '.svn' == dir])
toDelete.extend([root + os.sep + file for file in files if 'svn' in file])
print "Items to be deleted:"
for candidate in toDelete:
print candidate
print "Delete all %d items? [y|n]" % len(toDelete)
choice = raw_input()
if choice == 'y':
deleted = 0
for filedir in toDelete:
if os.path.exists(filedir): # could have been deleted already by rmtree
try:
if os.path.isdir(filedir):
shutil.rmtree(filedir)
else:
os.unlink(filedir)
deleted += 1
except WindowsError:
print "WindowsError: Couldn't delete '%s'" % filedir
print "\nDeleted %d/%d files." % (deleted, len(toDelete))
exit()
if __name__ == "__main__":
cleandir(dir)
Not a single file is able to be deleted. What am I doing wrong?
To remove recursively all .svn I use this script. May be it will help someone.
import os, shutil, stat
def del_evenReadonly(action, name, exc):
os.chmod(name, stat.S_IWRITE)
os.remove(name)
for root, subFolders, files in os.walk(os.getcwd()):
if '.svn' in subFolders:
shutil.rmtree(root+'\.svn',onerror=del_evenReadonly)
Subversion usually makes all the .svn directories (and everything in them) write protected. Probably you have to remove the write protection before you can remove the files.
I'm not really sure how to do this best with Windows, but you should be able to use os.chmod() with the stat.S_IWRITE flag. Probably you have to iterate through all the files in the .svn directories and make them all writable individually.