Basically what I want to do in my script is copy some files from dest_path to source_path. You can set it up and see how it works -
But for some reason it copies the first file and tells me that the rest is already copied, which is not true. Is there something that I'm not seeing or that I did wrong? im pretty new to python so sorry if I did something obviously wrong, I just cant see it...
import time, shutil, os, datetime
source_path = r"C:\SOURCE" # Your Source path
dest_path = r"C:\DESTINATION" # Destination path
file_ending = '.txt' # Needed File ending
files = os.listdir(source_path) # defines
date = datetime.datetime.now().strftime('%d.%m.%Y') # get the current date
while True:
print("Beginning checkup")
print("=================")
if not os.path.exists(source_path or dest_path): # checks if directory exists
print("Destination/Source Path does not exist!")
else:
print("Destination exists, checking files...")
for f in files:
if f.endswith(file_ending):
new_path = os.path.join(dest_path, date, )
src_path = os.path.join(source_path, f)
if not os.path.exists(new_path): # create the folders if they dont already exists
print("copying " + src_path)
os.makedirs(new_path)
shutil.copy(src_path, new_path)
else:
print( src_path + " already copied")
# shutil.copy(src_path, new_path)
print("=================")
print('Checkup done, waiting for next round...')
time.sleep(10) # wait a few seconds between looking at the directory
Like #user2357112 mentioned if not os.path.exists(source_path or dest_path) is not doing what you think. Change to
if not os.path.exists(source_path) or not os.path.exists(dest_path):
This will only copy one file because it creates the directory new_path the first time through in the if. Something like this should work:
if f.endswith(file_ending):
new_path = os.path.join(dest_path, date, )
src_path = os.path.join(source_path, f)
if not os.path.exists(new_path): # create the folders if they dont already exists
os.makedirs(new_path)
if not os.path.exists(os.path.join(new_path,f)):
print("copying " + src_path)
shutil.copy(src_path, os.path.join(new_path,f))
else:
print( src_path + " already copied")
If the new_path directory doesn't exist then make the directory (this should only happen once and this if could be moved outside of the loop as well as new_path initialization). In a separate if check if the file exists inside that directory and if not copy the file to the location, else print your message.
Related
I need to copy a file to a subdirectory with its name changed to name_timestamp(by appending the time stamp).
I'm using a COPY method on a csv file and once that copy is done, I need to move the CSV file to a subdirectory and rename it to CSV_timestamp.
Below is the sample code. Can anyone help me or suggest me on how to do this?
import os, shutil, time
if not os.path.exists(dirName):
os.mkdir(dirName)
print("Directory " , dirName , " Created ")
else:
print("Directory " , dirName , " already exists")
def copyFile(src, dest):
try:
shutil.copy(src, dest)
# eg. src and dest are the same file
except shutil.Error as e:
print('Error: %s' % e)
# eg. source or destination doesn't exist
except IOError as e:
print('Error: %s' % e.strerror)
os.rename("path/to/current/age.csv", "path/to/new/destination/for/age.csv")
shutil.move("path/to/current/age.csv", "path/to/new/destination/for/age.csv")
Sticking to your approach (with a function that checks whether the directory exists etc.), you could do the following:
import os
import shutil
from time import gmtime, strftime
def copyFile(old_path, new_directory):
# check if the directory already exists
if not os.path.exists(new_directory):
os.mkdir(new_directory)
print(f"Directory {new_directory} Created.")
else:
print(f"Directory {new_directory} already exists.")
# create new path from new_directory, the filename and the timestamp
new_path = new_directory + old_path.split("/")[len(old_path)-1].split(".")[0] + strftime("%Y_%m_%d", gmtime()) + ".csv"
# copy the file to the new path
try:
shutil.copy(old_path, new_path)
# eg. src and dest are the same file
except shutil.Error as e:
print(f"Error: {e}")
# eg. source or destination doesn't exist
except IOError as e:
print(f"Error: {e.strerror}")
old_path = '/path/to/directory/file.csv'
new_directory = '/path/to/new/directory/'
copyFile(old_path, new_directory)
Please note that
As of Python 3.6, you can use f-strings to include variables in strings more easily.
It wasn't clear to me what kind of timestamp you are looking for.
This approach will give you a stamp in the format of YYYY_MM_DD
of the current day, but you could easily change that, see the
documentation on the time package.
After using shutil.copy, you don't need shutil.move any more, as
the first already copied your file and saved it to the destination
path.
It should be as simple as setting your destination path to whatever directory you want it in when you finish.
For example, say your file is in users/foo/bar/myfile.csv (this would be your src path). Suppose you want that file's copy in users/mydocuments/bar/foo/mynewfile.csv (this would be your dest path).
All you'd need to do is:
import shutil
import os
src = 'users/foo/bar/myfile.csv'
tstamp = os.path.getmtime(path)
dest = 'users/mydocuments/bar/foo/mynewfile' + tstamp + '.csv'
shutil.move(src,dest)
you can use os.rename() insteed
import time
timestamp_name=int(time.time())
os.rename('path/to/file/name.text','path/to/file/'+timestamp_name+'.txt)
What I'm hoping to accomplish is to scan all the files in the specified directory and remove the numbers that is contained in the name. The problem is I can't change the directory with the code that is below.
I'm currently in the Udacity Full Stack Nanodegree Program so if anyone can help me out that is also in the program that would be a plus.
Here is the code:
import os
def rename_files():
#(1) Get file names from a folder
file_list = os.listdir("/Users/bill/Documents/web/LocalServer/prank")
#print (file_list)
saved_path = os.getcwd()
print("Our current working directory is " + saved_path)
os.chdir('/Users/bill/Documents/web/LocalServer/prank')
print("Our current working directory is ", saved_path)
#(2) For each file, rename filename
for file_name in file_list:
#print("Old Name - " + file_name)
#print("New Name - " + file_name.translate("0123457689"))
os.rename(file_name, file_name.translate("0123457689"))
os.chdir(saved_path)
print("Our current working directory is ", saved_path)
rename_files()
Here is the output I get:
Our current working directory is /Users/bill/Documents/web/LocalServer
Our current working directory is /Users/bill/Documents/web/LocalServer
Our current working directory is /Users/bill/Documents/web/LocalServer
Update 1:
I've finally changed directories but I still can't rename files. (Ex: 68chicago.jpg to chicago.jpg)
Here is my current code:
import os
def rename_files():
#(1) Get file names from a folder
file_list = os.listdir("/Users/bill/Documents/web/LocalServer/prank")
#print (file_list)
saved_path = os.getcwd()
print("Our current working directory is " + saved_path)
os.chdir('/Users/bill/Documents/web/LocalServer/prank')
new_path = os.getcwd()
print("Our current working directory is ", new_path)
#(2) For each file, rename filename
for file_name in file_list:
#print("Old Name - " + file_name)
#print("New Name - " + file_name.translate("0123457689"))
os.rename(file_name, file_name.translate("0123457689"))
print("Our current working directory is ", new_path)
rename_files()
Here is my current output:
Our current working directory is /Users/bill/Documents/web/LocalServer
Our current working directory is /Users/bill/Documents/web/LocalServer/prank
Our current working directory is /Users/bill/Documents/web/LocalServer/prank
Update 2:
I've finally solved the problem thanks to #Dan.
Here is his code:
import os
def rename_files():
#(1) Get file names from a folder
file_list = os.listdir("/Users/bill/Documents/web/LocalServer/prank")
#print (file_list)
saved_path = os.getcwd()
print("Our current working directory is " + saved_path)
os.chdir('/Users/bill/Documents/web/LocalServer/prank')
new_path = os.getcwd()
print("Our current working directory is ", new_path)
#(2) For each file, rename filename
for file_name in file_list:
#print("Old Name - " + file_name)
#print("New Name - " + file_name.translate("0123457689"))
os.rename(file_name, ''.join([i for i in file_name if not i.isdigit()])) # This works on my machine
os.chdir(saved_path)
print("Our current working directory is ", saved_path)
rename_files()
It looks like you want to change to a dir and then change back to the first directory? If that's the case you want something like this:
import os
def rename_files():
#(1) Get file names from a folder
file_list = os.listdir("/Users/bill/Documents/web/LocalServer/prank")
#print (file_list)
saved_path = os.getcwd()
print("Our current working directory is " + saved_path)
os.chdir('/Users/bill/Documents/web/LocalServer/prank')
new_path = os.getcwd()
print("Our current working directory is ", new_path)
#(2) For each file, rename filename
for file_name in file_list:
#print("Old Name - " + file_name)
#print("New Name - " + file_name.translate("0123457689"))
os.rename(file_name, ''.join([i for i in file_name if not i.isdigit()])) # This works on my machine
os.chdir(saved_path)
print("Our current working directory is ", saved_path)
rename_files()
Your problem is that you use .translate() incorrectly. It does not change the file names at all, so rename actually renames X into X:
'68chicago.jpg'.translate('0123457689')
# '68chicago.jpg'
Try using list comprehension:
def clean_name(name):
return ''.join(x for x in name if not x.isdigit())
# 'chicago.jpg'
Here's a version that doesn't require changing the current directory. Just specify the directory where files are to be renamed. If you specify the full path to the old and new name changing the directory isn't required.
This also shows how to use translate properly. maketrans takes one, two, or three parameters (see docs). The three-parameter version takes two strings of equal length for 1:1 translating, plus a third parameter of characters to delete. It returns a dictionary suitable to be used with translate.
import os
def rename_files(directory):
xlat = str.maketrans('','','0123457689')
file_list = os.listdir(directory)
for file_name in file_list:
old_name = os.path.join(directory,file_name)
new_name = os.path.join(directory,file_name.translate(xlat))
os.rename(old_name,new_name)
rename_files('/Users/bill/Documents/web/LocalServer/prank')
The primary problem is your code isn't using the str.translate() method in the proper manner—your need to pass it a translation table for it to work.
This following works and avoids renaming files that don't need to be (i.e. doesn't rename them to their current name). This takes a little extra processing, but is probably faster that performing a useless OS-level operation.
Note: A more robust solution would also need to handle any exceptions the os.rename() call might raise—which definitely could occur for a number of reasons.
For example:
Perhaps a file with the translated name already exists.
All the characters of the original file get removed, so the new file name is the empty string.
File permissions or attribute prevent renaming.
etc.
The code:
from contextlib import contextmanager
import string
import os
#contextmanager
def temp_cd(path):
"""Temporarily change the current directory to path, yield, then restore it."""
saved_path = os.getcwd()
yield
os.chdir(saved_path)
def rename_files(path):
TRANS_TABLE = str.maketrans(dict.fromkeys(string.digits))
with temp_cd(path):
for file_name in os.listdir(path):
if any(ch in file_name for ch in string.digits):
# os.rename(file_name, file_name.translate(TRANS_TABLE))
print(file_name, '->', file_name.translate(TRANS_TABLE))
rename_files("/Users/bill/Documents/web/LocalServer/prank")
I have written a script to move video files from one directory to another, it will also search sub directories using os.walk. however if the script finds a video file it will only move the file and not the containing folder. i have added an if statement to check if the containing folder is different to the original search folder.
i cant find the code to actually move(or rename?) the folder and file to a different directory. i have read/watch a lot on moving files and there is a lot of information on that, but i cant find anything for moving folders.
i have tried using shutil.move and os.rename and i get an error both times. when i try and search for the problem i get a lot of results about how to move files, or how to change the current working directory of python.
any advice(even how to phrase the google search to accuratly describe how to find a tutorial on the subject) would be really appreciated. it's my first real world python program and ive learnt a lot but this last step is wearing me down!
EDIT: when trying to use os.rename(src_file, dst_file) i get the error WindowsError: error 3 The system cannot find the path specified.
when trying shutil.move(src_file, dst_file) i get ioerror errno 2 no such file or directory "H:\\Moviesfrom download...\OneOfTheVideoFilesNotInParentFolder ie the folder and the file needs to move.
thanks.
ps like i said it's my first script outside of code academy so any random suggestions would also be appreciated.
import os
import shutil
import time
movietypes = ('.3gp', '.wmv', '.asf', '.avi', '.flv', '.mov', '.mp4', '.ogm', '.mkv',
'. mpg', '.mpg', '.nsc', '.nsv', '.nut', '.a52', '.tta', '.wav', '.ram', '.asf',
'.wmv', '. ogg', '.mka', '.vid', '.lac', '.aac', '.dts', '.tac',
'.dts', '.mbv')
filewrite = open('H:\\Movies from download folder\\Logs\\logstest.txt', 'w')
dir_src = "C:\\Users\\Jeremy\\Downloads\\"
dir_dst = "H:\\Movies from download folder\\"
for root, dirs, files in os.walk(dir_src):
for file in files:
if file.endswith(movietypes) == True:
filestr = str(file)
locationoffoundfile = os.path.realpath(os.path.join(root,filestr))
folderitwasin = locationoffoundfile.replace(dir_src,'')
folderitwasin = folderitwasin.replace(filestr,'')
pathofdir = os.path.realpath(root) + "\\"
if pathofdir != dir_src:
src_file = locationoffoundfile
dst_file = dir_dst + folderitwasin + filestr
os.rename(src_file, dst_file) #****This line is the line im having issues with***
print src_file
print dst_file
filewrite.write(file + " " + "needs to have dir and file moved Moved!" + '\n')
else:
src_file = os.path.join(dir_src, file)
dst_file = os.path.join(dir_dst, file)
print src_file
print dst_file
shutil.move(src_file, dst_file)
filewrite.write(os.path.dirname(file) + '\n')
filewrite.write(file + " " + "needs to have file moved Moved!" + '\n')
filewrite.close()
Looks like you're only moving files, without doing anything about the folders. So if you try to move
C:\Users\Jeremy\Downloads\anime\pokemon.avi
to
H:\Movies from download folder\anime\pokemon.avi
it will fail because there's no anime directory on H:\ yet.
Before iterating through files, iterate through dirs to ensure that the directory exists at your destination, creating it if necessary.
for root, dirs, files in os.walk(dir_src):
for dir in dirs:
dest_dir = os.path.join(dir_dst, dir)
if not os.path.isdir(dest_dir):
os.mkdir(dest_dir)
for file in files:
#rest of code goes here as usual...
As these are MS Windows paths use forward slashes instead and declare path as a string literal; e.g.
dir_dst = r"H:/Movies from download folder/"
This is probably a simple question, but I'm brand new to python and programming in general.
I'm working on a simple program to copy/move .mp3 files from on location to another while mirroring the directory structure of the source location. What I have so far works, however it also creates new folders in the destination location even if the source folder contained no mp3 files. I only want to create the new directories if the source contains .mp3s, otherwise it could lead to a bunch of empty folders in the destination.
Here is what I have so far:
import os
import shutil #Used for copying files
##CONFIG
source_dir = "C:\Users\username\Desktop\iTunes\\" #set the root folder that you want to scan and move files from. This script will scan recursively.
destPath = "C:\Users\username\Desktop\converted From iTunes" #set the destination root that you want to move files to. Any non-existing sub directories will be created.
ext = ".mp3" #set the type of file you want to search for.
count = 0 #initialize counter variable to count number of files moved
##
##FIND FILES
for dirName, subdirList, fileList in os.walk(source_dir):
#set the path for the destination folder(s)
dest = destPath + dirName.replace(source_dir, '\\')
#if the source directory doesn't exist in the destination folder
#then create a new folder
if not os.path.isdir(dest):
os.mkdir(dest)
print('Directory created at: ' + dest)
for fname in fileList:
if fname.endswith(ext) :
#determine source & new file locations
oldLoc = dirName + '\\' + fname
newLoc = dest + '\\' + fname
if os.path.isfile(newLoc): # check to see if the file already exists. If it does print out a message saying so.
print ('file "' + newLoc + fname + '" already exists')
if not os.path.isfile(newLoc): #if the file doesnt exist then copy it and print out confirmation that is was copied/moved
try:
shutil.move(oldLoc, newLoc)
print('File ' + fname + ' copied.')
count = count + 1
except IOError:
print('There was an error copying the file: "' + fname + '"')
print 'error'
print "\n"
print str(count) + " files were moved."
print "\n"
so if the folder structure is something like:
root->
band 1->
album name->
song.m4a,
song2.m4a
right now it will create all those folders in the destination driectory, even though there are no .mp3s to copy.....
Any help is appreciated!
I think I would create my own wrapper around copy for this sort of thing:
def fcopy(src,dest):
"""
Copy file from source to dest. dest can include an absolute or relative path
If the path doesn't exist, it gets created
"""
dest_dir = os.path.dirname(dest)
try:
os.makedirs(dest_dir)
except os.error as e:
pass #Assume it exists. This could fail if you don't have permissions, etc...
shutil.copy(src,dest)
Now you can just walk the tree calling this function on any .mp3 file.
The simplest thing to do I can think of for your existing code would be to just make it skip over any folders that don't have any .mp3 files in them. This can easily be done by adding the following items and if statement to the top of your loop. The itertools.ifilter() and fnmatch.fnmatch() functions can be used together to simplify checking for files with the proper extension.
from itertools import ifilter
from fnmatch import fnmatch
ext = '.mp3'
fnPattern = '*'+ext
for dirName, subdirList, fileList in os.walk(source_dir):
if not any(ifilter(lambda fname: fnmatch(fname, fnPattern), fileList)):
print ' skipping "{}"'.format(dirName)
continue
...
You will also have to change the os.mkdir(dest) to os.makedirs(dest) in the code further down to ensure that any subdirectories skipped by earlier iterations get created when there's a need to copy files to a corresponding subbranch of the destination directory.
You could optimize things a bit by creating and saving a possibly empty iterator of matching files that have the extension, and then use it again later to to determine what files to copy:
from itertools import ifilter
from fnmatch import fnmatch
ext = '.mp3'
fnPattern = '*'+ext
for dirName, subdirList, fileList in os.walk(source_dir):
# generate list of files in directory with desired extension
matches = ifilter(lambda fname: fnmatch(fname, fnPattern), fileList)
# skip subdirectory if it does not contain any files of interest
if not matches:
continue
...
... create destination directory with os.makedirs()
...
# copy each file to destination directory
for fname in matches:
... copy file
Would shutils.copytree not do what you want in fewer lines?
Currently, I have the following code...
file_name = content.split('=')[1].replace('"', '') #file, gotten previously
fileName = "/" + self.feed + "/" + self.address + "/" + file_name #add folders
output = open(file_name, 'wb')
output.write(url.read())
output.close()
My goal is to have python write the file (under file_name) to a file in the "address" folder in the "feed" folder in the current directory (IE, where the python script is saved)
I've looked into the os module, but I don't want to change my current directory and these directories do not already exist.
First, I'm not 100% confident I understand the question, so let me state my assumption:
1) You want to write to a file in a directory that doesn't exist yet.
2) The path is relative (to the current directory).
3) You don't want to change the current directory.
So, given that:
Check out these two functions: os.makedirs and os.path.join. Since you want to specify a relative path (with respect to the current directory) you don't want to add the initial "/".
dir_path = os.path.join(self.feed, self.address) # will return 'feed/address'
os.makedirs(dir_path) # create directory [current_path]/feed/address
output = open(os.path.join(dir_path, file_name), 'wb')
This will create the file feed/address/file.txt in the same directory as the current script:
import os
file_name = 'file.txt'
script_dir = os.path.dirname(os.path.abspath(__file__))
dest_dir = os.path.join(script_dir, 'feed', 'address')
try:
os.makedirs(dest_dir)
except OSError:
pass # already exists
path = os.path.join(dest_dir, file_name)
with open(path, 'wb') as stream:
stream.write('foo\n')
Commands like os.mkdir don't actually require that you make the folder in your current directory; you can put a relative or absolute path.
os.mkdir('../new_dir')
os.mkdir('/home/you/Desktop/stuff')
I don't know of a way to both recursively create the folders and open the file besides writing such a function yourself - here's approximately the code in-line. os.makedirs will get you most of the way there; using the same mysterious self object you haven't shown us:
dir = "/" + self.feed + "/" + self.address + "/"
os.makedirs(dir)
output = open(os.path.join(dir, file_name), 'wb')