I created a simple Python script to move files from a folder/directory to a newly created folder within the same path. The idea is to move files that were modified more than five days ago. I'm having issues where it is moving everything in the base path, including other subfolders. Is there a way to just move files, and not folders?
I tried the .endswith function from the os.path module, but no luck. I believe I am missing something revolving around a wildcard.
This is my code:
if not os.path.exists(new_path):
os.mkdir(new_path)
print('Successfully created the directory %s' % new_path)
else:
print('The directory %s already exists' % new_path)
for f in os.listdir(dir_path):
path_and_file = os.path.join(dir_path,f)
if int((datetime.datetime.fromtimestamp(os.path.getmtime(f)).strftime("%Y-%m-%d")).replace("-","")) < int(threshold_date.replace("-","")):
destpath_and_file = os.path.join(new_path, f)
shutil.move(path_and_file,destpath_and_file)
The code works, moving everything within the base folder. However, it is moving subfolders too. I would like to move files only. (For example, only move .xls/.xlsx files.)
Thank you very much.
Check that f is a file with os.path.isfile().
for f in os.listdir(dir_path):
if os.path.isfile(f):
...
Related
I had 120 files in my source folder which I need to move to a new directory (destination). The destination is made in the function I wrote, based on the string in the filename. For example, here is the function I used.
path ='/path/to/source'
dropbox='/path/to/dropbox'
files = = [os.path.join(path,i).split('/')[-1] for i in os.listdir(path) if i.startswith("SSE")]
sam_lis =list()
for sam in files:
sam_list =sam.split('_')[5]
sam_lis.append(sam_list)
sam_lis =pd.unique(sam_lis).tolist()
# Using the above list
ID = sam_lis
def filemover(ID,files,dropbox):
"""
Function to move files from the common place to the destination folder
"""
for samples in ID:
for fs in files:
if samples in fs:
desination = dropbox + "/"+ samples + "/raw/"
if not os.path.isdir(desination):
os.makedirs(desination)
for rawfiles in fnmatch.filter(files, pat="*"):
if samples in rawfiles:
shutil.move(os.path.join(path,rawfiles),
os.path.join(desination,rawfiles))
In the function, I am creating the destination folders, based on the ID's derived from the files list. When I tried to run this for the first time it threw me FILE NOT exists error.
However, later when I checked the source all files starting with SSE were missing. In the beginning, the files were there. I want some insights here;
Whether or not os.shutil.move moves the files to somewhere like a temp folder instead of destination folder?
whether or not the os.shutil.move deletes the files from the source in any circumstance?
Is there any way I can test my script to find the potential reasons for missing files?
Any help or suggestions are much appreciated?
It is late but people don't understand the op's question. If you move a file into a non-existing folder, the file seems to become a compressed binary and get lost forever. It has happened to me twice, once in git bash and the other time using shutil.move in Python. I remember the python happens when your shutil.move destination points to a folder instead of to a copy of the full file path.
For example, if you run the code below, a similar situation to what the op described will happen:
src_folder = r'C:/Users/name'
dst_folder = r'C:/Users/name/data_images'
file_names = glob.glob(r'C:/Users/name/*.jpg')
for file in file_names:
file_name = os.path.basename(file)
shutil.move(os.path.join(src_folder, file_name), dst_folder)
Note that dst_folder in the else block is just a folder. It should be dst_folder + file_name. This will cause what the Op described in his question. I find something similar on the link here with a more detailed explanation of what went wrong: File moving mistake with Python
shutil.move does not delete your files, if for any reason your files failed to move to a given location, check the directory where your code is stored, for a '+' folder your files are most likely stored there.
def copy_excel():
srcpath = "C:\\Aloha" #where the excel files are located
srcfiles = os.listdir(srcpath) #sets srcfiles as list of file names in the source path folder
destpath = "C:\\" #destination where the folders will be created
destdir = list(set([filename[19:22] for filename in srcfiles])) #extract three digits from filename to use for folder creation (i.e 505, 508, 517,...)
#function to handle folder creation
def create(dirname, destpath):
full_path = os.path.join(destpath, dirname)
if os.path.exists(full_path):
pass
else:
os.mkdir(full_path)
return full_path
#function to handle moving files to appropriate folders
def move(filename, dirpath):
shutil.move(os.path.join(srcpath, filename), dirpath)
#creates the folders with three digits as folder name by calling the create function above
targets = [(folder, create(folder, destpath)) for folder in destdir]
#handles moving files to appropriate folders if the three digits in file name matches the created folder
for dirname, full_path in targets:
for filename in srcfiles:
if dirname == filename[19:22]:
move(filename, full_path)
else:
pass
I am somewhat new to Python so please bear with me! I was able to find this code block on this site and tailored it to my specific use case. The code works well for creating the specified folders and dropping the files into the corresponding folders. However, when I run the code again for new files that are dropped into the "C:\\Aloha" I get a Error 183: Cannot create a file when that file already exists. In this case, the folder already exists because it was previously created when the script was run the first time.
The code errors out when targets attempts to create folders that already exist. My question is, what is the logic to handle folders that already exists and to ignore the error and just move the files to the corresponding folders? The script should only create folders if they are not already there.
Any help would be greatly appreciated! I have attempted try/except and nesting if/else statements as well as os.path.isdir(path) to check to see if the directory exists but I haven't had any luck. I apologize for any of the comments that are wrong, I am still learning Python logic as I build this script out.
You could use os.makedirs which not only will create cascading directories such that for instance C:\foo\bar\qux will create foo, bar and qux in case they do not exist, but you can also set the exist_ok=True such that no error is thrown if the directory exists.
So:
os.makedirs(full_path,exist_ok=True)
In case you want to throw an error or stop the processing if the directory exists, you can use os.path.exists(full_path) before mkdir.
Another option I just came by...
try:
os.mkdir(path)
except FileExistsError:
pass
I have written the Python script below to find the last modified date of a file:
for root, dirs, files in os.walk(folder_path):
for f in files:
last_modified = datetime.fromtimestamp(os.path.getmtime(f)).day
#do something
This code works fine for normal files like .txt files. However, if it encounters files with extensions like .tar.gz or .run, it throws an error saying "No such file or directory" when finding last_modified.
* EDIT *
I realized that, if I change the code as below, it works fine.
for f in files:
name = os.path.join(root, f)
last_modified = datetime.fromtimestamp(os.path.getmtime(name)).day
#do something
But why does this happen?
os.walk return filenames relative to the folder it is walking. E.g. if it is walking a directory called foo with a file called bar, it will return bar, not foo/bar. However, since your script (presumably) is not in the folder os is walking, it needs an absolute path, or a path relative to itself. Therefore, you must join the folder path to the filename, so that python can find the file.
As the docs say:
Note that the names in the lists contain no path components. To get a full path (which begins with top) to a file or directory in dirpath, do os.path.join(dirpath, name).
Why it happens to work with the text files I'm not sure. Do you have text files with the same name in the folder containing your script?
I have this script, which I have no doubt is flawed:
import fnmatch, os, sys
def findit (rootdir, find, pattern):
for folder, dirs, files in os.walk(rootdir):
print (folder)
for filename in fnmatch.filter(files,pattern):
with open(filename) as f:
s = f.read()
f.close()
if find in s :
print(filename)
findit(sys.argv[1], sys.argv[2], sys.argv[3])
when I run it I get Errno2, no such file or directory. BUT the file exists. For instance if I execute it by going: findit.py c:\python "folder" *.py it will work just fine, listing all the *.py files which contain the word "folder". BUT if I go findit.py c:\php\projects1 "include" *.php
as an example I get [Errno2] no such file or directory: 'About.php' (for example). But About.php exists. I don't understand what it's doing, or what I'm doing wrong.
If you look at any of the examples for os.walk, you'll see that they all do os.path.join(root, name). You need to do that too.
Why? Quoting from the docs:
filenames is a list of the names of the non-directory files in dirpath. Note that the names in the lists contain no path components. To get a full path (which begins with top) to a file or directory in dirpath, do os.path.join(dirpath, name).
If you just use the filename as a path, it's going to look for a file of the same name in the current working directory. If there's no such file, you'll get a FileNotFoundError. If there is such a file, you'll open and read the wrong file. Only if you happen to be looking inside the current working directory will it work.
There's also another major problem in your code: os.walk walks a directory tree recursively, finding all files in the given top directory, or any subdirectory of top, or any subdirectory of… and so on, yielding once for each directory. But you're not doing anything useful with that (except printing out the folders). Instead, you wait until it finishes, and then use the files from whichever directory it happened to reach last.
If you just want to get a flat listing of the files directly in a directory, use os.listdir, not os.walk. (Or maybe use glob.glob instead of explicitly listing everything then filtering with fnmatch.)
On the other hand, if you want to walk the tree, you have to move your second for loop inside the first one.
You've also got a minor problem: You call f.close() inside a with open(…) as f:, which leads to f being closed twice. This is guaranteed to be completely harmless (at least in 2.5+, including 3.x), but it's still a bad idea.
Putting it together, here's a working version of your code:
def findit (rootdir, find, pattern):
for folder, dirs, files in os.walk(rootdir):
print (folder)
for filename in fnmatch.filter(files,pattern):
pathname = os.path.join(folder, filename)
with open(pathname) as f:
s = f.read()
if find in s:
print(pathname)
You are using a relative filename. But your current directory does not contain the file. And you don't want to search there anyway. Use os.path.join(folder, filename) to make an absolute path.
This is my first time using python and I keep running into error 183. The script I created searches the network for all '.py' files and copies them to my backup drive. Please don't laugh at my script as this is my first.
Any clue to what I am doing wrong in the script?
import os
import shutil
import datetime
today = datetime.date.today()
rundate = today.strftime("%Y%m%d")
for root,dirr,filename in os.walk("p:\\"):
for files in filename:
if files.endswith(".py"):
sDir = os.path.join(root, files)
dDir = "B:\\Scripts\\20120124"
modname = rundate + '_' + files
shutil.copy(sDir, dDir)
os.rename(os.path.join(dDir, files), os.path.join(dDir, modname))
print "Renamed %s to %s in %s" % (files, modname, dDir)
I'm guessing you are running the script on windows. According to the list of windows error codes error 183 is ERROR_ALREADY_EXISTS
So I would guess the script is failing because you're attempting to rename a file over an existing file.
Perhaps you are running the script more than once per day? That would result in all the destination files already being there, so the rename is failing when the script is run additional times.
If you specifically want to overwrite the files, then you should probably delete them using os.unlink first.
Given the fact that error 183 is [Error 183] Cannot create a file when that file already exists, you're most likely finding 2 files with the same name in the os.walk() call and after the first one is renamed successfully, the second one will fail to be renamed to the same name so you'll get a file already exists error.
I suggest a try/except around the os.rename() call to treat this case (append a digit after the name or something).
[Yes, i know it's been 7 years since this question was asked but if I got here from a google search maybe others are reaching it too and this answer might help.]
I just encounter the same issue, when you trying to rename a folder with a folder that existed in the same directory has the same name, Python will raise an error.
If you trying to do that in Windows Explorer, it will ask you if you want to merge these two folders. however, Python doesn't have this feature.
Below is my codes to achieve the goal of rename a folder while a same name folder already exist, which is actually merging folders.
import os, shutil
DEST = 'D:/dest/'
SRC = 'D:/src/'
for filename in os.listdir(SRC): # move files from SRC to DEST folder.
try:
shutil.move(SRC + filename, DEST)
# In case the file you're trying to move already has a copy in DEST folder.
except shutil.Error: # shutil.Error: Destination path 'D:/DEST/xxxx.xxx' already exists
pass
# Now delete the SRC folder.
# To delete a folder, you have to empty its files first.
if os.path.exists(SRC):
for i in os.listdir(SRC):
os.remove(os.path.join(SRC, i))
# delete the empty folder
os.rmdir(SRC)