I'm trying to add a list of image locations into a list for each parent folder for image comparison. However when I pull the list of images from the folders with OS and then check them from the list with os.path.exists, some of the paths apparently do not exist, even though the files do exist when i manually check.
How do I fix this or work out why it is now saying the file paths do not exist? I have already tried to strip out white spaces
import os
directory = '$$$'
listFiles = os.listdir(directory)
tester = []
for entry in listFiles:
fullpath = os.path.join(directory, entry)
test = fullpath
listFiles = os.listdir(fullpath)
print(listFiles)
for n in listFiles:
fullpath = os.path.join(test,n)
fullpath = fullpath.strip()
tester.append(fullpath)
for n in range (len(tester)):
print(tester[n].strip())
print(os.path.exists(tester[n]))
break
It looks like you're trying to append all the files in the directory to the list tester, including subdirectories. If you're using os.listdir(), you need to then recursively run that on all the entries which are also dirs, perhaps by using is_dir(). But instead of writing all that by hand, just use os.walk() which already recurses folders:
directory = '$$$'
tester = []
for parent, folders, files in os.walk(directory):
for file in files:
tester.append(os.path.join(parent, file))
print(tester)
That can be shortened to:
directory = '$$$'
tester = [os.path.join(parent, file)
for parent, folders, files in os.walk(directory)
for file in files]
Note: Spaces in filenames or folders don't matter when using such functions. If you were trying to execute something on command line, then spaces matter and require quoting or escaping.
.strip() doesn't remove whitespaces everywhere, just and the beginning and the end of the string.
Try to rename your directory content removing spaces everywhere, or it's better to rename it to a short string (ex. a number)
dir = '....'
content = os.listdir(dir)
for i in range(len(content)):
path = os.path.join(dir, content[i])
# if file to number : new_path = path + '/' + i + '.' + os.path.splitext(content[i])[-1]
new_path = path.replace(' ', '')
print(path, new_path)
os.rename(path, new_path)
Related
I have question regarding moving one file in each sub directories to other new sub directories. So for example if I have directory as it shown in the image
And from that, I want to pick only the first file in each sub directories then move it to another new sub directories with the same name as you can see from the image. And this is my expected result
I have tried using os.walk to select the first file of each sub directories, but I still don't know how to move it to another sub directories with the same name
path = './test/'
new_path = './x/'
n = 1
fext = ".png"
for dirpath, dirnames, filenames in os.walk(path):
for filename in [f for f in filenames if f.endswith(fext)][:n]:
print(filename) #this only print the file name in each sub dir
The expected result can be seen in the image above
You are almost there :)
All you need is to have both full path of file: an old path (existing file) and a new path (where you want to move it).
As it mentioned in this post you can move files in different ways in Python. You can use "os.rename" or "shutil.move".
Here is a full tested code-sample:
import os, shutil
path = './test/'
new_path = './x/'
n = 1
fext = ".png"
for dirpath, dirnames, filenames in os.walk(path):
for filename in [f for f in filenames if f.endswith(fext)][:n]:
print(filename) #this only print the file name in each sub dir
filenameFull = os.path.join(dirpath, filename)
new_filenameFull = os.path.join(new_path, filename)
# if new directory doesn't exist - you create it recursively
if not os.path.exists(new_path):
os.makedirs(new_path)
# Use "os.rename"
#os.rename(filenameFull, new_filenameFull)
# or use "shutil.move"
shutil.move(filenameFull, new_filenameFull)
I've tried to write some code which will rename some files in a folder - essentially, they're listed as xxx_(a).bmp whereas they need to be xxx_a.bmp, where a runs from 1 to 2000.
I've used the inbuilt os.rename function to essentially swap them inside of a loop to get the right numbers, but this gives me FileNotFoundError [WinError2] the system cannot find the file specified Z:/AAA/BBB/xxx_(1).bmp' -> 'Z:/AAA/BBB/xxx_1.bmp'.
I've included the code I've written below if anyone could point me in the right direction. I've checked that I'm working in the right directory and it gives me the directory I'm expecting so I'm not sure why it can't find the files.
import os
n = 2000
folder = r"Z:/AAA/BBB/"
os.chdir(folder)
saved_path = os.getcwd()
print("CWD is" + saved_path)
for i in range(1,n):
old_file = os.path.join(folder, "xxx_(" + str(i) + ").bmp")
new_file = os.path.join(folder, "xxx_" +str(i)+ ".bmp")
os.rename(old_file, new_file)
print('renamed files')
The problem is os.rename doesn't create a new directory if the new name is a filename in a directory that does not currently exist.
In order to create the directory first, you can do the following in Python3:
os.makedirs(dirname, exist_ok=True)
In this case dirname can contain created or not-yet-created subdirectories.
As an alternative, one may use os.renames, which handles new and intermediate directories.
Try iterating files inside the directory and processing the files that meet your criteria.
from pathlib import Path
import re
folder = Path("Z:/AAA/BBB/")
for f in folder.iterdir():
if '(' in f.name:
new_name = f.stem.replace('(', '').replace(')', '')
# using regex
# new_name = re.sub('\(([^)]+)\)', r'\1', f.stem)
extension = f.suffix
new_path = f.with_name(new_name + extension)
f.rename(new_path)
This is probably a simple question, but I'm brand new to python and programming in general.
I'm working on a simple program to copy/move .mp3 files from on location to another while mirroring the directory structure of the source location. What I have so far works, however it also creates new folders in the destination location even if the source folder contained no mp3 files. I only want to create the new directories if the source contains .mp3s, otherwise it could lead to a bunch of empty folders in the destination.
Here is what I have so far:
import os
import shutil #Used for copying files
##CONFIG
source_dir = "C:\Users\username\Desktop\iTunes\\" #set the root folder that you want to scan and move files from. This script will scan recursively.
destPath = "C:\Users\username\Desktop\converted From iTunes" #set the destination root that you want to move files to. Any non-existing sub directories will be created.
ext = ".mp3" #set the type of file you want to search for.
count = 0 #initialize counter variable to count number of files moved
##
##FIND FILES
for dirName, subdirList, fileList in os.walk(source_dir):
#set the path for the destination folder(s)
dest = destPath + dirName.replace(source_dir, '\\')
#if the source directory doesn't exist in the destination folder
#then create a new folder
if not os.path.isdir(dest):
os.mkdir(dest)
print('Directory created at: ' + dest)
for fname in fileList:
if fname.endswith(ext) :
#determine source & new file locations
oldLoc = dirName + '\\' + fname
newLoc = dest + '\\' + fname
if os.path.isfile(newLoc): # check to see if the file already exists. If it does print out a message saying so.
print ('file "' + newLoc + fname + '" already exists')
if not os.path.isfile(newLoc): #if the file doesnt exist then copy it and print out confirmation that is was copied/moved
try:
shutil.move(oldLoc, newLoc)
print('File ' + fname + ' copied.')
count = count + 1
except IOError:
print('There was an error copying the file: "' + fname + '"')
print 'error'
print "\n"
print str(count) + " files were moved."
print "\n"
so if the folder structure is something like:
root->
band 1->
album name->
song.m4a,
song2.m4a
right now it will create all those folders in the destination driectory, even though there are no .mp3s to copy.....
Any help is appreciated!
I think I would create my own wrapper around copy for this sort of thing:
def fcopy(src,dest):
"""
Copy file from source to dest. dest can include an absolute or relative path
If the path doesn't exist, it gets created
"""
dest_dir = os.path.dirname(dest)
try:
os.makedirs(dest_dir)
except os.error as e:
pass #Assume it exists. This could fail if you don't have permissions, etc...
shutil.copy(src,dest)
Now you can just walk the tree calling this function on any .mp3 file.
The simplest thing to do I can think of for your existing code would be to just make it skip over any folders that don't have any .mp3 files in them. This can easily be done by adding the following items and if statement to the top of your loop. The itertools.ifilter() and fnmatch.fnmatch() functions can be used together to simplify checking for files with the proper extension.
from itertools import ifilter
from fnmatch import fnmatch
ext = '.mp3'
fnPattern = '*'+ext
for dirName, subdirList, fileList in os.walk(source_dir):
if not any(ifilter(lambda fname: fnmatch(fname, fnPattern), fileList)):
print ' skipping "{}"'.format(dirName)
continue
...
You will also have to change the os.mkdir(dest) to os.makedirs(dest) in the code further down to ensure that any subdirectories skipped by earlier iterations get created when there's a need to copy files to a corresponding subbranch of the destination directory.
You could optimize things a bit by creating and saving a possibly empty iterator of matching files that have the extension, and then use it again later to to determine what files to copy:
from itertools import ifilter
from fnmatch import fnmatch
ext = '.mp3'
fnPattern = '*'+ext
for dirName, subdirList, fileList in os.walk(source_dir):
# generate list of files in directory with desired extension
matches = ifilter(lambda fname: fnmatch(fname, fnPattern), fileList)
# skip subdirectory if it does not contain any files of interest
if not matches:
continue
...
... create destination directory with os.makedirs()
...
# copy each file to destination directory
for fname in matches:
... copy file
Would shutils.copytree not do what you want in fewer lines?
I have a directory structure that resembles the following:
Dir1
Dir2
Dir3
Dir4
L SubDir4.1
L SubDir4.2
L SubDir4.3
I want to generate a list of files (with full paths) that include all the contents of Dirs1-3, but only SubDir4.2 inside Dir4. The code I have so far is
import fnmatch
import os
for root, dirs, files in os.walk( '.' )
if 'Dir4' in dirs:
if not 'SubDir4.2' in 'Dir4':
dirs.remove( 'Dir4' )
for file in files
print os.path.join( root, file )
My problem is that the part where I attempt to exclude any file that does not have SubDir4.2 in it's path is excluding everything in Dir4, including the things I would like to remain. How should I amend that above to to do what I desire?
Update 1: I should add that there are a lot of directories below Dir4 so manually listing them in an excludes list isn't a practical option. I'd like to be able to specify SubDur4.2 as the only subdirectory within Dir4 to be read.
Update 2: For reason outside of my control, I only have access to Python version 2.4.3.
There are a few typos in your snippet. I propose this:
import os
def any_p(iterable):
for element in iterable:
if element:
return True
return False
include_dirs = ['Dir4/SubDir4.2', 'Dir1/SubDir4.2', 'Dir3', 'Dir2'] # List all your included folder names in that
for root, dirs, files in os.walk( '.' ):
dirs[:] = [d for d in dirs if any_p(d in os.path.join(root, q_inc) for q_inc in include_dirs)]
for file in files:
print file
EDIT: According to comments, I have changed that so this is include list, instead of an exclude one.
EDIT2: Added a any_p (any() equivalent function for python version < 2.5)
EDIT3bis: if you have other subfolders with the same name 'SubDir4.2' in other folders, you can use the following to specify the location:
include_dirs = ['Dir4/SubDir4.2', 'Dir1/SubDir4.2']
Assuming you have a Dir1/SubDir4.2.
If they are a lot of those, then you may want to refine this approach with fnmatch, or probably a regex query.
I altered mstud's solution to give you what you are looking for:
import os;
for root, dirs, files in os.walk('.'):
# Split the root into its path parts
tmp = root.split(os.path.sep)
# If the lenth of the path is long enough to be your path AND
# The second to last part of the path is Dir4 AND
# The last part of the path is SubDir4.2 THEN
# Stop processing this pass.
if (len(tmp) > 2) and (tmp[-2] == 'Dir4') and (tmp[-1] != 'SubDir4.2'):
continue
# If we aren't in Dir4, print the file paths.
if tmp[-1] != 'Dir4':
for file in files:
print os.path.join(root, file)
In short, the first "if" skips the printing of any directory contents under Dir4 that aren't SubDir4.2. The second "if" skips the printing of the contents of the Dir4 directory.
for root, dirs, files in os.walk('.'):
tmp = root.split(os.path.sep)
if len(tmp)>2 and tmp[-2]=="Dir4" and tmp[-1]=="SubDir4.2":
continue
for file in files:
print os.path.join(root, file)
I am trying to list directories and files (recursivley) in a directory with python:
./rootdir
./file1.html
./subdir1
./file2.html
./file3.html
./subdir2
./file4.html
Now I can list the directories and files just fine (borrowed it from here). But I would like to list it in the following format and ORDER (which is very important for what I am doing.
/rootdir/
/rootdir/file1.html
/rootdir/subdir1/
/rootdir/subdir1/file2.html
/rootdir/subdir1/file3.html
/rootdir/subdir2/
/rootdir/file4.html
I don't care how it gets done. If I walk the directory and then organize it or get everything in order. Either way, thanks in advance!
EDIT: Added code below.
# list books
import os
import sys
lstFiles = []
rootdir = "/srv/http/example/www/static/dev/library/books"
# Append the directories and files to a list
for path, dirs, files in os.walk(rootdir):
#lstFiles.append(path + "/")
lstFiles.append(path)
for file in files:
lstFiles.append(os.path.join(path, file))
# Open the file for writing
f = open("sidebar.html", "w")
f.write("<ul>")
for item in lstFiles:
splitfile = os.path.split(item)
webpyPath = splitfile[0].replace("/srv/http/example/www", "")
itemName = splitfile[1]
if item.endswith("/"):
f.write('<li>' + itemName + '</li>\n')
else:
f.write('<li>' + itemName + '</li>\n')
f.write("</ul>")
f.close()
Try the following:
for path, dirs, files in os.walk("."):
print path
for file in files:
print os.path.join(path, file)
You do not need to print entries from dirs because each directory will be visited as you walk the path, so you will print it later with print path.