I am trying to locate all my files with externsion mp4 in a folder(and subfolder) and copy them in another dir. I manage to find all files with the extension mp4 however I didnt manage to keep the dir of those files. My code is the following:
import os
from shutil import copyfile
path = "videos/"
for root, dirs, files in os.walk(path):
for name in files:
if name.endswith((".mp4", ".mp4")):
print(name)
# copyfile(src, dst)
I want to find the path of the name (corresponding to my vids). How can I do so?
Use os.path.join()
import os
from shutil import copyfile
path = "videos/"
for root, dirs, files in os.walk(path):
for name in files:
if name.endswith((".mp4", ".mp4")):
print(os.path.join(root, name))
# copyfile(src, dst)
Although it is considered better to use absolute paths but you can use os.path.relpath if you want relative path. From the os.path.relpath documentation
os.path.relpath(path[, start])
Return a relative filepath to path either from the current directory or from an optional start directory. This is a path computation: the filesystem is not accessed to confirm the existence or nature of path or start.
start defaults to os.curdir.
Availability: Windows, Unix.
New in version 2.6.
Why not just use glob:
import glob, shutil
for file in glob.iglob('/foo/*.mp4'):
shutil.copy2(file, '/bar/{0}'.format(file))
From the documentation on os.walk:
dirpath is a string, the path to the directory. dirnames is a list of the names of the subdirectories in dirpath (excluding '.' and '..'). filenames is a list of the names of the non-directory files in dirpath. Note that the names in the lists contain no path components. To get a full path (which begins with top) to a file or directory in dirpath, do os.path.join(dirpath, name).
So your code should look like this:
import os
from shutil import copyfile
path = "videos/"
for root, dirs, files in os.walk(path):
for name in files:
if name.endswith((".mp4", ".mp4")):
print(name)
src = os.path.join(root, name)
copyfile(src, dst)
Related
I am trying to collect all files with all sub-directories and move to another directory
Code used
#collects all mp3 files from folders to a new folder
import os
from pathlib import Path
import shutil
#run once
path = os.getcwd()
os.mkdir("empetrishki")
empetrishki = path + "/empetrishki" #destination dir
print(path)
print(empetrishki)
#recursive collection
for root, dirs, files in os.walk(path, topdown=True, onerror=None, followlinks=True):
for name in files:
filePath = Path(name)
if filePath.suffix.lower() == ".mp3":
print(filePath)
os.path.join
filePath.rename(empetrishki.joinpath(filePath))
I have trouble with the last line of moving files: filePath.rename() nor shutil.move nor joinpath() have worked for me. Maybe that's because I am trying to change the element in the tuple - the output from os.walk
Similar code works with os.scandir but this would collect files only in the current directory
How can I fix that, thanks!
If you use pathlib.Path(name) that doesn't mean that something exists called name. Hence, you do need to be careful that you have a full path, or relative path, and you need to make sure to resolve those. In particular I am noting that you don't change your working directory and have a line like this:
filePath = Path(name)
This means that while you may be walking down the directory, your working directory may not be changing. You should make your path from the root and the name, it is also a good idea to resolve so that the full path is known.
filePath = Path(root).joinpath(name).resolve()
You can also place the Path(root) outside the inner loop as well. Now you have an absolute path from '/home/' to the filename. Hence, you should be able to rename with .rename(), like:
filePath.rename(x.parent.joinpath(newname))
#Or to another directory
filePath.rename(other_dir.joinpath(newname))
All together:
from pathlib import os, Path
empetrishki = Path.cwd().joinpath("empetrishki").resolve()
for root, dirs, files in os.walk(path, topdown=True, onerror=None, followlinks=True):
root = Path(root).resolve()
for name in files:
file = root.joinpath(name)
if file.suffix.lower() == ".mp3":
file.rename(empetrishki.joinpath(file.name))
for root, dirs, files in os.walk(path, topdown=True, onerror=None, followlinks=True):
if root == empetrishki:
continue # skip the destination dir
for name in files:
basename, extension = os.path.splitext(name)
if extension.lower() == ".mp3":
oldpath = os.path.join(root, name)
newpath = os.path.join(empetrishki, name)
print(oldpath)
shutil.move(oldpath, newpath)
This is what I suggest. Your code is running on the current directory, and the file is at the path os.path.join(root, name) and you need to provide such path to your move function.
Besides, I would also suggest to use os.path.splitext for extracting the file extension. More pythonic. And also you might want to skip scanning your target directory.
The loop is working but once I put the if statements in it only prints I am a dir
If the if statements are not there I am able to print the dirpath, dirname, filename to the console
I am trying to list all the file names in a directory and get the MD5 sum.
from os import walk
import hashlib
import os
path = "/home/Desktop/myfile"
for (dirpath, dirname, filename) in walk(path):
if os.path.isdir(dirpath):
print("I am a dir")
if os.path.isfile(dirpath):
print(filename, hashlib.md5(open(filename, 'rb').read()).digest())
You're only checking dirpath. What you have as dirname and filename are actually collections of directory names and files under dirpath. Taken from the python docs, and modified slightly, as their example removes the files:
import os
for root, dirs, files in os.walk(top):
for name in files:
print(os.path.join(root, name))
for name in dirs:
print(os.path.join(root, name))
Will print the list of of directories and files under top and then will recurse down the directories in under top and print the folders and directories there.
From the Python documentation about os.walk:
https://docs.python.org/2/library/os.html
dirpath is a string, the path to the directory. dirnames is a list of
the names of the subdirectories in dirpath (excluding '.' and '..').
filenames is a list of the names of the non-directory files in
dirpath.
With os.path.isfile(dirpath) you are checking whether dirpath is a file, which is never the case. Try changing the code to:
full_filename = os.path.join(dirpath, filename)
if os.path.isfile(full_filename):
print(full_filename, hashlib.md5(open(full_filename, 'rb').read()).digest())
I have a folder (with other subfolders) from which i would like to copy only the .js files to another existing folder (this also has subfolders with the same folder structure as the first one, except this one has only the folders, so no file)
How can i do that with python? I tried shutil.copytree but it fails because some folders already exists.
use os.path.splitext or glob.iglob
glob.iglob(pathname)
Return an iterator which yields the same values as glob() without
actually storing them all simultaneously.
I propose a solution with os.path.splitext, walking with os.walk. I make use of os.path.relpath to find relative path in the duplicate tree.
source_dir is your source uppermost source folder, dest_dir your uppermost destination folder.
import os, shutil, glob
source_dir = "F:\CS\PyA"
dest_dir = "F:\CS\PyB"
for root, dirnames, filenames in os.walk(source_dir):
for file in filenames:
(shortname, extension) = os.path.splitext(file)
if extension == ".txt" :
shutil.copy2(os.path.join(root,file), os.path.join(dest_dir,
os.path.relpath(os.path.join(root,file),source_dir)))
from glob import glob
from shutil import copy
import os
def copyJS(src, dst):
listing = glob(src + '/*')
for f in listing:
if os.path.isdir(f):
lastToken = f.split('/')[-1]
copyJS(src+'/' + lastToken, dst+ '/' + lastToken)
elif f[-3:] == '.js':
copy(f, dst)
I would like to find all the files in a directory and all sub-directories.
code used:
import os
import sys
path = "C:\\"
dirs = os.listdir(path)
filename = "C.txt"
FILE = open(filename, "w")
FILE.write(str(dirs))
FILE.close()
print dirs
The problem is - this code only lists files in directories, not sub-directories. What do I need to change in order to also list files in subdirectories?
To traverse a directory tree you want to use os.walk() for this.
Here's an example to get you started:
import os
searchdir = r'C:\root_dir' # traversal starts in this directory (the root)
for root, dirs, files in os.walk(searchdir):
for name in files:
(base, ext) = os.path.splitext(name) # split base and extension
print base, ext
which would give you access to the file names and the components.
You'll find the functions in the os and os.path module to be of great use for this sort of work.
This function will help you: os.path.walk() http://docs.python.org/library/os.path.html#os.path.walk
I'd like to copy the files that have a specific file extension to a new folder. I have an idea how to use os.walk but specifically how would I go about using that? I'm searching for the files with a specific file extension in only one folder (this folder has 2 subdirectories but the files I'm looking for will never be found in these 2 subdirectories so I don't need to search in these subdirectories). Thanks in advance.
import glob, os, shutil
files = glob.iglob(os.path.join(source_dir, "*.ext"))
for file in files:
if os.path.isfile(file):
shutil.copy2(file, dest_dir)
Read the documentation of the shutil module to choose the function that fits your needs (shutil.copy(), shutil.copy2() or shutil.copyfile()).
If you're not recursing, you don't need walk().
Federico's answer with glob is fine, assuming you aren't going to have any directories called ‘something.ext’. Otherwise try:
import os, shutil
for basename in os.listdir(srcdir):
if basename.endswith('.ext'):
pathname = os.path.join(srcdir, basename)
if os.path.isfile(pathname):
shutil.copy2(pathname, dstdir)
Here is a non-recursive version with os.walk:
import fnmatch, os, shutil
def copyfiles(srcdir, dstdir, filepattern):
def failed(exc):
raise exc
for dirpath, dirs, files in os.walk(srcdir, topdown=True, onerror=failed):
for file in fnmatch.filter(files, filepattern):
shutil.copy2(os.path.join(dirpath, file), dstdir)
break # no recursion
Example:
copyfiles(".", "test", "*.ext")
This will walk a tree with sub-directories. You can do an os.path.isfile check to make it a little safer.
for root, dirs, files in os.walk(srcDir):
for file in files:
if file[-4:].lower() == '.jpg':
shutil.copy(os.path.join(root, file), os.path.join(dest, file))
Copy files with extension "extension" from srcDir to dstDir...
import os, shutil, sys
srcDir = sys.argv[1]
dstDir = sys.argv[2]
extension = sys.argv[3]
print "Source Dir: ", srcDir, "\n", "Destination Dir: ",dstDir, "\n", "Extension: ", extension
for root, dirs, files in os.walk(srcDir):
for file_ in files:
if file_.endswith(extension):
shutil.copy(os.path.join(root, file_), os.path.join(dstDir, file_))