I am trying to expose the selective path of a file, found using os.walk. The walk function is working correctly and is exposing all the files I want however right now I am able to expose either only the file name or full path of the file.
path = 'C:/Users/testing_recurssion'
for root, d_names, f_names in os.walk(path):
for name in f_names:
print(os.path.join(root, name))
This returns
C:/Users/testing_recurssion\folder1\file3.txt
C:/Users/testing_recurssion\folder1\folder3\file4.txt
However, I want it to return
folder1\file3.txt
folder1\folder3\file4.txt**
Use os.relpath
directory = "C:/Users/testing_recurssion"
for root, d_names, f_names in os.walk(directory):
for name in f_names:
path = os.path.join(root, name)
print(os.path.relpath(path, directory))
Also - consider using pathlib.Path which is more recent and object oriented:
from pathlib import Path
directory = Path("C:/Users/testing_recurssion")
for path in directory.rglob("*"):
print(path.relative_to(directory))
Related
I am trying to collect all files with all sub-directories and move to another directory
Code used
#collects all mp3 files from folders to a new folder
import os
from pathlib import Path
import shutil
#run once
path = os.getcwd()
os.mkdir("empetrishki")
empetrishki = path + "/empetrishki" #destination dir
print(path)
print(empetrishki)
#recursive collection
for root, dirs, files in os.walk(path, topdown=True, onerror=None, followlinks=True):
for name in files:
filePath = Path(name)
if filePath.suffix.lower() == ".mp3":
print(filePath)
os.path.join
filePath.rename(empetrishki.joinpath(filePath))
I have trouble with the last line of moving files: filePath.rename() nor shutil.move nor joinpath() have worked for me. Maybe that's because I am trying to change the element in the tuple - the output from os.walk
Similar code works with os.scandir but this would collect files only in the current directory
How can I fix that, thanks!
If you use pathlib.Path(name) that doesn't mean that something exists called name. Hence, you do need to be careful that you have a full path, or relative path, and you need to make sure to resolve those. In particular I am noting that you don't change your working directory and have a line like this:
filePath = Path(name)
This means that while you may be walking down the directory, your working directory may not be changing. You should make your path from the root and the name, it is also a good idea to resolve so that the full path is known.
filePath = Path(root).joinpath(name).resolve()
You can also place the Path(root) outside the inner loop as well. Now you have an absolute path from '/home/' to the filename. Hence, you should be able to rename with .rename(), like:
filePath.rename(x.parent.joinpath(newname))
#Or to another directory
filePath.rename(other_dir.joinpath(newname))
All together:
from pathlib import os, Path
empetrishki = Path.cwd().joinpath("empetrishki").resolve()
for root, dirs, files in os.walk(path, topdown=True, onerror=None, followlinks=True):
root = Path(root).resolve()
for name in files:
file = root.joinpath(name)
if file.suffix.lower() == ".mp3":
file.rename(empetrishki.joinpath(file.name))
for root, dirs, files in os.walk(path, topdown=True, onerror=None, followlinks=True):
if root == empetrishki:
continue # skip the destination dir
for name in files:
basename, extension = os.path.splitext(name)
if extension.lower() == ".mp3":
oldpath = os.path.join(root, name)
newpath = os.path.join(empetrishki, name)
print(oldpath)
shutil.move(oldpath, newpath)
This is what I suggest. Your code is running on the current directory, and the file is at the path os.path.join(root, name) and you need to provide such path to your move function.
Besides, I would also suggest to use os.path.splitext for extracting the file extension. More pythonic. And also you might want to skip scanning your target directory.
Kind of new to python. But after searching and trying to unzip some folders, then rename them that don't have static names. For example the file is New_05222016. The #s are the date, and that always changes. I want it to be a unzipped folder that is labeled "New".
This is what i have so far. It will unzipp my file, but won't rename it.
import zipfile,fnmatch,os
rootPath = r"C:/Users/Bob/Desktop/Bill"
pattern = '*.zip'
New = 'New*'
for root, dirs, files in os.walk(rootPath):
for filename in fnmatch.filter(files, pattern):
print(os.path.join(root, filename))
zipfile.ZipFile(os.path.join(root, filename)).extractall(os.path.join(root, os.path.splitext(filename)[0]))
for root, dirs, files in os.walk(rootPath):
for filename in fnmatch.filter(dir,New):
os.rename(dir,'C:/Users/Bob/Desktop/Bill/New')
if tried other ways. Such as just os.rename and typing it out. But i'm at a loss of what to do.
os.rename() will work fine, just be sure to specify the full path.
I've modified your example using os.listdir() to store the name of the unzipped directory and then renamed it using os.rename(). I also used re to leave the name of the zipped file intact.
import zipfile,fnmatch,os, re
rootPath = r"C:\Users\Bob\Desktop\Bill"
pattern = '*.zip'
New = 'New*'
for root, dirs, files in os.walk(rootPath):
for filename in fnmatch.filter(files, pattern):
print(os.path.join(root, filename))
zipfile.ZipFile(os.path.join(root, filename)).extractall(os.path.join(root, os.path.splitext(filename)[0]))
for dirName in os.listdir(rootPath):
if not re.search("zip", dirName):
os.rename(os.path.join(rootPath, dirName), os.path.join(rootPath,"New"))
I hope this helps!
I am trying to locate all my files with externsion mp4 in a folder(and subfolder) and copy them in another dir. I manage to find all files with the extension mp4 however I didnt manage to keep the dir of those files. My code is the following:
import os
from shutil import copyfile
path = "videos/"
for root, dirs, files in os.walk(path):
for name in files:
if name.endswith((".mp4", ".mp4")):
print(name)
# copyfile(src, dst)
I want to find the path of the name (corresponding to my vids). How can I do so?
Use os.path.join()
import os
from shutil import copyfile
path = "videos/"
for root, dirs, files in os.walk(path):
for name in files:
if name.endswith((".mp4", ".mp4")):
print(os.path.join(root, name))
# copyfile(src, dst)
Although it is considered better to use absolute paths but you can use os.path.relpath if you want relative path. From the os.path.relpath documentation
os.path.relpath(path[, start])
Return a relative filepath to path either from the current directory or from an optional start directory. This is a path computation: the filesystem is not accessed to confirm the existence or nature of path or start.
start defaults to os.curdir.
Availability: Windows, Unix.
New in version 2.6.
Why not just use glob:
import glob, shutil
for file in glob.iglob('/foo/*.mp4'):
shutil.copy2(file, '/bar/{0}'.format(file))
From the documentation on os.walk:
dirpath is a string, the path to the directory. dirnames is a list of the names of the subdirectories in dirpath (excluding '.' and '..'). filenames is a list of the names of the non-directory files in dirpath. Note that the names in the lists contain no path components. To get a full path (which begins with top) to a file or directory in dirpath, do os.path.join(dirpath, name).
So your code should look like this:
import os
from shutil import copyfile
path = "videos/"
for root, dirs, files in os.walk(path):
for name in files:
if name.endswith((".mp4", ".mp4")):
print(name)
src = os.path.join(root, name)
copyfile(src, dst)
I would like to:
Write a script that takes a single directory path as command line argument, and then walks all subdirectories of that path looking for files with the extension '.py', copying each one to a temporary directory in your file system (eg /tmp/pyfiles). Your script should check for the existence of the temporary directory, and remove it if it already exists; it should then create a new directory, before beginning to copy files.
I have this:
#!/usr/bin/env python
import os, sys
import shutil
#import module
rootdir = sys.argv[1]
#take input directory
if os.path.exists('tmp/pyfiles'):
shutil.rmtree('tmp/pyfiles')
if not os.path.exists('tmp/pyfiles'):
os.makedirs('tmp/pyfiles')
#check whether directory exists, if it exists remove and remake, if not make
for root, dirs, files in os.walk(rootdir):
for f in files:
if os.path.splitext(f)[1] in ['.py']:
shutil.copy2(f, tmp/pyfiles)
#find files ending with .py, copy them and place in tmp/pyfiles directory
I get this error:
Traceback (most recent call last):
File "seek.py", line 20, in <module>
shutil.copy2(f, tmp/pyfiles)
NameError: name 'tmp' is not defined
Could anyone help me out?:) Thx
Your code says shutil.copy2(f, tmp/pyfiles), I believe it meant to be
shutil.copy2(f, 'tmp/pyfiles').
When you use the
os.walk()
method you loose track of the file full path. What I would do is to analyze each directory using the
os.listdir()
method and then copying each file taking into account its absolute path. Something like this:
for root, dirs, files in os.walk(rootdir):
for dir in dirs:
for f in os.listdir(dir):
if os.path.splitext(f)[1] in ['.py']:
shutil.copy2(os.path.join(root, dir, f), "tmp/pyfiles")
I hope this helps, maybe there is a cleaner solution.
You have to check if the root directory exists and walk in to remove everything if not create a new one.
To copy from dir to yours you have to check for files in dir that filename ends with .py, then replace the dir path with root path and create a new file in root dir with the content of matching file.
If we found a directory in dir we should create a new one in the root directory.
After that just call the function recursively to copy all content of dir to the root directory
import os, sys
rootdir = sys.argv[1]
PATH = "/tmp/pyfiles/"
def check(path, _file):
global rootdir, PATH
for item in os.listdir(path):
newpath = os.path.join(path, item)
if os.path.isdir(newpath):
os.mkdir(os.path.join(PATH, newpath.replace(rootdir, '')))
check(newpath, _file)
else:
if item.endswith(_file):
source = open(newpath, 'r')
print os.path.join(path, newpath.replace(rootdir, ''))
output = open(os.path.join(PATH, newpath.replace(rootdir, '')), 'w')
output.write(source.read())
output.close()
source.close()
if __name__ == '__main__':
if os.path.isdir(PATH):
for root, dirs, files in os.walk(PATH, topdown=False):
for name in files:
os.remove(os.path.join(root, name))
for name in dirs:
os.rmdir(os.path.join(root, name))
os.rmdir(PATH)
os.mkdir(PATH)
check(rootdir, '.py')
How do I get the absolute paths of all the files in a directory that could have many sub-folders in Python?
I know os.walk() recursively gives me a list of directories and files, but that doesn't seem to get me what I want.
os.path.abspath makes sure a path is absolute. Use the following helper function:
import os
def absoluteFilePaths(directory):
for dirpath,_,filenames in os.walk(directory):
for f in filenames:
yield os.path.abspath(os.path.join(dirpath, f))
If you have Python 3.4 or newer you can use pathlib (or a third-party backport if you have an older Python version):
import pathlib
for filepath in pathlib.Path(directory).glob('**/*'):
print(filepath.absolute())
If the argument given to os.walk is absolute, then the root dir names yielded during iteration will also be absolute. So, you only need to join them with the filenames:
import os
for root, dirs, files in os.walk(os.path.abspath("../path/to/dir/")):
for file in files:
print(os.path.join(root, file))
Try:
import os
for root, dirs, files in os.walk('.'):
for file in files:
p=os.path.join(root,file)
print p
print os.path.abspath(p)
print
You can use os.path.abspath() to turn relative paths into absolute paths:
file_paths = []
for folder, subs, files in os.walk(rootdir):
for filename in files:
file_paths.append(os.path.abspath(os.path.join(folder, filename)))
Starting with python 3.5 the idiomatic solution would be:
import os
def absolute_file_paths(directory):
path = os.path.abspath(directory)
return [entry.path for entry in os.scandir(path) if entry.is_file()]
This not just reads nicer but also is faster in many cases.
For more details (like ignoring symlinks) see original python docs:
https://docs.python.org/3/library/os.html#os.scandir
All files and folders:
x = [os.path.abspath(os.path.join(directory, p)) for p in os.listdir(directory)]
Images (.jpg | .png):
x = [os.path.abspath(os.path.join(directory, p)) for p in os.listdir(directory) if p.endswith(('jpg', 'png'))]
from glob import glob
def absolute_file_paths(directory):
return glob(join(directory, "**"))
Try:
from pathlib import Path
path = 'Desktop'
files = filter(lambda filepath: filepath.is_file(), Path(path).glob('*'))
for file in files:
print(file.absolute())
I wanted to keep the subdirectory details and not the files and wanted only subdirs with one xml file in them. I can do it this way:
for rootDirectory, subDirectories, files in os.walk(eventDirectory):
for subDirectory in subDirectories:
absSubDir = os.path.join(rootDirectory, subDirectory)
if len(glob.glob(os.path.join(absSubDir, "*.xml"))) == 1:
print "Parsing information in " + absSubDir
for root, directories, filenames in os.walk(directory):
for directory in directories:
print os.path.join(root, directory)
for filename in filenames:
if filename.endswith(".JPG"):
print filename
print os.path.join(root,filename)
Try This
pth=''
types=os.listdir(pth)
for type_ in types:
file_names=os.listdir(f'{pth}/{type_}')
file_names=list(map(lambda x:f'{pth}/{type_}/{x}',file_names))
train_folder+=file_names