We Keep Coding

Apply script to multiple folders using string in file path [duplicate]

This question already has answers here:
How to use glob() to find files recursively?
(28 answers)
Closed 1 year ago.
I am new to the programing world and I have hit a snag on a piece of code.
What I have: I have a piece of code that identifies all .MP4 files and calculates the size of the files within a directory. So far I can only apply this code to a specific folder that I input manually. But the code works.
Problem: I would like to apply what I have to multiple folders within a directory. I have several folders with years in the file path and I would like to apply this code to each year individually. For example: I need a piece of code/direction to code that can allow me to run this code on all folders with '2021' in the name.
Any pointers or suggestions are welcome.
# import module
import os
# assign size
size = 0
# assign folder path
Folderpath = r'C:\file_path_name_here'
# get size
for path, dirs, files in os.walk(Folderpath):
for f in files:
if not f.endswith('.MP4'):
continue
else:
fp = os.path.join(path, f)
size += os.path.getsize(fp)
# display size
print("Folder size: " + str(size))

You can use glob for that.
If i understand you correctly you want to iterate through all Folders right?
I myself am not a routined coder aswell but i use it in for a Script where i have to iterate over an unknown number of files and folders. In my case PDF's
which then get scanned/indexed/merged...
This obviously returns a list of of files with which you then could workd through. os.path commonpath is also handy for that.
def getpdflisting(fpath):
filelist = []
for filepath in Path(os.path.join(config['Ordner']['path'] +\
fpath)).glob('**/*.pdf'):
filelist.append(str(filepath))
if filelist:
filelist = [x for x in filelist if x]
logger.info(filelist)
return filelist
Or even better https://stackoverflow.com/a/66042729/16573616

Python. Create folder if it does not exist. But more complicated [duplicate]

This question already has answers here:
Creating 100+ folders and naming them sequentially [closed]
(5 answers)
Closed 2 years ago.
I'm making a python program using the Flask framework, but I have a problem.
I'll explain.
I need to save some images in a directory. So I have to create a directory called Folder, but first I have to check if this directory doesn't already exist.
So I should check if Folder exists, if it doesn't exist I create Folder directory, otherwise I create Folder1 directory.
But in the same way I have to check if Folder1 exists and if it already exists I see for the Folder2 directory and so on ...
After creating the directory I need to always use the same directory to save all the images.
That is, even if I terminate the program, the next time I run it must always save the other images in the directory created.
I ask for your help in doing this, because I don't know how to do it.
I tried to do this, but it doesn't work as it should:
path = "path/to/folder"
def create_directory():
global path
if(os.path.isdir(path)==False):
os.makedirs(path)
else:
cont=1
new_path = path
while(os.path.isdir(new_path)==True):
new_path = str(path)+str(cont)
cont= cont +1
os.makedirs(new_path)

Hope this code helps:
import os
# define the name of the directory to be created
path = "/root/directory1"
try:
os.mkdir(path)
except OSError:
print ("Creation of the directory %s failed" % path)
else:
print ("Successfully created the directory %s " % path)

This can use a number of error handling (in case regex fails, etc) but should get you started on the correct path:
import regex
from pathlib import Path
def mkdir(path_str):
path = Path(path_str)
# this while loop ensures you get the next folder name (the correct number appended to the base folder name)
while path.exists():
s= re.search('(\w+)(\d+)',path.name)
base,number = s.groups()
new_path_str = base + str(int(number)+1)
path = path.parent.joinpath(new_path_str)
try:
path.mkdir()
print(f'The directory {path.name} was created!')
except OSError:
print (f"Creation of the directory {path} failed")

How can I remove a prefix from a filename in Python? [duplicate]

This question already has answers here:
Get relative path from comparing two absolute paths
(6 answers)
Closed 5 years ago.
I want to write a script that receives a path to a directory and a path to a file contained in that directory (possibly nested many directories deep) and returns a path to this file relative to the outer directory.
For example, if the outer directory is /home/hugomg/foo and the inner file is /home/hugomg/foo/bar/baz/unicorns.txt I would like the script to output bar/baz/unicorns.txt.
Right now I am doing it using realpath and string manipulation:
import os
dir_path = "/home/hugomg/foo"
file_path = "/home/hugomg/foo/bar/baz/unicorns.py"
dir_path = os.path.realpath(dir_path)
file_path = os.path.realpath(file_path)
if not file_path.startswith(dir_path):
print("file is not inside the directory")
exit(1)
output = file_path[len(dir_path):]
output = output.lstrip("/")
print(output)
But is there a more robust way to do this? I'm not confident that my current solution is the right way to do this. Is using startswith together with realpath a correct way to test that one file is inside another? And is there a way to avoid that awkward situation with the leading slash that I might need to remove?

You can use the commonprefix and relpath of the os.path module to find the longest common prefix of two paths. Its always preferred to use realpath.
import os
dir_path = os.path.realpath("/home/hugomg/foo")
file_path = os.path.realpath("/home/hugomg/foo/bar/baz/unicorns.py")
common_prefix = os.path.commonprefix([dir_path,file_path])
if common_prefix != dir_path:
print("file is not inside the directory")
exit(1)
print(os.path.relpath(file_path, dir_path))
Output:
bar/baz/unicorns.txt

Count the number of folders in a directory and subdirectories

I've got a script that will accurately tell me how many files are in a directory, and the subdirectories within. However, I'm also looking into identify how many folders there are within the same directory and its subdirectories...
My current script:
import os, getpass
from os.path import join, getsize
user = 'Copy of ' + getpass.getuser()
path = "C://Documents and Settings//" + user + "./"
folder_counter = sum([len(folder) for r, d, folder in os.walk(path)])
file_counter = sum([len(files) for r, d, files in os.walk(path)])
print ' [*] ' + str(file_counter) + ' Files were found and ' + str(folder_counter) + ' folders'
This code gives me the print out of: [*] 147 Files were found and 147 folders.
Meaning that the folder_counter isn't counting the right elements. How can I correct this so the folder_counter is correct?

Python 2.7 solution
For a single directory and in you can also do:
import os
print len(os.walk('dir_name').next()[1])
which will not load the whole string list and also return you the amount of directories inside the 'dir_name' directory.
Python 3.x solution
Since many people just want an easy and fast solution, without actually understanding the solution, I edit my answer to include the exact working code for Python 3.x.
So, in Python 3.x we have the next method instead of .next. Thus, the above snippet becomes:
import os
print(len(next(os.walk('dir_name'))[1]))
where dir_name is the directory that you want to find out how many directories has inside.

I think you want something like:
import os
files = folders = 0
for _, dirnames, filenames in os.walk(path):
# ^ this idiom means "we won't be using this value"
files += len(filenames)
folders += len(dirnames)
print "{:,} files, {:,} folders".format(files, folders)
Note that this only iterates over os.walk once, which will make it much quicker on paths containing lots of files and directories. Running it on my Python directory gives me:
30,183 files, 2,074 folders
which exactly matches what the Windows folder properties view tells me.
Note that your current code calculates the same number twice because the only change is renaming one of the returned values from the call to os.walk:
folder_counter = sum([len(folder) for r, d, folder in os.walk(path)])
# ^ here # ^ and here
file_counter = sum([len(files) for r, d, files in os.walk(path)])
# ^ vs. here # ^ and here
Despite that name change, you're counting the same value (i.e. in both it's the third of the three returned values that you're using)! Python functions do not know what names (if any at all; you could do print list(os.walk(path)), for example) the values they return will be assigned to, and their behaviour certainly won't change because of it. Per the documentation, os.walk returns a three-tuple (dirpath, dirnames, filenames), and the names you use for that, e.g. whether:
for foo, bar, baz in os.walk(...):
or:
for all_three in os.walk(..):
won't change that.

If interested only in the number of folders in /input/dir (and not in the subdirectories):
import os
folder_count = 0 # type: int
input_path = "/path/to/your/input/dir" # type: str
for folders in os.listdir(input_path): # loop over all files
if os.path.isdir(os.path.join(input_path, folders): # if it's a directory
folder_count += 1 # increment counter
print("There are {} folders".format(folder_count))

>>> import os
>>> len(list(os.walk('folder_name')))
According to os.walk the first argument dirpath enumerates all directories.

Moving into a directory without knowing its name

Im a noob to python and I am trying to complete this simple task. I want to access multiple directories one by one that are all located inside one directory. I do not have the names of the multiple directories. I need to enter into the directory, combine some files, move back out of that directory, then into the next directory, combine some files, move back out of it, and so on........ I need to make sure I dont access the same directory more than once.
I looked around and tried various commands and nothing yet.

try using something like the following piece of code.:
import os, fnmatch
def find_files(directory, pattern):
for root, dirs, files in os.walk(directory):
for basename in files:
if fnmatch.fnmatch(basename, pattern):
filename = os.path.join(root, basename)
yield filename
use it something like this:
for filename in find_files('/home/', '*.html')
# do something

Sometimes I find glob to be useful:
from glob import glob
import os
nodes = glob('/tmp/*/*')
for node in nodes:
try:
print 'now in directory {}'.format(os.path.dirname(node))
with open(node, 'r') as f:
# Do something with f...
print len(f.read())
except IOError: # Because node may be a directory, which we cannot 'open'
continue