We Keep Coding

Python: read folders that are not ZIP folders

Currently I read folders individually like so:
input_location = r'path\202206'
I would like to read in all the folders at once + not read in the zipped files.
Essentially the logic id like to perform is input_location = r'path\202206' + r'path\202207' + r'path\202207' + r'path\202208'
I cant just do input_location = r'path\ as it may read in those zip files I do not want.
Is there a way to read in the folders without reading in the zip files? Or explicitly list the folder names in one variable (input_location)?

IIUC: Collecting all directories from a directory can be done using a simple list comprehension as follows.
The glob library is nice, as it returns the full filepath, whereas a function such as os.listdir() only returns the filenames.
import os
from glob import glob
dirs = [f for f in glob('/path/to/files/*') if os.path.isdir(f)]
Output:
['/path/to/files/202207',
'/path/to/files/202206',
'/path/to/files/202208',
'/path/to/files/202209']
Then, your script can iterate over the list of directories as required.
For completeness, the directory content is a follows:
202206
202206.zip
202207
202207.zip
202208
202208.zip
202209
202209.zip

If you use pathlib from Pythons standard library you can get all entries in the folder and check if the entry is a folder.
from pathlib import Path
for entry in Path('/path/to/folder').glob('*'):
if entry.is_dir():
print(entry)

A os base approach. Notice that os.listdir returns the content of the directory in a basename form and not as a path.
import os
def my_dirs(wd):
return list(filter(os.path.isdir, (os.path.join(wd, f) for f in os.listdir('.'))))
working_dir = # add path
print(*my_dirs(working_dir), sep='\n')
Remarks: to make your program platform independent you always stuffs like os.path.join or os.sep for path manipulation

How to run script for all files in a folder/directry

I am new to python. I have successful written a script to search for something within a file using :
open(r"C:\file.txt) and re.search function and all works fine.
Is there a way to do the search function with all files within a folder? Because currently, I have to manually change the file name of my script by open(r"C:\file.txt),open(r"C:\file1.txt),open(r"C:\file2.txt)`, etc.
Thanks.

You can use os.walk to check all the files, as the following:
import os
for root, _, files in os.walk(path):
for filename in files:
with open(os.path.join(root, filename), 'r') as f:
#your code goes here
Explanation:
os.walk returns tuple of (root path, dir names, file names) in the folder, so you can iterate through filenames and open each file by using os.path.join(root, filename) which basically joins the root path with the file name so you can open the file.

Since you're a beginner, I'll give you a simple solution and walk through it.
Import the os module, and use the os.listdir function to create a list of everything in the directory. Then, iterate through the files using a for loop.
Example:
# Importing the os module
import os
# Give the directory you wish to iterate through
my_dir = <your directory - i.e. "C:\Users\bleh\Desktop\files">
# Using os.listdir to create a list of all of the files in dir
dir_list = os.listdir(my_dir)
# Use the for loop to iterate through the list you just created, and open the files
for f in dir_list:
# Whatever you want to do to all of the files
If you need help on the concepts, refer to the following:
for looops in p3: http://www.python-course.eu/python3_for_loop.php
os function Library (this has some cool stuff in it): https://docs.python.org/2/library/os.html
Good luck!

You can use the os.listdir(path) function:
import os
path = '/Users/ricardomartinez/repos/Salary-API'
# List for all files in a given PATH
file_list = os.listdir(path)
# If you want to filter by file type
file_list = [file for file in os.listdir(path) if os.path.splitext(file)[1] == '.py']
# Both cases yo can iterate over the list and apply the operations
# that you have
for file in file_list:
print(file)
#Operations that you want to do over files

Copy files from multiple directories into one directory using Python

What is the easiest way to copy files from multiple directories into just one directory using python? To be more clear, I have a tree that looks like this
+Home_Directory
++folder1
-csv1.csv
-csv2.csv
++folder2
-csv3.csv
-csv4.csv
and I want to put csv1,csv2,...etc all into some specified directory without the folder hierarchy.
+some_folder
-csv1.csv
-csv2.csv
-csv3.csv
-csv4.csv
Some solutions I have looked at:
Using shutil.copytree will not work because it will preserve the file structure which is not what I want.
The code I am playing with is very similar to what is posted in this question:
copy multiple files in python
the problem is that I do not know how to do this iteratively. Presumably it would just be another for loop on top of this but I am not familiar enough with the os and shutil libraries to know exactly what I am iterating over. Any help on this?

This is what I thought of. I am assuming you are only pulling csv files from 1 directory.
RootDir1 = r'*your directory*'
TargetFolder = r'*your target folder*'
for root, dirs, files in os.walk((os.path.normpath(RootDir1)), topdown=False):
for name in files:
if name.endswith('.csv'):
print "Found"
SourceFolder = os.path.join(root,name)
shutil.copy2(SourceFolder, TargetFolder) #copies csv to new folder
Edit: missing a ' at the end of RootDir1. You can also use this as a starting guide to make it work as desired.

import glob
import shutil
#import os
dst_dir = "E:/images"
print ('Named explicitly:')
for name in glob.glob('E:/ms/*/*/*'):
if name.endswith(".jpg") or name.endswith(".pdf") :
shutil.copy(name, dst_dir)
print ('\t', name)

You can use it to move all subfolders from the same to a different directory to wherever you want.
import shutil
import os
path=r'* Your Path*'
arr = os.listdir(path)
for i in range(len(arr)):
source_dir=path+'/'+arr[i]
target_dir = r'*Target path*'
file_names = os.listdir(source_dir)
for file_name in file_names:
shutil.move(os.path.join(source_dir, file_name), target_dir)

Directory is not being recognized in Python

I'm uploading a zipped folder that contains a folder of text files, but it's not detecting that the folder that is zipped up is a directory. I think it might have something to do with requiring an absolute path in the os.path.isdir call, but can't seem to figure out how to implement that.
zipped = zipfile.ZipFile(request.FILES['content'])
for libitem in zipped.namelist():
if libitem.startswith('__MACOSX/'):
continue
# If it's a directory, open it
if os.path.isdir(libitem):
print "You have hit a directory in the zip folder -- we must open it before continuing"
for item in os.listdir(libitem):

The file you've uploaded is a single zip file which is simply a container for other files and directories. All of the Python os.path functions operate on files on your local file system which means you must first extract the contents of your zip before you can use os.path or os.listdir.
Unfortunately it's not possible to determine from the ZipFile object whether an entry is for a file or directory.
A rewrite or your code which does an extract first may look something like this:
import tempfile
# Create a temporary directory into which we can extract zip contents.
tmpdir = tempfile.mkdtemp()
try:
zipped = zipfile.ZipFile(request.FILES['content'])
zipped.extractall(tmpdir)
# Walk through the extracted directory structure doing what you
# want with each file.
for (dirpath, dirnames, filenames) in os.walk(tmpdir):
# Look into subdirectories?
for dirname in dirnames:
full_dir_path = os.path.join(dirpath, dirname)
# Do stuff in this directory
for filename in filenames:
full_file_path = os.path.join(dirpath, filename)
# Do stuff with this file.
finally:
# ... Clean up temporary diretory recursively here.

Usually to make things handle relative paths etc when running scripts you'd want to use os.path.
It seems to me that you're reading from a Zipfile the items you've not actually unzipped it so why would you expect the file/dirs to exist?
Usually I'd print os.getcwd() to find out where I am and also use os.path.join to join with the root of the data directory, whether that is the same as the directory containing the script I can't tell. Using something like scriptdir = os.path.dirname(os.path.abspath(__file__)).
I'd expect you would have to do something like
libitempath = os.path.join(scriptdir, libitem)
if os.path.isdir(libitempath):
....
But I'm guessing at what you're doing as it's a little unclear for me.

Deleting files by type in Python on Windows

I know how to delete single files, however I am lost in my implementation of how to delete all files in a directory of one type.
Say the directory is \myfolder
I want to delete all files that are .config files, but nothing to the other ones.
How would I do this?
Thanks Kindly

Use the glob module:
import os
from glob import glob
for f in glob ('myfolder/*.config'):
os.unlink (f)

I would do something like the following:
import os
files = os.listdir("myfolder")
for f in files:
if not os.path.isdir(f) and ".config" in f:
os.remove(f)
It lists the files in a directory and if it's not a directory and the filename has ".config" anywhere in it, delete it. You'll either need to be in the same directory as myfolder, or give it the full path to the directory. If you need to do this recursively, I would use the os.walk function.

Here ya go:
import os
# Return all files in dir, and all its subdirectories, ending in pattern
def gen_files(dir, pattern):
for dirname, subdirs, files in os.walk(dir):
for f in files:
if f.endswith(pattern):
yield os.path.join(dirname, f)
# Remove all files in the current dir matching *.config
for f in gen_files('.', '.config'):
os.remove(f)
Note also that gen_files can be easily rewritten to accept a tuple of patterns, since str.endswith accepts a tuple