I only know how to write python for GIS purposes. There is more to this code using arcpy and geoprocessing tools.... this is just the beginning part I'm stuck on that is for trying to get data ready so I can then use the shapefiles within the zipped folder for the rest of my script
I am trying to prompt the user to enter a directory to search through. For use of this script it will be searching for a compressed zip file, then extract all the files to that same directory.
import zipfile, os
# ask what directory to search in
mypath = input("Enter .zip folder path: ")
extension = ".zip"
os.chdir(mypath) # change directory from working dir to dir with files
for item in os.listdir(mypath):
if item.endswith(extension):
filename = os.path.abspath(item)
zip_ref = zipfile.ZipFile(filename)
zip_ref.extractall(mypath)
zip_ref.close()
Tried with y'alls suggestions and still have issues with the following:
import zipfile, os
mypath = input("Enter folder: ")
if os.path.isdir(mypath):
for dirpath, dirname, filenames in os.listdir(mypath):
for file in filenames:
if file.endswith(".zip"):
print(os.path.abspath(file))
with zipfile.ZipFile(os.path.abspath(file)) as z:
z.extractall(mypath)
else:
print("Directory does not exist.")
I'm not sure on the use of arcpy. However...
To iterate over entries in a directory, use os.listdir:
for entry_name in os.listdir(directory_path):
# ...
Inside the loop, entry_name will be the name of an item in the directory at directory_path.
When checking if it ends with ".zip", keep in mind that the comparison is case sensitive. You can use str.lower to effectively ignore case when using str.endswith:
if entry_name.lower().endswith('.zip'):
# ...
To get the full path to the entry (in this case, your .zip), use os.path.join:
entry_path = os.path.join(directory_path, entry_name)
Pass this full path to zipfile.ZipFile.
Your first if-block will negate the else-block if the location is invalid. I'd remove the 'else' operator entirely. If you keep it, the if-check effectively kills the program. The "if folderExist" is sufficient to replace the else.
import arcpy, zipfile, os
# ask what directory to search in
folder = input("Where is the directory? ")
# set workspace as variable so can change location
arcpy.env.workspace = folder
# check if invalid entry - if bad, ask to input different location
if len(folder) == 0:
print("Invalid location.")
new_folder = input("Try another directory?")
new_folder = folder
# does the above replace old location and re set as directory location?
# check to see if folder exists
folderExist = arcpy.Exists(folder)
if folderExist:
# loop through files in directory
for item in folder:
# check for .zip extension
if item.endswith(".zip"):
file_name = os.path.abspath(item) # get full path of files
print(file_name)
zip_ref = zipfile.ZipFile(file_name) # create zipfile object
zip_ref.extractall(folder) # extract all to directory
zip_ref.close() # close file
This may be neater if you're okay not using your original code:
import zipfile, os
from tkinter import filedialog as fd
# ask what directory to search in
folder = fd.askdirectory(title="Where is the directory?")
# loop through files in directory
for item in os.listdir(folder):
# check for .zip extension
if zipfile.is_zipfile(item):
file_name = os.path.abspath(item) # get full path of files
# could string combine to ensure path
# file_name = folder + "/" + item
print(file_name)
zip_ref = zipfile.ZipFile(file_name) # create zipfile object
zip_ref.extractall(folder) # extract all to directory
zip_ref.close() # close file
[UPDATED]
I was able to solve the same with the following code
import os
import zipfile
mypath = raw_input('Enter Folder: ')
if os.path.isdir(mypath):
for file in os.listdir(mypath):
if file.endswith('.zip'):
with zipfile.ZipFile(os.path.join(mypath, file)) as z:
z.extractall(mypath)
else:
print('Directory does not exist')
Related
Usually I navigate to the folder I am extracting data from and copy the file name directly:
df2=pd.read_csv('10_90_bnOH-MEA.csv',usecols=[1])
If I have multiple files and want to do the same for all the files, how do I specify the folder to open and get all the files inside?
I want to run the above code without specifying the file's full path
(C:\Users\X\Desktop\Y\Z\10_90_bnOH-MEA.csv)
You want listdir from the os module.
import os
path = "C:\\Users\\X\\Desktop\\Y\\Z\\"
files = os.listdir(path)
print(files)
dataframe_list = []
for filename in files:
dataframe_list.append(pd.read_csv(os.path.join(path,filename)))
You should open the desired directory and loop through all the files then do something to them.
# import required module
import os
# assign directory
directory = 'files'
# iterate over files in
def goThroughDirectory(directory):
for filename in os.listdir(directory):
f = os.path.join(directory, filename)
# checking if it is a file
if os.path.isfile(f):
# do something
If you also want to loop through all the files in a directory you should add a check for if os.path.isdir(f) like this
...
def goThroughDirectory(directory):
for filename in os.listdir(directory):
f = os.path.join(directory, filename)
# checking if it is a file
if os.path.isfile(f):
# do something
elif os.path.isdir(f):
# its not a file but a directory then loop through that directory aswell
goThroughDirectory(directory + "\" + f)
for more information you should check geeksforgeeks
I have problems solving a task. Below you can see my code. I want to work with sub-sub-folders too but my code only moves the sub-folders. How can I do this recursively that it moves all folders in main folder?
path = (r"C:\Users\Desktop\testfolder")
os.chdir(path)
all_files = list(os.listdir())
outputs = os.getcwd()
for files in all_files:
try:
inputs = glob.glob(files + "\\*")
for a in inputs:
shutil.move(a, outputs)
except:
pass
for files in os.listdir("."):
dir_folder = files[:32]
if not os.path.isdir(dir_folder):
try:
os.mkdir(dir_folder)
except:
pass
Here is the new code, that will maintain the folder structure in the output folder as it was in the input folder
from pathlib import Path
import shutil
import os
for path, dir_folder, files in os.walk('./test_folder'):
for file in files:
full_path = os.path.join(path,file)
## Create the path to the subfolder
sub_folder_path = '/'.join(path.split('/')[2:])
# Create output path by joining subfolder path and output directory
output_path = os.path.join(outputs, sub_folder_path)
# Create the output path directory structure if it does not exists, ignore if it does
Path(output_path).mkdir(parents=True, exist_ok=True)
# Move file
shutil.move(full_path, output_path)
You can use os.walk to recursively visit each folder and sub folder and then move all the files in that folder.
import shutil
import os
for path, dir_folder, files in os.walk(input_folder):
for file in files:
full_path = os.path.join(path,file)
move_path = os.path.join(output_path,file)
print(move_path)
shutil.move(full_path, move_path)
print(file)
Where output_path is the path to the folder you move to move the files to.
This will not create the same folder structure in output folder, just move all the files there, do you also want to maintain structure in output folder? If so, I can edit my answer to reflect that
I work as an IT help desk associate and have to constantly clear out disk space for VDIs. I want to write a code that can clear out specific folders all at once to save time. I wrote a code, but only have it to run one folder at a time. Is there a way for me to make it a function and just make variables for each path and have them all cleared at once. Sample:
import os
#path to delete
path = r"C:\Users\jwals\test test"
#will check if the path is a dir or not
if os.path.exists(path):
#iterating through subfolders
for root_folder, folders, files in os.walk(path):
#checking for files
for file in files:
#file path
file_path = os.path.join(root_folder, file)
#delete the files in that path
if not os.remove(file_path):
#will print if successful
print(f"{file_path} deleted successfully")
#unsuccessful message (if this prints something went wrong)
else:
print(f"Unable to delete the {file_path}")
IIUIC, you can simply do this:
import os
#paths to delete
paths = [r"C:\Users\jwals\test test", r"C:\Users\jwals\another test"]
def delete_files(path):
#will check if the path is a dir or not
if os.path.exists(path):
#iterating through subfolders
for root_folder, folders, files in os.walk(path):
#checking for files
for file in files:
#file path
file_path = os.path.join(root_folder, file)
...
for path in paths:
delete_files(path)
I have some code to export all files within a zipfile to a path but what I want to do is create a new folder with the same name as the zipfile minus the ".zip" just like the windows explorer option does. I have commented out the code that doesn't work. It seems to be the os.makedirs that doesn't work.
File "C:/Users/brentond/Documents/Python/Unzip all zip files in path.py", line 12
Output = os.path.join(path, filename.replace(".zip", "")) # get new folder path name
^
SyntaxError: invalid syntax
the code:
import os, zipfile
# Define path of zip files to variable
path = r'C:\Users\brentond\Documents\TA2\HA GDMS'
for foldername, subfolders, filenames in os.walk(path): # walk directory
for filename in filenames: # loop through files
if filename.endswith(".zip"): # find zip files
filepath = os.path.join(foldername, filename) # get zip file abs path
#os.makedirs(os.path.join(path, filename.replace(".zip", "")) # create new folder same name as zip file
#Output = os.path.join(path, filename.replace(".zip", "")) # get new folder path name
ZipRef = zipfile.ZipFile(filepath) # create zip file object
ZipRef.extractall(path) # extract all. This to put everything in the path folder
#ZipRef.extractall(Output) # This to put the zip file contents into a folder with same name
ZipRef.close() # close zip
I have resolved this now and simplified the code a little
import os, zipfile
# Define path of zip files to variable
path = r'C:\Users\brentond\Documents\TA2\HA GDMS'
for foldername, subfolders, filenames in os.walk(path): # walk directory
for filename in filenames: # loop through files
if filename.endswith(".zip"): # find zip files
filepath = os.path.join(foldername, filename) # get zip file abs path
filefolder = filename.replace(".zip","")
os.makedirs(os.path.join(path, filefolder)) # create new folder same name as zip file
Output = os.path.join(path, filefolder) # get new folder path name
ZipRef = zipfile.ZipFile(filepath) # create zip file object
#ZipRef.extractall(path) # extract all. This to put everything in the path folder
ZipRef.extractall(Output) # This to put the zip file contents into a folder with same name
ZipRef.close() # close zip
This is probably a simple question, but I'm brand new to python and programming in general.
I'm working on a simple program to copy/move .mp3 files from on location to another while mirroring the directory structure of the source location. What I have so far works, however it also creates new folders in the destination location even if the source folder contained no mp3 files. I only want to create the new directories if the source contains .mp3s, otherwise it could lead to a bunch of empty folders in the destination.
Here is what I have so far:
import os
import shutil #Used for copying files
##CONFIG
source_dir = "C:\Users\username\Desktop\iTunes\\" #set the root folder that you want to scan and move files from. This script will scan recursively.
destPath = "C:\Users\username\Desktop\converted From iTunes" #set the destination root that you want to move files to. Any non-existing sub directories will be created.
ext = ".mp3" #set the type of file you want to search for.
count = 0 #initialize counter variable to count number of files moved
##
##FIND FILES
for dirName, subdirList, fileList in os.walk(source_dir):
#set the path for the destination folder(s)
dest = destPath + dirName.replace(source_dir, '\\')
#if the source directory doesn't exist in the destination folder
#then create a new folder
if not os.path.isdir(dest):
os.mkdir(dest)
print('Directory created at: ' + dest)
for fname in fileList:
if fname.endswith(ext) :
#determine source & new file locations
oldLoc = dirName + '\\' + fname
newLoc = dest + '\\' + fname
if os.path.isfile(newLoc): # check to see if the file already exists. If it does print out a message saying so.
print ('file "' + newLoc + fname + '" already exists')
if not os.path.isfile(newLoc): #if the file doesnt exist then copy it and print out confirmation that is was copied/moved
try:
shutil.move(oldLoc, newLoc)
print('File ' + fname + ' copied.')
count = count + 1
except IOError:
print('There was an error copying the file: "' + fname + '"')
print 'error'
print "\n"
print str(count) + " files were moved."
print "\n"
so if the folder structure is something like:
root->
band 1->
album name->
song.m4a,
song2.m4a
right now it will create all those folders in the destination driectory, even though there are no .mp3s to copy.....
Any help is appreciated!
I think I would create my own wrapper around copy for this sort of thing:
def fcopy(src,dest):
"""
Copy file from source to dest. dest can include an absolute or relative path
If the path doesn't exist, it gets created
"""
dest_dir = os.path.dirname(dest)
try:
os.makedirs(dest_dir)
except os.error as e:
pass #Assume it exists. This could fail if you don't have permissions, etc...
shutil.copy(src,dest)
Now you can just walk the tree calling this function on any .mp3 file.
The simplest thing to do I can think of for your existing code would be to just make it skip over any folders that don't have any .mp3 files in them. This can easily be done by adding the following items and if statement to the top of your loop. The itertools.ifilter() and fnmatch.fnmatch() functions can be used together to simplify checking for files with the proper extension.
from itertools import ifilter
from fnmatch import fnmatch
ext = '.mp3'
fnPattern = '*'+ext
for dirName, subdirList, fileList in os.walk(source_dir):
if not any(ifilter(lambda fname: fnmatch(fname, fnPattern), fileList)):
print ' skipping "{}"'.format(dirName)
continue
...
You will also have to change the os.mkdir(dest) to os.makedirs(dest) in the code further down to ensure that any subdirectories skipped by earlier iterations get created when there's a need to copy files to a corresponding subbranch of the destination directory.
You could optimize things a bit by creating and saving a possibly empty iterator of matching files that have the extension, and then use it again later to to determine what files to copy:
from itertools import ifilter
from fnmatch import fnmatch
ext = '.mp3'
fnPattern = '*'+ext
for dirName, subdirList, fileList in os.walk(source_dir):
# generate list of files in directory with desired extension
matches = ifilter(lambda fname: fnmatch(fname, fnPattern), fileList)
# skip subdirectory if it does not contain any files of interest
if not matches:
continue
...
... create destination directory with os.makedirs()
...
# copy each file to destination directory
for fname in matches:
... copy file
Would shutils.copytree not do what you want in fewer lines?