I have a script that runs on a folder to create contour lines. Since I have roughly 2700 DEM which need to be processed, I need a way using the script to run on all folders within the parent folder saving them to an output folder. I am not sure how to script this but it would be greatly appreciated if I could get some guidance.
The following is the script I currently have which works on a single folder.
import arcpy
from arcpy import env
from arcpy.sa import *
env.workspace = "C:/DATA/ScriptTesting/test"
inRaster = "1km17670"
contourInterval = 5
baseContour = 0
outContours = "C:/DATA/ScriptTesting/test/output/contours5.shp"
arcpy.CheckOutExtension("Spatial")
Contour(inRaster,outContours, contourInterval, baseContour)
You're probably looking for os.walk(), which can recursively walk through all subdirectories of the given directory. You can either use the current working directory, or calculate your own parent folder and start from there, or whatever - but it'll give you the filenames for everything beneath what it starts with. From there, you can make a subroutine to determine whether or not to perform your script on that file.
You can get a list of all directories like this:
import arcpy
from arcpy import env
from arcpy.sa import *
import os
# pass in your root directory here
directories = os.listdir(root_dir)
Then you can iterate over this dirs:
for directory in directories:
# I assume you want the workspace attribute set to the subfolders
env.workspace = os.path.realpath(directory)
inRaster = "1km17670"
contourInterval = 5
baseContour = 0
# here you need to adjust the outputfile name if there is a file for every subdir
outContours = "C:/DATA/ScriptTesting/test/output/contours5.shp"
arcpy.CheckOutExtension("Spatial")
Contour(inRaster,outContours, contourInterval, baseContour)
As #a625993 mentioned, os.walk could be useful too if you have recursively nested directories. But as I can read from your question, you have just single subdirectories which directly contain the files and no further directories. That's why listing just the dirs underneath your root directory should be enough.
Related
Im trying to make a data parser but using python for my project but each files are inside a separate folders to each other. Currently i am able to read the first folder but i havent figured out how to read the folder after that and put it in a for loop
import os
r_path='//esw-fs01/esw_niagara_no_bck/BuildResults/master/0.1.52.68_390534/installation_area/autotestlogs_top/'
sd_path='/.'
root = os.listdir(r_path)
subdir=os.listdir(sd_path)
for entry in root:
# print(entry)
if os.path.isdir(os.path.join(r_path, entry)):
for subentry in subdir:
if os.path.isdir(os.path.join(r_path,'/ConfigurationsTest_19469')):
print(subentry)
For the second for loop, i want to iterate every folder that are in autotestlogs folder. i tried to make it but it doesnt work apparently. Please help Thanks
I think you messed your order up a little bit. If you do subdir = os.listdir(sd_path) before the loop you can't possibly get the sub directories because you need to use the parent directory to get to them.
So in your loop after you checked that an "entry" is a folder you can store the absolute path of this folder in a variable and then list it's contents with os.listdir().
Then you can loop through those and parse them.
How I would do it:
import os
r_path='//esw-fs01/esw_niagara_no_bck/BuildResults/master/0.1.52.68_390534/installation_area/autotestlogs_top/'
root = os.listdir(r_path)
for entry in root:
# print(entry)
subdir_path = os.path.join(r_path, entry) # create the absolute path of the subdir
if os.path.isdir(subdir_path): # check if it is a folder
subdir_entries = os.listdir(subdir_path) # get the content of the subdir
for subentry in subdir_entries:
subentry_path = os.path.join(subdir_path, subentry) # absolute path of the subentry
# here you can check everything you want for example if the subentry has a specific name etc
print(subentry_path)
I am attempting to write a program that loops through a bunch of files in a .pdb format and convert them to a .pdbqt format using a module called prepare_ligand4.py. I am fairly certain that it is correct up to the point that prepare_ligan4.py is called, but once it reaches this point all that happens is that the wordpad file pops up containing the code for prepare_ligand4.py. It should be modifying the files in the indicated directory instead. Does anyone have any advice what I should be doing? Is there a special way I need to call prepare_ligand4.py?
#convert pdb files to pdbqt
import os
import sys
#change directory to directory containing pdb files
os.chdir('C:\\Users\\Collin\\Documents\\fragments.pdb')
#path to pdb files
path = 'C:\\Users\\Collin\\Documents\\fragments.pdb'
dirs = os.listdir(path)
#finding number of pdb files in the directory
x = len(dirs)
#loop through all files in directory and convert to pdbqt
for i in range(x):
y = dirs[i]
os.system('C:\\Python27\\MGLTools-1.5.6\\Lib\\site-packages\\AutoDockTools\\Utilities24\\prepare_ligand4.py -l y -v')
ligand_pdbqt = y[:-4]+".pdbqt"
#os.rename(os.path.join ('C:\\Users\\Collin\\Documents\\fragments_under_150.pdb',y), os.path.join('C:\\Users\\Documents\\pdbqt', ligand_pdbqt)
I have a fun little script that i would like to make a copy of itself in a random directory - then run that copy of itself.
I know how to run files with (hacky):
os.system('Filename.py')
And i know how to replicate files with shuttle - but i am stuck at the random directory. Maybe if i could somehow get a list of all directories available and then pick one at random from the list - then remove this directory from the list?
Thanks,
Itechmatrix
You can get list of all dirs and subdirs, and shuffle it in random as follows:
import os
import random
all_dirs = [x[0] for x in os.walk('/tmp')]
random.shuffle(all_dirs)
for a_dir in all_dirs:
print(a_dir)
# do something witch each directory, e.g. copy some file there.
You can get a list of directories and then randomly select:
import os
import random
dirs = [d for d in os.listdir('.') if os.path.isdir(d)]
n = random.randrange(len(dirs))
print(dirs[n])
If you're on a Mac, there are a fair amount of hidden and restricted directories near the root. You can potentially run into errors with readability and writability. One way to get around that is to iterate through the available directories and sort all the no goes using the os module.
After that you can use the random.choice module to pick a random directory from that list.
import os, random
writing_dir = []
for directory in os.listdir():
if os.access(directory, W_OK) # W_OK ensures that the path is writable
writing_dir.append(directory)
path = random.choice(writing_dir)
I'm working on a similar script right now.
Say there's a directory hierarchy like this: .../A/B/main.py.
main.py is the file being executed, and I need to get the full path of folder A, how can I do that?
Use the os module:
import os
a_path = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
To get the path of the current directory, you can use:
import os
print os.path.realpath('.')
since . represents the current directory in the file system. So to get the path of the higher level directory, just replace . with .., which represents parent directory
import os
print os.path.realpath('..')
You can also use the os.getcwd() method to get the current working directory and then get its parent directory with the .. representation.
In Python 3:
from pathlib import Path
mypath = Path().absolute().parent.parent # each '.parent' goes one level up - vary as required
print(mypath)
How can I set the current path of my python file "myproject.py" to the file itself?
I do not want something like this:
path = "the path of myproject.py"
In mathematica I can set:
SetDirectory[NotebookDirectory[]]
The advantage with the code in Mathematica is that if I change the path of my Mathematica file, for example if I give it to someone else or I put it in another folder, I do not need to do anything extra. Each time Mathematica automatically set the directory to the current folder.
I want something similar to this in Python.
The right solution is not to change the current working directory, but to get the full path to the directory containing your script or module then use os.path.join to build your files path:
import os
ROOT_PATH = os.path.dirname(os.path.abspath(__file__))
# then:
myfile_path = os.path.join(ROOT_PATH, "myfile.txt")
This is safer than messing with current working directory (hint : what would happen if another module changes the current working directory after you did but before you access your files ?)
I want to set the directory in which the python file is, as working directory
There are two step:
Find out path to the python file
Set its parent directory as the working directory
The 2nd is simple:
import os
os.chdir(module_dir) # set working directory
The 1st might be complex if you want to support a general case (python file that is run as a script directly, python file that is imported in another module, python file that is symlinked, etc). Here's one possible solution:
import inspect
import os
module_path = inspect.getfile(inspect.currentframe())
module_dir = os.path.realpath(os.path.dirname(module_path))
Use the os.getcwd() function from the built in os module also there's os.getcwdu() which returns a unicode object of the current working directory
Example usage:
import os
path = os.getcwd()
print path
#C:\Users\KDawG\Desktop\Python