I want to be able to write a script in python that will create a certain number of files/folders recursively to a certain location. So for example I want some sort of for loop that will create folder1, folder2, folder3 to a path C:\Temp. I assume a for loop is the best way to do this and using os also? Any help would be great!
Have a look at makedirs. It may help.
Here is how to create a folder recursively and then create a file in that folder.
from pathlib import Path
import os
folder_path = Path(os.getcwd() + os.path.join('\my\folders')) #define folder structure
if not os.path.exists(path): # create folders if not exists
os.makedirs(path)
file_path = os.path.join(path, 'file.xlsx') # add file to the folder path
f= open(file,"w+") # open in w+(write mode) or a+(append mode)
Since Python 3.4 you can use the pathlib to create a folder and all parent folders:
from pathlib import Path
Path("my/path/to/create").mkdir(parents=True, exist_ok=True)
If you want to raise an error on existent folder you can set exist_ok to False.
Related
how can i copy only the structure (excluding the files) of a path with all subfolders?
For example
path1\subfolder1\subsubfolder1
\subsubfolder2
\subfolder2\subsubfolder1
\subsubfolder2
copy to
path2\subfolder1\subsubfolder1
\subsubfolder2
\subfolder2\subsubfolder1
\subsubfolder2
I need this because i am loading pickles from path1 and calculating some things. The save need to be to a new path so i can compare the changes.
Already tried it with path1.split('/') but this only works if path1 is not changing so i need to change this everytime for new calculations if I want to change the pickles.
You can use os.walk to recursively search through all sub folders for the given path.
Then for each subfolder found you can replace the input path with the output path and use the os.makedirs to create the folders if they do not exist yet.
import os
def copy_folder_structure(input_dir, output_dir):
all_subfolders = [x[0] for x in os.walk(input_dir)]
for folder in all_subfolders:
output_folder = folder.replace(input_dir, output_dir)
if not os.path.exists(output_folder):
os.makedirs(output_folder)
copy_folder_structure("path1", "path2")
I have the following directory, in the parent dir there are several folders lets say ABCD and within each folder many zips with names as displayed and the letter of the parent folder included in the name along with other info:
-parent--A-xxxAxxxx_timestamp.zip
-xxxAxxxx_timestamp.zip
-xxxAxxxx_timestamp.zip
--B-xxxBxxxx_timestamp.zip
-xxxBxxxx_timestamp.zip
-xxxBxxxx_timestamp.zip
--C-xxxCxxxx_timestamp.zip
-xxxCxxxx_timestamp.zip
-xxxCxxxx_timestamp.zip
--D-xxxDxxxx_timestamp.zip
-xxxDxxxx_timestamp.zip
-xxxDxxxx_timestamp.zip
I need to unzip only selected zips in this tree and place them in the same directory with the same name without the .zip extension.
Output:
-parent--A-xxxAxxxx_timestamp
-xxxAxxxx_timestamp
-xxxAxxxx_timestamp
--B-xxxBxxxx_timestamp
-xxxBxxxx_timestamp
-xxxBxxxx_timestamp
--C-xxxCxxxx_timestamp
-xxxCxxxx_timestamp
-xxxCxxxx_timestamp
--D-xxxDxxxx_timestamp
-xxxDxxxx_timestamp
-xxxDxxxx_timestamp
My effort:
for path in glob.glob('./*/xxx*xxxx*'): ##walk the dir tree and find the files of interest
zipfile=os.path.basename(path) #save the zipfile path
zip_ref=zipfile.ZipFile(path, 'r')
zip_ref=extractall(zipfile.replace(r'.zip', '')) #unzip to a folder without the .zip extension
The problem is that i dont know how to save the A,B,C,D etc to include them in the path where the files will be unzipped. Thus, the unzipped folders are created in the parent directory. Any ideas?
The code that you have seems to be working fine, you just to make sure that you are not overriding variable names and using the correct ones. The following code works perfectly for me
import os
import zipfile
import glob
for path in glob.glob('./*/xxx*xxxx*'): ##walk the dir tree and find the files of interest
zf = os.path.basename(path) #save the zipfile path
zip_ref = zipfile.ZipFile(path, 'r')
zip_ref.extractall(path.replace(r'.zip', '')) #unzip to a folder without the .zip extension
Instead of trying to do it in a single statement , it would be much easier and more readable to do it by first getting list of all folders and then get list of files inside each folder. Example -
import os.path
for folder in glob.glob("./*"):
#Using *.zip to only get zip files
for path in glob.glob(os.path.join(".",folder,"*.zip")):
filename = os.path.split(path)[1]
if folder in filename:
#Do your logic
I am new to python. I have successful written a script to search for something within a file using :
open(r"C:\file.txt) and re.search function and all works fine.
Is there a way to do the search function with all files within a folder? Because currently, I have to manually change the file name of my script by open(r"C:\file.txt),open(r"C:\file1.txt),open(r"C:\file2.txt)`, etc.
Thanks.
You can use os.walk to check all the files, as the following:
import os
for root, _, files in os.walk(path):
for filename in files:
with open(os.path.join(root, filename), 'r') as f:
#your code goes here
Explanation:
os.walk returns tuple of (root path, dir names, file names) in the folder, so you can iterate through filenames and open each file by using os.path.join(root, filename) which basically joins the root path with the file name so you can open the file.
Since you're a beginner, I'll give you a simple solution and walk through it.
Import the os module, and use the os.listdir function to create a list of everything in the directory. Then, iterate through the files using a for loop.
Example:
# Importing the os module
import os
# Give the directory you wish to iterate through
my_dir = <your directory - i.e. "C:\Users\bleh\Desktop\files">
# Using os.listdir to create a list of all of the files in dir
dir_list = os.listdir(my_dir)
# Use the for loop to iterate through the list you just created, and open the files
for f in dir_list:
# Whatever you want to do to all of the files
If you need help on the concepts, refer to the following:
for looops in p3: http://www.python-course.eu/python3_for_loop.php
os function Library (this has some cool stuff in it): https://docs.python.org/2/library/os.html
Good luck!
You can use the os.listdir(path) function:
import os
path = '/Users/ricardomartinez/repos/Salary-API'
# List for all files in a given PATH
file_list = os.listdir(path)
# If you want to filter by file type
file_list = [file for file in os.listdir(path) if os.path.splitext(file)[1] == '.py']
# Both cases yo can iterate over the list and apply the operations
# that you have
for file in file_list:
print(file)
#Operations that you want to do over files
Say I have a path to a file:
/path/to/some/directory/file.ext
In python, I'd like to create a symlink in the same directory as the file, that
points to the file. I'd like to end up with this:
/path/to/some/directory/symlink -> file.ext
I can do this fairly easily using os.chdir() to cd into the directory and
create the symlinks. But os.chdir() is not thread safe, so I'd like to avoid
using it. Assuming that the current working directory of the process is not
the directory with the file (os.getcwd() != '/path/to/some/directory'),
what's the best way to do this?
I guess I could create a busted link in whatever directory I'm in, then
move it to the directory with the file:
import os, shutil
os.symlink('file.ext', 'symlink')
shutil.move('symlink', '/path/to/some/directory/.')
Is there a better way to do this?
Note, I don't want to end up with is this:
/path/to/some/directory/symlink -> /path/to/some/directory/file.ext
You can also use os.path.relpath() so that you can use symlinks with relative paths. Say your script is in a directory foo/ and this directory has subdirectories src/ and dst/, and you want to create relative symlinks in dst/ to point to the files in src/. To do so, you can do:
import os
from glob import glob
for src_path in glob('src/*'):
os.symlink(
os.path.relpath(
src_path,
'dst/'
),
os.path.join('dst', os.path.basename(src_path))
)
Listing the contents of dst/ then shows:
1.txt -> ../src/1.txt
2.txt -> ../src/2.txt
Relative symlinks are useful for if you want to create a tarball of the whole foo directory tree, as I don't believe tar updates symlinks to point to the relative path inside of the generated tarball.
You could just set the second argument to the destination, like:
import os
os.symlink('file.ext', '/path/to/some/directory/symlink')
python function to create a relative symlink:
def relative_symlink(src, dst):
dir = os.path.dirname(dst)
Src = os.path.relpath(src, dir)
Dst = os.path.join(dir, os.path.basename(src))
return os.symlink(Src, Dst)
Nowadays, this can be accomplished using pathlib
from pathlib import Path
target = Path('../target.txt')
my_symlink = Path('symlink.txt')
my_symlink.symlink_to(target)
where target is a relative Path or str.
Given the following strings:
dir/dir2/dir3/dir3/file.txt
dir/dir2/dir3/file.txt
example/directory/path/file.txt
I am looking to create the correct directories and blank files within those directories.
I imported the os module and I saw that there is a mkdir function, but I am not sure what to do to create the whole path and blank files. Any help would be appreciated. Thank you.
Here is the answer on all your questions (directory creation and blank file creation)
import os
fileList = ["dir/dir2/dir3/dir3/file.txt",
"dir/dir2/dir3/file.txt",
"example/directory/path/file.txt"]
for file in fileList:
dir = os.path.dirname(file)
# create directory if it does not exist
if not os.path.exists(dir):
os.makedirs(dir)
# Create blank file if it does not exist
with open(file, "w"):
pass
First of all, given that try to create directory under a directory that doesn't exist, os.mkdir will raise an error. As such, you need to walk through the paths and check whether each of the subdirectories has or has not been created (and use mkdir as required). Alternative, you can use os.makedirs to handle this iteration for you.
A full path can be split into directory name and filename with os.path.split.
Example:
import os
(dirname, filename) = os.path.split('dir/dir2/dir3/dir3/file.txt')
os.makedirs(dirname)
Given we have a set of dirs we want to create, simply use a for loop to iterate through them. Schematically:
dirlist = ['dir1...', 'dir2...', 'dir3...']
for dir in dirlist:
os.makedirs( ... )