I am an absolute beginner to programming so I apologize if this is really basic. I've looked at other questions that appear to be related, but haven't found a solution to this particular problem--at least not that I can understand.
I need to generate a list of files in a directory; create a separate directory for each of those files with the directory name being based on each file's name; and put each file in its corresponding directory.
You should have a look at the glob, os and shutil libraries.
I've written an example for you. This will remove the file extension of each file in a given folder, create a new subdirectory, and move the file into the corresponding folder, i.e.:
C:\Test\
-> test1.txt
-> test2.txt
will become
C:\Test\
-> test1\
-> test1.txt
-> test2\
-> test2.txt
Code:
import glob, os, shutil
folder = 'C:/Test/'
for file_path in glob.glob(os.path.join(folder, '*.*')):
new_dir = file_path.rsplit('.', 1)[0]
os.mkdir(os.path.join(folder, new_dir))
shutil.move(file_path, os.path.join(new_dir, os.path.basename(file_path)))
This will throw an error if the folder already exist. To avoid that, handle the exception:
import glob, os, shutil
folder = 'C:/Test/'
for file_path in glob.glob(os.path.join(folder, '*.*')):
new_dir = file_path.rsplit('.', 1)[0]
try:
os.mkdir(os.path.join(folder, new_dir))
except WindowsError:
# Handle the case where the target dir already exist.
pass
shutil.move(file_path, os.path.join(new_dir, os.path.basename(file_path)))
PS: This will not work for files without extensions. Consider using a more robust code for cases like that.
Here is some advice on listing files using Python.
To create a directory, use os.mkdir (docs). To move a file, use os.rename (docs) or shutil.move (docs).
Related
I am struggling with the paths and directories to solve this problem. Basically, I have a long list of .lammps files in one directory. My goal is to copy each file and move it into its own folder (which is one directory back) where its folder name is the file name minus the .lammps. All of the folders are already made, I just can't seem to figure out moving them. The entire list of files is in the Files directory. The individual folders are in the ROTATED FILES directory. Here is what I have. Any tips greatly appreciated.
Here is a file example
n-optimized.new.10_10-90-10_10.Ni00Nj01.lammps
The folder for this file is then named
n-optimized.new.10_10-90-10_10.Ni00Nj01
import os
file_directory = os.chdir("C:\Py Practice\ROTATED FILES\Files")
files = os.listdir()
for file in files:
# get the file -.lammps string
name1 = file.split('.')[0:4]
name2 = ".".join(name1)
# get the path for the files new respective folder (back a directory and paste folder name)
file_folder = "C:\Py Practice\ROTATED FILES/" + name2
# Move
combined_path = os.path.join(file, file_folder)
I've tried shutil and figured path join might be easier.
First of all, the code you have here shouldn't work since you either have to escape backslashes or use a raw string. Secondly, rather than using os for file system operations, it's much better to learn how to use pathlib (also a core python module) which provides a more modern object-oriented approach to file operations.
Using pathlib and shutil you can do something like
from pathlib import Path
from shutil import copyfile
file_directory = Path(r"C:\Py Practice\ROTATED FILES\Files")
# get the list of source files
source_files = [f for f in file_directory.glob('*.lammps')]
# create target file paths
target_files = [file_directory.parent / f.stem/ f.name for f in source_files]
for source, target in zip(source_files, target_files):
copyfile(str(source), str(target))
Here we're accessing different parts of file path using a convenient OOP structure. For example, if your file f is located in 'c:/foo/bar/boo.txt' then f.name is just the name of file: boo.txt, f.stem is the stem part of the file name (excluding the extension) boo, f.parent is its parent directory 'c:/foo/bar/' etc.
There's a really handy graphic of pathlib Path objects here.
The only inconvenience is that not all of core modules support Path objects yet so for copyfile we just need to get the string representation by calling str on the object.
And you don't even need to have target folders created beforehand, it's very easy to create the necessary folder structure as you go along:
from pathlib import Path
from shutil import copyfile
file_directory = Path(r"C:\Py Practice\ROTATED FILES\Files")
# get the list of source files
source_files = [f for f in file_directory.glob('*.lammps')]
# create target file paths
target_files = [file_directory.parent / f.stem/ f.name for f in source_files]
for source, target in zip(source_files, target_files):
# check that target directory exists
# and create a folder if not
if not target.parent.is_dir():
target.parent.mkdir()
copyfile(str(source), str(target))
I'm still very new to Python so I'm trying to apply Python in to my own situation for some experience
One useful program is to delete files, in this case by file type from a directory
import os
target = "H:\\documents\\"
for x in os.listdir(target):
if x.endswith(".rtf"):
os.unlink(target + x)
Taking this program, I have tried to expand it to delete ost files in every local profiles:
import os
list = []
folder = "c:\\Users"
for subfolder in os.listdir(folder):
list.append(subfolder)
ost_folder = "c:\\users\\%s\\AppData\\Local\\Microsoft\\Outlook"
for users in list:
ost_list = os.listdir(ost_folder%users)
for file in ost_list:
if file.endswith(".txt"):
print(file)
This should be printing the file name but spits an error that the file directory cannot be found
Not every folder under C:\Users will have a AppData\Local\Microsoft\Outlook subdirectory (there are typically hidden directories there that you may not see in Windows Explorer that don't correspond to a real user, and have never run Outlook, so they don't have that folder at all, but will be found by os.listdir); when you call os.listdir on such a directory, it dies with the exception you're seeing. Skip the directories that don't have it. The simplest way to do so is to have the glob module do the work for you (which avoids the need for your first loop entirely):
import glob
import os
for folder in glob.glob(r"c:\users\*\AppData\Local\Microsoft\Outlook"):
for file in os.listdir(folder):
if file.endswith(".txt"):
print(os.path.join(folder, file))
You can simplify it even further by pushing all the work to glob:
for txtfile in glob.glob(r"c:\users\*\AppData\Local\Microsoft\Outlook\*.txt"):
print(txtfile)
Or do the more modern OO-style pathlib alternative:
for txtfile in pathlib.Path(r'C:\Users').glob(r'*\AppData\Local\Microsoft\Outlook\*.txt'):
print(txtfile)
I want to be able to take a file and duplicate it in the same directory with a different name, but I can't figure out how to change the name while copying so it doesn't give me and error that the file already exists or changes the name of the original file.
shutil.copy() can do that if you specify the destination filename as well as destination folder:
import shutil
shutil.copy(r'c:\temp\file1.txt', r'c:\temp\file2.txt')
Adding on to TessellatingHeckler's answer, if you have a file with an arbitrary directory, you can use os.path.dirname and os.path.join to create a new filename in the same directory:
import os
import shutil
original = r'c:\temp\file1.txt'
original_dir = os.path.dirname(original) # r'c:\temp'
new_name = 'file2.txt'
new_path = os.path.join(original_dir, new_name) # r'c:\temp\file2.txt'
shutil.copy(original, new_path)
You could of course do that more compactly if you wish. You'll also notice from the shutil.copy() docs that shutil features multiple different copying methods, each one with pros and cons. For example, shutil.copy2() attempts to preserve metadata. It's up to you to decide which method is best for your situation.
I have a folder with 7500 images. I need to copy the first 600 images to a new folder using the shutil module in Python.
I tried to look for relevant stuff on the net but the usage of the paths is a bit confusing. What exactly should be my sequence of commands? I guess it will start like:
import os
import shutil
l=os.listdir(path)
for file in l[0:600]:
Edit: after having clarification on what shutil.copy() does, I came up with:
import os
import shutil
l=os.listdir(path)
for file in l[0:600]:
shutil.copy(file, destination, *, follow_symlinks = True)
But it's highlighting the comma after *, and giving the error iterable argument unpacking follows keyword argument unpacking. What's going wrong in the syntax?
Well, os.listdir() will return files randomly sorted, one thing you could do is that you can call os.stat(file).st_mtime on each file which will return timestamp when that file was last modified and then you can sort the files by that time to get first/last files. But it really depends on your use-case and how you interpret what first files are for you. But when it comes to shutil library you can just call:
for file in l[0:600]:
shutil.copy(file, f'./destination/{file}')
which will copy 600 files into directory that is in your current directory and named 'destination'.
os.listdir(path) will list files and sub direcotries in your directory you're searching.
I'm making the assumption that all you're files will be .jpg so I would use the glob module.
import glob
path = "D:Pictures\*.jpg"
destination = r"E:\new_pictures\\"
files = glob.glob(path)
for f in sorted(files)[:600]:
shutil.copy(f, destination)
Hi is there a function that copies a parent folder along with all its content to a specified destination in python.
I have used different functions but they seem to copy the contents excluding the parent folder.
Many thanks
shutil.copytree comes to mind immediately, but your issue is that copying directory foo in bar doesn't create bar/foo.
My proposal:
import shutil,os
def copytree2(source,dest):
os.mkdir(dest)
dest_dir = os.path.join(dest,os.path.basename(source))
shutil.copytree(source,dest_dir)
first create destination
then generate destination dir, which is the destination added source basename
perform copytree with the new destination, so source folder name level appears under dest
There's no subtle check about dest directory already exist or whatnot. I'll let you add that if needed (using os.path.isdir(dest) for instance)
Note that functions from shutil come with a note which encourages users to copy and modify them to better suit their needs.
In python 3.* we can use shutil
import shutil
old_folder = "D:/old_folder"
new_folder = "D:/new_folder"
shutil.copytree(old_folder, new_folder, dirs_exist_ok=True)
dirs_exist_ok=True is for ignoring the exception when the folder already exists in the new location.
Simply append the source directory you want to copy in the destination:
import shutil
shutil.copytree("source", "destination/source")
If you do not have fixed strings then use os.path.basename() to determine basename and combine it in the destination with os.path.join()
import os.path
import shutil
source = "/Projekte/python/source"
shutil.copytree(source, os.path.join("destination", os.path.basename(source)))
import shutil
shutil.copytree(srcDir, dst, symlinks=False, ignore=None)