my way of getting the name of the next file in a folder is just terrible but works.I would like to remove the requirement of calling the os.listdir function as it is slow in large folders but dont know how. Any help is welcome.
Note:
Fullpathtoimagefile is the old file
changeimage is the name of a function that gets the next file
def nextfile(self):
folder, filename = os.path.split(fullpathtoimagefile)
filelist = os.listdir(folder)
nextfile = filelist.index(filename) + 1
nextfilename = filelist[nextfile]
changeimage(os.path.join(folder, nextfilename))
tl;dr More efficient way to get the full name of the next file in a folder
Maybe you can work with this one out.
[(os.path.join(folder, d), d, os.path.isdir(os.path.join(folder, d ))) for d in os.listdir(folder)]
simple code..
import os
arq = (os.listdir(".")) # "." = directory name
for file in arq:
print(file)
Another example:
import os
arq = (os.listdir("."))
for file in arq:
if (os.path.splitext(file)[1] == ".jpg"): #if a JPG file, do something
print(file)
This kind of smells like an instance of the XY problem. It looks like you're trying to iterate over images in a folder - in that case, use the pathlib module (it's been around since Python 3.4):
from pathlib import Path
for path in Path("path/to/folder/with/images").glob("*.jpg"):
print(path)
Related
I Selected my needed Data Like This:
import pathlib
from pathlib import Path
import glob, os
folder = Path('D:/xyz/123/Files')
os.chdir(folder)
for file in glob.glob("*.json"):
JsonFiles = os.path.join(folder, file)
print (JsonFiles)
As Output I will get all my needed .json Files
D:/xyz/123/Files/Data.json
D:/xyz/123/Files/Stuff.json
D:/xyz/123/Files/Random.json
D:/xyz/123/Files/Banana.json
D:/xyz/123/Files/Apple.json
For my further coding in need a Variable to the diffrens Json Paths. So my Idear was insted of printing them to store them in a List. But thats not working?
ListJson =[JsonFiles]
print (ListJson[1])
I get this Error:
print (ListJson[2])
IndexError: list index out of range
How wold you solve this Problem I just need an possibility to work with the Paths I already sorted.
Solution 1 with append():
If you change
for file in glob.glob("*.json"):
JsonFiles = os.path.join(folder, file)
print (JsonFiles)
to
ListJson = []
for file in glob.glob("*.json"):
JsonFile = os.path.join(folder, file)
ListJson.append(JsonFile)
You create an empty list and add during each iteration one file (the result of os.path.join to it.
then you would have what you want
Solution 2 with list comprehensions:
If you want to use list comprehensions ( https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions ) , then following would do:
ListJson = [os.path.join(folder, file) for file in glob.glob("*.json")]
Solution 3: with pathlib, absolute() and glob():
On the other hand you don't even have to chdir() into the given directory and if path objects are good enough in your context you could directly do:
ListJson = list(folder.absolute().glob('*.json'))
or if you really need strings:
ListJson = [str(path) for path in folder.absolute().glob('*.json')]
You can use Python List append() Method
import pathlib
from pathlib import Path
import glob, os
folder = Path('D:/xyz/123/Files')
ListJson=[]
os.chdir(folder)
for file in glob.glob("*.json"):
JsonFiles = os.path.join(folder, file)
ListJson.append(JsonFiles)
print(ListJson)
I have been working this challenge for about a day or so. I've looked at multiple questions and answers asked on SO and tried to 'MacGyver' the code used for my purpose, but still having issues.
I have a directory (lets call it "src\") with hundreds of files (.txt and .xml). Each .txt file has an associated .xml file (let's call it a pair). Example:
src\text-001.txt
src\text-001.xml
src\text-002.txt
src\text-002.xml
src\text-003.txt
src\text-003.xml
Here's an example of how I would like it to turn out so each pair of files are placed into a single unique folder:
src\text-001\text-001.txt
src\text-001\text-001.xml
src\text-002\text-002.txt
src\text-002\text-002.xml
src\text-003\text-003.txt
src\text-003\text-003.xml
What I'd like to do is create an associated folder for each pair and then move each pair of files into its respective folder using Python. I've already tried working from code I found (thanks to a post from Nov '12 by Sethdd, but am having trouble figuring out how to use the move function to grab pairs of files. Here's where I'm at:
import os
import shutil
srcpath = "PATH_TO_SOURCE"
srcfiles = os.listdir(srcpath)
destpath = "PATH_TO_DEST"
# grabs the name of the file before extension and uses as the dest folder name
destdirs = list(set([filename[0:9] for filename in srcfiles]))
def create(dirname, destpath):
full_path = os.path.join(destpath, dirname)
os.mkdir(full_path)
return full_path
def move(filename, dirpath):
shutil.move(os.path.join(srcpath, filename)
,dirpath)
# create destination directories and store their names along with full paths
targets = [
(folder, create(folder, destpath)) for folder in destdirs
]
for dirname, full_path in targets:
for filename in srcfile:
if dirname == filename[0:9]:
move(filename, full_path)
I feel like it should be easy, but Python isn't something I work with everyday and it's been a while since my scripting days... Any help would be greatly appreciated!
Thanks,
WK2EcoD
Use the glob module to interate all of the 'txt' files. From that you can parse and create the folders and copy the files.
The process should be as simple as it appears to you as a human.
for file_name in os.listdir(srcpath):
dir = file_name[:9]
# if dir doesn't exist, create it
# move file_name to dir
You're doing a lot of intermediate work that seems to be confusing you.
Also, insert some simple print statements to track data flow and execution flow. It appears that you have no tracing output so far.
You can do it with os module. For every file in directory check if associated folder exists, create if needed and then move the file. See the code below:
import os
SRC = 'path-to-src'
for fname in os.listdir(SRC):
filename, file_extension = os.path.splitext(fname)
if file_extension not in ['xml', 'txt']:
continue
folder_path = os.path.join(SRC, filename)
if not os.path.exists(folder_path):
os.mkdir(folderpath)
os.rename(
os.path.join(SRC, fname),
os.path.join(folder_path, fname)
)
My approach would be:
Find the pairs that I want to move (do nothing with files without a pair)
Create a directory for every pair
Move the pair to the directory
#! /usr/bin/env python
# -*- coding: utf-8 -*-
import os, shutil
import re
def getPairs(files):
pairs = []
file_re = re.compile(r'^(.*)\.(.*)$')
for f in files:
match = file_re.match(f)
if match:
(name, ext) = match.groups()
if ext == 'txt' and name + '.xml' in files:
pairs.append(name)
return pairs
def movePairsToDir(pairs):
for name in pairs:
os.mkdir(name)
shutil.move(name+'.txt', name)
shutil.move(name+'.xml', name)
files = os.listdir()
pairs = getPairs(files)
movePairsToDir(pairs)
NOTE: This script works when called inside the directory with the pairs.
My data is organized as such:
I have 30 folders. In each of them, 3 subfolders. In each of them, one file.
I would like to write a script that writes, in a text file 1 located in folder 1, the paths to the files located in the subfolders of this folder 1; and so on for every other folder.
The problem is that the script only writes, in each text file, the 3rd file (file in subfolder 3) rather than the files in subfolders 1, 2, 3.
This is what I tried:
import glob
import os
gotofolders = '/path/to/folderslocation/'
foldersname = open('/path/to/foldersname.txt').read().split()
for folders in foldersname:
foldersdirectory = os.path.join(gotofolders,foldersname)
filepaths = glob.glob(os.path.join(foldersdirectory)+'*subfolders/*files')
for filepath in filepaths:
savethepaths = os.path.join(foldersdirectory)+'files_path_in_that_folder.txt'
with open (savethepaths,'w') as f:
f.write(filepath+'\n')
As said, it almost works, excepts that in each 'files_path_in_that_folder.txt' I have the 3rd element of the "filepath" list, rather than all 3 elements.
Thanks!
Okay, I figured it out; I had to add:
with open (savethepaths,'w') as f:
f.writelines(list("%s\n" %filepath for filepath in filepaths))
import os
def directory_into_file(_path, file_obj, depth):
# depth is a string of asterisk, just for better printing. starts with empty string
file_obj.write(depth + _path + '\n')
if(os.path.isdir(_path)):
file_list = os.listdir(_path)
os.chdir(_path)
for file in file_list:
directory_into_file(file, file_obj, depth+'*')
os.chdir("..")
this should work.
_path - the path of the directory,
file_obj - send the object file to the function and first,
depth - at first call send an empty string
hope this would work. didn't try it myself...
I was messing around just trying to make a script that deletes items by ".zip" extension.
import sys
import os
from os import listdir
test=os.listdir("/Users/ben/downloads/")
for item in test:
if item.endswith(".zip"):
os.remove(item)
Whenever I run the script I get:
OSError: [Errno 2] No such file or directory: 'cities1000.zip'
cities1000.zip is obviously a file in my downloads folder.
What did I do wrong here? Is the issue that os.remove requires the full path to the file? If this is the issue, than how can I do that in this current script without completely rewriting it.
You can set the path in to a dir_name variable, then use os.path.join for your os.remove.
import os
dir_name = "/Users/ben/downloads/"
test = os.listdir(dir_name)
for item in test:
if item.endswith(".zip"):
os.remove(os.path.join(dir_name, item))
For this operation you need to append the file name on to the file path so the command knows what folder you are looking into.
You can do this correctly and in a portable way in python using the os.path.join command.
For example:
import os
directory = "/Users/ben/downloads/"
test = os.listdir( directory )
for item in test:
if item.endswith(".zip"):
os.remove( os.path.join( directory, item ) )
Alternate approach that avoids join-ing yourself over and over: Use glob module to join once, then let it give you back the paths directly.
import glob
import os
dir = "/Users/ben/downloads/"
for zippath in glob.iglob(os.path.join(dir, '*.zip')):
os.remove(zippath)
I think you could use Pathlib-- a modern way, like the following:
import pathlib
dir = pathlib.Path("/Users/ben/downloads/")
zip_files = dir.glob(dir / "*.zip")
for zf in zip_files:
zf.unlink()
If you want to delete all zip files recursively, just write so:
import pathlib
dir = pathlib.Path("/Users/ben/downloads/")
zip_files = dir.rglob(dir / "*.zip") # recursively
for zf in zip_files:
zf.unlink()
Just leaving my two cents on this issue: if you want to be chic you can use glob or iglob from the glob package, like so:
import glob
import os
files_in_dir = glob.glob('/Users/ben/downloads/*.zip')
# or if you want to be fancy, you can use iglob, which returns an iterator:
files_in_dir = glob.iglob('/Users/ben/downloads/*.zip')
for _file in files_in_dir:
print(_file) # just to be sure, you know how it is...
os.remove(_file)
origfolder = "/Users/ben/downloads/"
test = os.listdir(origfolder)
for item in test:
if item.endswith(".zip"):
os.remove(os.path.join(origfolder, item))
The dirname is not included in the os.listdir output. You have to attach it to reference the file from the list returned by said function.
Prepend the directory to the filename
os.remove("/Users/ben/downloads/" + item)
EDIT: or change the current working directory using os.chdir.
I would like to edit the file name of several files in a list of folders and export the entire file to a new folder. While I was able to rename the file okay, the contents of the file didn't migrate over. I think I wrote my code to just create a new empty file rather than edit the old one and move it over to a new directory. I feel that the fix should be easy, and that I am missing a couple of important lines of code. Below is what I have so far:
import libraries
import os
import glob
import re
directory
directory = glob.glob('Z:/Stuff/J/extractions/test/*.fsa')
The two files in the directory look like this when printed out
Z:/Stuff/J/extractions/test\c2_D10.fsa
Z:/Stuff/J/extractions/test\c3_E10.fsa
for fn in directory:
print fn
this script was designed to manipulate the file name and export the manipulated file to a another folder
for fn in directory:
output_directory = 'Z:/Stuff/J/extractions/test2'
value = os.path.splitext(os.path.basename(fn))[0]
matchObj = re.match('(.*)_(.*)', value, re.M|re.I)
new_fn = fn.replace(str(matchObj.group(0)), str(matchObj.group(2)) + "_" + str(matchObj.group(1)))
base = os.path.basename(new_fn)
v = open(os.path.join(output_directory, base), 'wb')
v.close()
My end result is the following:
Z:/Stuff/J/extractions/test2\D10_c2.fsa
Z:/Stuff/J/extractions/test2\E10_c3.fsa
But like I said the files are empty (0 kb) in the output_directory
As Stefan mentioned:
import shutil
and replace:
v = open(os.path.join(output_directory, base), 'wb')
v.close()
with:
shutil.copyfile (fn, os.path.join(output_directory, base))
If I'am not wrong, you are only opening the file and then you are immediately closing it again?
With out any writing to the file it is surely empty.
Have a look here:
http://docs.python.org/2/library/shutil.html
shutil.copyfile(src, dst) ;)