Using python v 3.7.3, pytorch v 0.4.1, imgaug 0.3.0, windows 10, Jupyter Notebook
I am trying to iterate through several folders containing images, augment each image 6 times, then save a hard copy of each augmented image inside that folder. I am using the imgaug library to augment the images.
I am able to iterate through the folders, and augment and display the images inside the folders with this code:
for folder in os.listdir(path):
for i in os.listdir(path + '\\' + folder):
img = imageio.imread(path + '\\' + folder + '\\' + i)
print('Original:')
ia.imshow(img)
img_aug = seq.augment_image(img)
print('Augmented:')
ia.imshow(img_aug)
But, I would like to ultimately augment each image 6 times and create 6 new hard files per image. I am trying to use this tutorial to make these changes.
Right now, I am just trying the step to save a hard copy of the augmented images. Using this code:
for folder in os.listdir(path):
for i in os.listdir(path + '\\' + folder):
img = imageio.imread(path + '\\' + folder + '\\' + i)
print('Original:')
ia.imshow(img)
img_aug = seq.augment_image(img)
print('Augmented:')
ia.imshow(img_aug)
for im, im_aug in enumerate(img_aug):
imageio.imwrite(os.path.join(path, path + '\\' + folder + '\\' + folder + "%06d.png" % (im)), im_aug)
While the augmented images show up normally when I print them in Jupyter labs, they are being saved as a hard copy as completely flat. It's also saving hundreds of these images:
Why would my image show up correctly augmented in Jupyter Labs, but be saved in that format when I try to save a hard copy?
Answered here, needed to be incremented:
for folder in os.listdir(path):
i = 0
for fname in os.listdir(path + '\\' + folder):
img = imageio.imread(path + '\\' + folder + '\\' + fname)
print('Original:')
ia.imshow(img)
img_aug = seq.augment_image(img)
print('Augmented:')
ia.imshow(img_aug)
imageio.imwrite(os.path.join(path, path + '\\' + folder + '\\' + folder + "%06d.png" % (i,)), img_aug)
i += 1
Related
I am really looking for a solution to this problem. I have been generating frames from a python code which i end up having into different folders as shown below,
**folder names** swimming_0, swimming_1, noswimming_0, noswimming_1.
swimming_0 noswimming_0
frame0.jpg frame0.jpg
frame1.jpg frame1.jpg
frame2.jpg frame2.jpg
swimming_1 noswimming_1
frame0.jpg frame0.jpg
frame1.jpg frame1.jpg
frame2.jpg frame2.jpg
What i expect:
In a single folder
swimming_0_0.jpg
swimming_0_1.jpg
swimming_0_2.jpg
swimming_1_0.jpg
swimming_1_1.jpg
swimming_1_2.jpg
noswimming_0_0.jpg
noswimming_0_1.jpg
noswimming_0_2.jpg
noswimming_1_0.jpg
noswimming_1_1.jpg
noswimming_1_2.jpg
The steps we need is that:
1. rename the images in each folder
2. combine the images
Python code used:
step1: renaming and adding incremental numbers
import os
folderpath = r'/swimming'
fileNumber = 1
for filename in os.listdir(folderpath):
os.rename(folderpath + '//' + filename, folderpath + "_" + str(fileNumber))
fileNumber +=1
Although this code works with my expectations for step 1 however I am looking for a more efficient way to do this. Could someone please help here.
Update:
It is also fine if we dont keep track of the filenames but atleast in that case i would expect the results to be.
swimming_0.jpg
swimming_1.jpg
swimming_2.jpg
swimming_3.jpg
swimming_4.jpg
swimming_5.jpg
noswimming_6.jpg
noswimming_7.jpg
noswimming_8.jpg
noswimming_9.jpg
noswimming_10.jpg
noswimming_11.jpg
Ok, so if you want to move files in separate folders to one you could try this:
First of you should have a structure like this:
from_here/
+ noswimming_0/
+ frame0.png
+ frame1.png
+ frame2.png
+ noswimming_1/
+ frame0.png
+ frame1.png
+ frame2.png
+ swimming_0/
+ frame0.png
+ frame1.png
+ frame2.png
+ swimming_1/
+ frame0.png
+ frame1.png
+ frame2.png
move_here/
script.py
script.py:
import os
from_path = os.getcwd() + '/from_here'
to_path = os.getcwd() + '/move_here'
for folder in os.listdir(from_path):
for file in os.listdir(os.path.join(from_path, folder)):
os.rename(os.path.join(from_path, folder, file), os.path.join(to_path, f'{folder}_{file}'))
This should now move the files from those separate folders to the one (obviously you could have more folders to move from or more files in those folders)
Trying to retrieve all the paths of the pngs in different sub folders.
All sub folders are located within a main folder - logs.
pngs = []
for idx, device in enumerate(udid):
pngs += glob.glob(os.getcwd() + "/logs/" + device + "_" + get_model_of_android_phone(device) + "/" + "*.png")
File structure
logs/123456789_SM-G920I/123456789google_search_android.png
The values in bold will change. I have added in *.png for the changing pngs.
But how do i get the paths of the pngs when i do not have an absolute path to the png file?
Update
get_model_of_android_phone(device) is a method to get the following value here.
E.g. 123456789_SM-G920I
I am thinking to remove it cause it is not really working as intended. Would like to replace the method with something like *
You can use following in simplified way to get all file names:
for name in glob.glob(os.getcwd() + "/logs/**/*.png", recursive=True):
print '\t', name
When recursive is set, ** will matches 0 or more subdirectories when followed by a separator.
If you just want to make list, use the following code snippet :
pngs = glob.glob(os.getcwd() + "/logs/**/*.png", recursive=True)
It will return a list of all png file paths.
Reference : https://docs.python.org/3/library/glob.html
for idx, device in enumerate(udid):
path_device = os.getcwd() + "/logs/" + device + "_" + get_model_of_android_phone(device) + "/"
file_list = os.listdir(path_device)
pngs = [path_device+file_png for file_png in file_list if str(file_png).endswith(".png")]
import os
src = "/home/user/Desktop/images/"
ext = ".jpg"
for i,filename in enumerate(os.listdir(src)):
# print(i,filename)
if filename.endswith(ext):
os.rename(src + filename, src + str(i) + ext)
print(filename, src + str(i) + ext)
else :
os.remove(src + filename)
this code will rename all the images in a folder starting with 0.jpg,1.jpg etc... and remove none jpg but what if i already had some images in that folder, let's say i had images 0.jpg, 1.jpg, 2.jpg, then i added a few others called im5.jpg and someImage.jpg.
What i want to do is adjust the code to read the value of the last image number, in this case 2 and start counting from 3 .
In other words i'll ignore the already labeled images and proceed with the new ones counting from 3.
Terse and semi-tested version:
import os
import glob
offset = sorted(int(os.path.splitext(os.path.basename(filename))[0])
for filename in glob.glob(os.path.join(src, '*' + ext)))[-1] + 1
for i, filename in enumerate(os.listdir(src), start=offset):
...
Provided all *.jpg files consist of a only a number before their extension. Otherwise you will get a ValueError.
And if there happens to be a gap in the numbering, that gap will not be filled with new files. E.g., 1.jpg, 2.jpg, 3.jpg, 123.jpg will continue with 124.jpg (which is safer anyway).
If you need to filter out filenames such as im5.jpg or someImage.jpg, you could add an if-clause to the list comprehension, with a regular expression:
import os
import glob
import re
offset = sorted(int(os.path.splitext(os.path.basename(filename))[0])
for filename in glob.glob(os.path.join(src, '*' + ext))
if re.search('\d+' + ext, filename))[-1] + 1
Of course, by now the three lines are pretty unreadable, and may not win the code beauty contest.
I've got some python 2.7.3 code on the Raspberry Pi B+ that loops and creates some dummy files. Unfortunately, the filenames get garbled somewhere between python and the filesystem.
settime = strftime("%Y-%m-%d %H:%M:%S")
for n in range(int(setlength)):
time.sleep(GetShutterSpeed(shutter))
image = 'img/'
image += settime
image += shutter + ' '
image += 'ISO' + iso
image += ' #' + str(n+1).rjust(len(setlength), '0')
image += ' of ' + setlength
image += '.jpg'
with open(image, 'w') as f:
f.write('')
ro(strftime("%H:%M:%S") + ' > ' + image)
Here's what I get when I browse to where the images are saved from a windows share over samba.
Why are these files named so strangely?
Is this because I'm trying to save text files named as .jpgs? Or am I messing up the character encoding somewhere? I don't have any problems opening files in the same python code that already exist. I tried removing the #s, but that didn't matter. If I pass in a /, it balks completely, which is as expected. Just can't figure out what I'm doing wrong with the filename itself.
I am trying to make a script which selects every .png file in a folder beginning with the letters "LG". I then want the scipt create a shapefile, replacing the "LG" with "SH", and then i want the script to buffer that shapefile and rename the buffer with the first 2 letters being "SB"!
I keep getting an error 99999 error message at line 37!
( gp.RasterToPolygon_conversion(INPUT_RASTER, Output_polygon_features, "SIMPLIFY", "VALUE") )
Can anyone see why this isnt working? I am very, very new to this and have been staring at this script pulling out my hair for days!!
Here is the script:
# Load required toolboxes...
gp.AddToolbox("C:/Program Files/ArcGIS/ArcToolbox/Toolboxes/Conversion Tools.tbx")
gp.AddToolbox("C:/Program Files/ArcGIS/ArcToolbox/Toolboxes/Analysis Tools.tbx")
# Script arguments...
folder = "D:\\J04-0083\\IMAGEFILES"
for root, dirs, filenames in os.walk(folder): # returms root, dirs, and files
for filename in filenames:
filename_split = os.path.splitext(filename) # filename and extensionname (extension in [1])
filename_zero = filename_split[0]
try:
first_2_letters = filename_zero[0] + filename_zero[1]
except:
first_2_letters = "XX"
if first_2_letters == "LG":
Output_polygon_features = "D:\\J04-0083\\ShapeFiles.gdb\\" + "SH_" + filename + ".shp"
# Process: Raster to Polygon...
INPUT_RASTER = os.path.join(root + "\\" + filename_zero + ".png")
gp.RasterToPolygon_conversion(INPUT_RASTER, Output_polygon_features, "SIMPLIFY", "VALUE")
Distance__value_or_field_ = "5 Meters"
Raster_Buffer_shp = "SB_" + filename + ".shp"
# Process: Buffer...
gp.Buffer_analysis(Output_polygon_features, Raster_Buffer_shp, Distance__value_or_field_, "FULL", "ROUND", "NONE", "")
Is .png the format that this function wants? PNG is a compressed format so I would think that something like this would be expecting an uncompressed format. In fact, since the name of the function is RasterToPolygon_conversion, wouldn't the function be expecting a raster format? The docs say that the input should be an integer raster dataset. In addition, The input raster can have any cell size and may be any valid integer raster dataset. Anyway, I suspect that is the real problem.
The last thing to check, if the file is in the correct format as per above, is if there is a field VALUE in the file.
try using a GRID or TIFF file instead of a PNG.
You can convert the PNG with:
http://webhelp.esri.com/arcgiSDEsktop/9.3/index.cfm?TopicName=raster_to_other_format_(multiple)_(conversion)
and then process it's output into the Raster to Polygon conversion.
You could also check the file path of the INPUT RASTER to make sure it looks correct by:
INPUT_RASTER = os.path.join(root + "\\" + filename_zero + ".png")
print INPUT_RASTER
gp.RasterToPolygon_conversion(INPUT_RASTER, Output_polygon_features, "SIMPLIFY", "VALUE")
There is also a method of building a filepath by:
import os
root + os.sep + filename_zero + '.png'