Apply the same code to multiple files in the same directory - python

I have a code that already works but I need to use it to analyse many files in the same folder. How can I re-write it to do this? All the files have similar names (e.g. "pos001", "pos002", "pos003").
This is the code at the moment:
pos001 = mpimg.imread('pos001.tif')
coord_pos001 = np.genfromtxt('treat_pos001_fluo__spots.csv', delimiter=",")
Here I label the tif file "pos001" to differentiate separate objects in the same image:
label_im = label(pos001)
regions = regionprops(label_im)
Here I select only the object of interest by setting its pixel values == 1 and all the others == 0 (I'm interested in many objects, I show only one here):
cell1 = np.where(label_im != 1, 0, label_im)
Here I convert the x,y coordinates of the spots in the csv file to a 515x512 image where each spot has value 1:
x = coord_pos001[:,2]
y = coord_pos001[:,1]
coords = np.column_stack((x, y))
img = Image.new("RGB", (512,512), "white")
draw = ImageDraw.Draw(img)
dotSize = 1
for (x,y) in coords:
draw.rectangle([x,y,x+dotSize-1,y+dotSize-1], fill="black")
im_invert = ImageOps.invert(img)
bin_img = im_invert.convert('1')
Here I set the values of the spots of the csv file equal to 1:
bin_img = np.where(bin_img == 255, 1, bin_img)
I convert the arrays from 2d to 1d:
bin_img = bin_img.astype(np.int64)
cell1 = cell1.flatten()
bin_img = bin_img.flatten()
I multiply the arrays to get an array where only the spots overlapping the labelled object have value = 1:
spots_cell1 = []
for num1, num2 in zip(cell1, bin_img):
spots_cell1.append(num1 * num2)
I count the spots belonging to that object:
spots_cell1 = sum(float(num) == 1 for num in spots_cell1)
print(spots_cell1)
I hope it's clear. Thank you in advance!

You can define a function that takes the .tif file path and the .csv file path and processes the two
def process(tif_file, csv_file):
pos = mpimg.imread(tif_file)
coord = np.genfromtxt(csv_file, delimiter=",")
# Do other processing with pos and coord
To process a single pair of files, you'd do:
process('pos001.tif', 'treat_pos001_fluo__spots.csv')
To list all the files in your tif file directory, you'd use the example in this answer:
import os
tif_file_directory = "/home/username/path/to/tif/files"
csv_file_directory = "/home/username/path/to/csv/files"
all_tif_files = os.listdir(tif_file_directory)
for file in all_tif_files:
if file.endswith(".tif"): # Make sure this is a tif file
fname = file.rstrip(".tif") # Get just the file name without the .tif extension
tif_file = f"{tif_file_directory}/{fname}.tif" # Full path to tif file
csv_file = f"{csv_file_directory}/treat_{fname}_fluo__spots.csv" # Full path to csv file
# Just to keep track of what is processed, print them
print(f"Processing {tif_file} and {csv_file}")
process(tif_file, csv_file)
The f"...{variable}..." construct is called an f-string. More information here: https://realpython.com/python-f-strings/

Related

Saving images with different name in folder

I tried save images in folder like this, it saves different images but every next image have all names of previously images.
db = h5py.File('results/Results.h5', 'r')
dsets = sorted(db['data'].keys())
for k in dsets:
db = get_data()
imnames = sorted(db['data'].keys())
slika = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
cv2.imwrite(f'spremljene_slike/ime_{imnames}.png', slika)
So i tried like this and it saves different names but only last generated picture is imwrited in folder, so different names - the same picture
NUM_IMG = -1
N = len(imnames)
global NUM_IMG
if NUM_IMG < 0:
NUM_IMG = N
start_idx,end_idx = 0,N #min(NUM_IMG, N)
**In different function:**
for u in range(start_idx,end_idx):
imname = imnames[u]
cv2.imwrite(f'spremljene_slike/ime_{imname}.png', imname)
Can someone help, I can't figure out.
I have script which generate images with rendered text and save it in .h5 file, and then from there I want to save this pictures with corresponding names in different folder.
Don't see how this works at all. On line 1 you define db=h5py.File(), then on line 4, you redefine it as db=get_data(). What is get_data()?
It's hard to write code without the schema. Answer below is my best-guess assuming your images are datasets in db['data'] and you want to use the dataset names (aka keys) as the image names.
with h5py.File('results/Results.h5', 'r') as db:
dsets = sorted(db['data'].keys())
for imname in dsets:
img_arr = db['data'][imname][()]
slika = cv2.cvtColor(img_arr, cv2.COLOR_BGR2RGB)
cv2.imwrite(f'spremljene_slike/ime_{imname}.png', slika)
That should be all you need to do. You will get 1 .png for each dataset named ime_{imname}.png (where imname is the matching dataset name).
Also, you can eliminate all of the intermediate variables (dsets, img_arr and slika). Compress the code above into a few lines:
with h5py.File('results/Results.h5', 'r') as db:
for imname in sorted(db['data'].keys()):
cv2.imwrite(f'spremljene_slike/ime_{imname}.png', \
cv2.cvtColor(db['data'][imname][()], cv2.COLOR_BGR2RGB))

Save files of a modeling program in different files .txt

hello I am New to Python, I have worked with a library design by a professor in my university
I need help to save the results in different files .txt and in different folders
The model consists of a mesh of 2500 cells with coordinates x, y, z. I want in every step of the iteration to get a file .txt that has cell index, the coordinates of each cell, and the different attributes of this cell, I wrote the code below but I obtained the last cell index and the position 0.0.0... in only one file txt, and I don't know how to save it in a folder...
if os.path.exists('Model'):
import shutil
shutil.rmtree('Model')
os.mkdir('Model')
def save_m1(path, fluid_index):
assert isinstance(path, str)
with open(path, 'w') as file:
for cell in model.cells:
x, y, z = cell.pos
file.write(f'{cell.pos} {model.cell_number} {cell.get_fluid(fluid_index).mass}\n')
if step % 100 == 0:
model.print_cells("Model" % step)
dt = 10.0
for step in range(10):
model.iterate(dt=dt)
thermal.iterate(dt=dt)
thermal.exchange_heat(model, dt=dt, fid=0, na_g = na_g, fa_T = ID_T, fa_c = ID_K)
thermal.exchange_heat(model, dt=dt, fid=1, na_g = na_g, fa_T = ID_T, fa_c = ID_K)
thermal.exchange_heat(model, dt=dt, fid=2, na_g = na_g, fa_T = ID_T, fa_c = ID_K)
print(f'step = {step}, m1_produced = {get_mass1_produced(1) - InitialMass1}') # kg
save_m1('xxx.txt', 1)

How to append the filenames of randomly selected images used in creating a single image to a csv file?

Thank you massively to Paul M who posted the following code in response to my first query on how to compile a "stack" of randomly selected images into a unique image:
from pathlib import Path
from random import choice
layers = [list(Path(directory).glob("*.png")) for directory in ("bigcircle/", "mediumcircle/")]
selected_paths = [choice(paths) for paths in layers]
img = Image.new("RGB", (4961, 4961), color=(0, 220, 15))
for path in selected_paths:
layer = Image.open(str(path), "r")
img.paste(layer, (0, 0), layer)
I have the code sat in a for _ in itertools.repeat(None, num): where num defines the number of different images being generated. I end the loop with the following to save each image with a unique (incremental) file name:
i = 0
while os.path.exists("Finished Pieces/image %s.png" % i):
i += 1
img.save("Finished Pieces/image %s.png" % i,)
So far, so good. The challenge I'm struggling with now is how to append to a data.csv file with the details of each image created.
For example, in loop 1 bigcircle1.png is selected from bigcircle/ folder and mediumcircle6.png from mediumcircle/, loop 2 uses bigcircle3.png and mediumcircle2.png, and so on. At the end of this loop the data.csv file would read:
Filename,bigcircle,mediumcircle
image 0,bigcircle1.png,mediumcircle6.png
image 1,bigcircle3.png,mediumcircle2.png
I've tried the following, which I know wouldn't give the desired result, but which I thought might be a good start for me to run and tweak until right, but it doesn't generate any output (and I am importing numpy as np):
np.savetxt('data.csv', [p for p in zip(img, layer)], delimiter=',', fmt='%s')
If it's not too much of an ask, ideally the first iteration of the loop would create data.csv and store the first record, with the second iteration onwards appending this file.
Me again ;)
I think it would make sense to split up the functionality of the program into separate functions. I would start maybe with a function called something like discover_image_paths, which discovers (via glob) all image paths. It might make sense to store the paths according to what kind of circle they represent - I'm envisioning a dictionary with "big" and "medium" keys, and lists of paths as associated values:
def discover_image_paths():
from pathlib import Path
keys = ("bigcircle", "mediumcircle")
return dict(zip(keys, (list(Path(directory).glob("*.png")) for directory in (key+"/" for key in keys))))
def main():
global paths
paths = discover_image_paths()
return 0
if __name__ == "__main__":
import sys
sys.exit(main())
In the terminal:
>>> paths["bigcircle"]
[WindowsPath('bigcircle/big1.png'), WindowsPath('bigcircle/big2.png'), WindowsPath('bigcircle/big3.png')]
>>> paths["mediumcircle"]
[WindowsPath('mediumcircle/med1.png'), WindowsPath('mediumcircle/med2.png'), WindowsPath('mediumcircle/med3.png')]
>>>
As you can see, to test the script, I created some dummy image files - three for each category.
Extending this by adding a function to generate an output image (given an iterable of paths to combine and an output file name) and a main loop to generate num_images number of images (sorry, I'm not familiar with numpy):
def generate_image(paths, color, output_filename):
from PIL import Image
dimensions = (4961, 4961)
image = Image.new("RGB", dimensions, color=color)
for path in paths:
layer = Image.open(path, "r")
image.paste(layer, (0, 0), layer)
image.save(output_filename)
def discover_image_paths(keys):
from pathlib import Path
return dict(zip(keys, (list(Path(directory).glob("*.png")) for directory in (key+"/" for key in keys))))
def main():
from random import choice, choices
from csv import DictWriter
field_names = ["filename", "color"]
keys = ["bigcircle", "mediumcircle"]
paths = discover_image_paths(keys)
num_images = 5
with open("data.csv", "w", newline="") as file:
writer = DictWriter(file, fieldnames=field_names+keys)
writer.writeheader()
for image_no in range(1, num_images + 1):
selected_paths = {key: choice(category_paths) for key, category_paths in paths.items()}
file_name = "output_{}.png".format(image_no)
color = tuple(choices(range(0, 256), k=3))
generate_image(map(str, selected_paths.values()), color, file_name)
row = {**dict(zip(field_names, [file_name, color])), **{key: path.name for key, path in selected_paths.items()}}
writer.writerow(row)
return 0
if __name__ == "__main__":
import sys
sys.exit(main())
Example output in the CSV:
filename,color,bigcircle,mediumcircle
output_1.png,"(49, 100, 190)",big3.png,med1.png
output_2.png,"(228, 37, 227)",big2.png,med3.png
output_3.png,"(251, 14, 193)",big1.png,med1.png
output_4.png,"(35, 12, 196)",big1.png,med3.png
output_5.png,"(62, 192, 170)",big2.png,med2.png

Duplicate Image Detector Not Deleting All Duplicate Files

I use a double for loop to iterate through the files in my Picture folder. If the first image is equal to the second image, I then delete the second image from the folder and delete the image from the list of files I'm iterating through. My program leaves some duplicate images and doesn't delete them, but when I run the program again it deletes the duplicate image. How to I properly iterate through my files so that I am able to delete all duplicate images?
I've tried cutting down on unnecessary if, else loops in order to make sure the file doesn't slip through the program.
path = "Pictures"
directory = os.listdir(path)
for first_file in directory:
for second_file in directory:
if first_file == second_file:
continue
if first_file.endswith(".jpg") and second_file.endswith(".jpg"):
first_file_path = R"Pictures\{}".format(first_file)
second_file_path = R"Pictures\{}".format(second_file)
img1 = cv2.imread(first_file_path, 1)
img2 = cv2.imread(second_file_path, 1)
img1 = cv2.resize(img1, (100,100))
img2 = cv2.resize(img2, (100,100))
difference = cv2.subtract(img1, img2)
b, g, r = cv2.split(difference)
if cv2.countNonZero(b) == 0 and cv2.countNonZero(g) == 0 and cv2.countNonZero(r) == 0:
os.remove(second_file_path)
directory.remove(second_file)

How to create variables for Facial_Recognition from database

I'm trying to be able to pull data from a database with a name and an image file name then put it into a face_recognition Python program. However, for the code that I'm using, the program learns the faces by calling variables with different names.
How can I create variables based on the amount of data that I have in the database?
What could be a better approach to solve this problem?
first_image = face_recognition.load_image_file("first.jpg")
first_face_encoding = face_recognition.face_encodings(first_image)[0]
second_image = face_recognition.load_image_file("second.jpg")
biden_face_encoding = face_recognition.face_encodings(second_image)[0]
You can use arrays instead of storing each image/encoding in an individual variable, and fill the arrays from a for loop.
Assuming you can change the filenames from first.jpg, second.jpg... to 1.jpg, 2.jpg... you can do this:
numberofimages = 10 # change this to the total number of images
images = [None] * (numberofimages+1) # create an array to store all the images
encodings = [None] * (numberofimages+1) # create an array to store all the encodings
for i in range(1, numberofimages+1):
filename = str(i) + ".jpg" # generate image file name (eg. 1.jpg, 2.jpg...)
# load the image and store it in the array
images[i] = face_recognition.load_image_file(filename)
# store the encoding
encodings[i] = face_recognition.face_encodings(images[i])[0]
You can then access eg. the 3rd image and 3rd encoding like this:
image[3]
encoding[3]
If changing image file names is not an option, you can store them in a dictionary and do this:
numberofimages = 3 # change this to the total number of images
images = [None] * (numberofimages+1) # create an array to store all the images
encodings = [None] * (numberofimages+1) # create an array to store all the encodings
filenames = {
1: "first",
2: "second",
3: "third"
}
for i in range(1, numberofimages+1):
filename = filenames[i] + ".jpg" # generate file name (eg. first.jpg, second.jpg...)
print(filename)
# load the image and store it in the array
images[i] = face_recognition.load_image_file(filename)
# store the encoding
encodings[i] = face_recognition.face_encodings(images[i])[0]

Categories