I built a CNN prediction model to do the prediction. And I append the result to a DataFrame as pic show. However, why does my prediction column has 2 brackets [[]] around my data? How to get rid of it and show the number only?
test_img = 'C:/Users/User/Desktop/GF_BSIF/Circle_Cropped_test_images/*.jpg'
Test_Path = 'C:/Users/User/Desktop/GF_BSIF/Circle_Cropped_test_images'
Name = []
result = []
for name in os.listdir(Test_Path):
Name.append(name[0:-5])
for img in glob.glob(test_img):
prediction = model.predict(prepare(img))
result.append(prediction)
Temp = {'File Name':Name, 'Prediction':result}
temp = pd.DataFrame(Temp)
temp
So After I change the result.append(prediction) to result.append(prediction[0])
test_img = 'C:/Users/User/Desktop/GF_BSIF/Circle_Cropped_test_images/*.jpg'
Test_Path = 'C:/Users/User/Desktop/GF_BSIF/Circle_Cropped_test_images'
Name = []
result = []
for name in os.listdir(Test_Path):
Name.append(name[0:-5])
for img in glob.glob(test_img):
prediction = model.predict(prepare(img))
result.append(prediction[0])
Temp = {'File Name':Name, 'Prediction':result}
temp = pd.DataFrame(Temp)
temp
The DataFrame shows 1 bracket left.
Any way to remove the bracket?
seems your prediction is 2d array, can you print the return type and value of model.predict(). like below:
print(type(prediction))
print(prediction)
Related
I have a dataset of images and two txt files in which each line contains the id of three pictures, the first one is for training and tells me that the first picture is most similar to the second one than to the third one. The second one is for testing: I have to predict wether the first image is most similar to the first or the second one for each line.
To do this I have trained a siamese network utilising triplet loss using as guideline this article: https://keras.io/examples/vision/siamese_network/
After training the network I do not know how to proceed to evaluate my testing dataset, to prepare the data I have done:
with open('test_triplets.txt') as f:
lines2 = f.readlines()
lines2 = [line.split('\n', 1)[0] for line in lines2]
anchor2 = [line.split()[0] for line in lines2]
pic1 = [line.split()[1] for line in lines2]
pic2 = [line.split()[2] for line in lines2]
anchor2 = ['food/' + item + '.jpg' for item in anchor2]
pic1 = ['food/' + item + '.jpg' for item in pic1]
pic2 = ['food/' + item + '.jpg' for item in pic2]
anchor2_dataset = tf.data.Dataset.from_tensor_slices(anchor2)
pic1_dataset = tf.data.Dataset.from_tensor_slices(pic1)
pic2_dataset = tf.data.Dataset.from_tensor_slices(pic2)
test_dataset = tf.data.Dataset.zip((anchor2_dataset, pic1_dataset, pic2_dataset))
test_dataset = test_dataset.map(preprocess_triplets)
test_dataset = test_dataset.batch(32, drop_remainder=False)
test_dataset = test_dataset.prefetch(8)
I have then tried to utilise a for loop as follows, but the running time is too high since I have around 50000 lines in the txt file.
n_images = len(anchor2)
results = np.zeros((n_images,2))
for i in range(n_images):
sample = next(iter(test_dataset))
anchor, positive, negative = sample
anchor_embedding, positive_embedding, negative_embedding = (
embedding(resnet.preprocess_input(anchor)),
embedding(resnet.preprocess_input(positive)),
embedding(resnet.preprocess_input(negative)),
)
cosine_similarity = metrics.CosineSimilarity()
positive_similarity = cosine_similarity(anchor_embedding, positive_embedding)
results[i,0] = positive_similarity.numpy()
negative_similarity = cosine_similarity(anchor_embedding, negative_embedding)
results[i,1] = negative_similarity.numpy()
How can I do to be able to generate predictions on my testing triplets ? My objective would be to have a vector [n_testing_triplets x 1] where each line is 1 if the first pic is most similar to the anchor or 0 otherwise.
You can stack your images first, then calculate all embedings in parallel like this :
import numpy as np
stack = np.stack([anchor0, positive0, negative0, ..., anchor999, positive999, negative999])
# then you calculate all embeding at the same time like this
embeddings = list(embedding(resnet.preprocess_input(stack)).numpy())
Then you compare the embeding as you want, in a loop :
cosine_similarity = metrics.CosineSimilarity()
positive_similarity = cosine_similarity(embeddings [0] , embeddings [1])
whatever_storage = positive_similarity.numpy()
negative_similarity = cosine_similarity(embeddings [0] , embeddings [2])
whatever_storage = negative_similarity.numpy()
I am trying to work on an image colorizer using autoencoders. The 'input' is a grayscale image and the 'labels' are their corresponding color images. I am trying to get it to work on google colab on a subset of the dataset. The problem is that the session crashes when I try to convert the list of images to a numpy array.
Here's what I tried:
X = []
y = []
errors = 0
count = 0
image_path = 'drive/My Drive/datasets/imagenette'
for file in tqdm(os.listdir(image_path)):
try:
# Load, transform image
color_image = io.imread(os.path.join(image_path,file))
color_image = transform.resize(color_image,(224,224))
lab_image = color.rgb2lab(color_image)
L = lab_image[:,:,0]/100.
ab = lab_image[:,:,1:]/128.
# Append to list
gray_scale_image = np.stack((L,L,L), axis=2)
X.append(gray_scale_image)
y.append(ab)
count += 1
if count == 5000:
break
except:
errors += 1
print(f'Errors encountered: {errors}')
X = np.array(X)
y = np.array(y)
print(X.shape)
print(y.shape)
Is there a way to load only a few batches, feed them and while the model is being trained on them, load another set of batches?
Trying to put an images pixel data into a pandas data frame to tun PCA across. I think I got it working but for some reason the layout is off. When I run the following code I get this result :
#read in image
img = cv2.imread('/Volumes/EXTERNAL/Stitched-Photos-for-Chris/p7_0015_20161005-949am-75m-pass-1.jpg.png',1)
row,col = img.shape[:2]
#print(row , col)
#get a unique pixel ID for each pixel
pixel = ['pixel-' + str(i) for i in range(0,row*col)]
bBand = ['bBand']
gBand = ['gBand']
rBand = ['rBand']
data = pd.DataFrame(columns=[bBand,gBand,rBand],index = pixel)
#populate data for each band
b,g,r = cv2.split(img)
data.loc[pixel,'bBand'] = b.flat[:]
data.loc[pixel,'gBand'] = g.flat[:]
data.loc[pixel,'rBand'] = r.flat[:]
print(data.head())
However, when I run the tutorial code I am basing this off I get the proper format:
genes = ['gene' + str(i) for i in range(1,101)]
wt = ['wt' + str(i) for i in range(1,6)]
ko = ['ko' + str(i) for i in range(1,6)]
data = pd.DataFrame(columns=[*wt,*ko],index = genes)
#create random data
for gene in genes:
data.loc[gene,'wt1':'wt5'] = np.random.poisson(lam=rd.randrange(10,10000),size=5)
data.loc[gene,'ko1':'ko5'] = np.random.poisson(lam=rd.randrange(10,10000),size=5)
print(data.head())
Trying to figure out if the extra gBand and rBand in the columns is an issuer or error that I have somewhere. Thanks for your help.
it looks like you are creating your columns incorrectly by making them a list
try:
pixel = ['pixel-' + str(i) for i in range(0,row*col)]
data = pd.DataFrame(columns=['bBand','gBand','rBand'],index = pixel)
#populate data for each band
b,g,r = cv2.split(img)
data.loc[pixel,'bBand'] = b.flat[:]
data.loc[pixel,'gBand'] = g.flat[:]
data.loc[pixel,'rBand'] = r.flat[:]
print(data.head())
I am trying to run a python code but I get an error, I am not familiar with python so I don't know how to debug the code.
please help and thank you.
I just found this code on this website :
http://www.paulvangent.com/2016/04/01/emotion-recognition-with-python-opencv-and-a-face-dataset/
here is the code :
import cv2
import glob
import random
import numpy as np
emotions = ["neutral", "anger", "contempt", "disgust", "fear", "happy", "sadness", "surprise"] #Emotion list
fishface = cv2.face.createFisherFaceRecognizer() #Initialize fisher face classifier
data = {}
def get_files(emotion): #Define function to get file list, randomly shuffle it and split 80/20
files = glob.glob("dataset\\%s\\*" %emotion)
random.shuffle(files)
training = files[:int(len(files)*0.8)] #get first 80% of file list
prediction = files[-int(len(files)*0.2):] #get last 20% of file list
return training, prediction
def make_sets():
training_data = []
training_labels = []
prediction_data = []
prediction_labels = []
for emotion in emotions:
training, prediction = get_files(emotion)
#Append data to training and prediction list, and generate labels 0-7
for item in training:
image = cv2.imread(item) #open image
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) #convert to grayscale
training_data.append(gray) #append image array to training data list
training_labels.append(emotions.index(emotion))
for item in prediction: #repeat above process for prediction set
image = cv2.imread(item)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
prediction_data.append(gray)
prediction_labels.append(emotions.index(emotion))
return training_data, training_labels, prediction_data, prediction_labels
def run_recognizer():
training_data, training_labels, prediction_data, prediction_labels = make_sets()
print ("training fisher face classifier")
print ("size of training set is:", len(training_labels), "images")
fishface.train(training_data, np.asarray(training_labels))
print ("predicting classification set")
cnt = 0
correct = 0
incorrect = 0
for image in prediction_data:
pred, conf = fishface.predict(image)
if pred == prediction_labels[cnt]:
correct += 1
cnt += 1
else:
incorrect += 1
cnt += 1
return ((100*correct)/(correct + incorrect))
#Now run it
metascore = []
for i in range(0,10):
correct = run_recognizer()
print ("got", correct, "percent correct!")
metascore.append(correct)
print ("\n\nend score:", np.mean(metascore), "percent correct!")
here is the output of this code :
training fisher face classifier
size of training set is: 351 images
predicting classification set
Traceback (most recent call last):
File "splitData.py", line 62, in <module>
correct = run_recognizer()
File "splitData.py", line 51, in run_recognizer
pred, conf = fishface.predict(image)
TypeError: 'int' object is not iterable
int is not iterable
a, b = c means that it will attempt to unpack c as an iterable object and assign to a, b. So, a, b = [1, 2] will set a to 1 and b to 2. a, b = [1] will error because there aren't enough values to unpack. a, b = [1, 2, 3] will error because there are too many values to unpack.
In your case, a, b = 1 would error because 1 can't be unpacked, because you cannot iterate through it. Iterating through an object means going through all of its elements (like a list, tuple, set, dict, etc). An integer is just a value; it isn't iterable.
This means that fishface.predict returns a number. I'm not sure what pred, conf are meant to be (I'm guessing prediction and confidence), but check the docs for FisherFaceRecognizer#predict to see what it returns.
I'm tryint to build an application that classifies different objects. I have a training folder with a bunch of images i want to use as training for my SVM.
Up untill now I have followed this (GREAT) answer:
using OpenCV and SVM with images
here is a sample of my code:
def getTrainingData():
address = "..//data//training"
labels = []
trainingData = []
for items in os.listdir(address):
## extracts labels
name = address + "//" + items
for it in os.listdir(name):
path = name + "//" + it
print path
img = cv.imread(path, cv.CV_LOAD_IMAGE_GRAYSCALE)
d = np.array(img, dtype = np.float32)
q = d.flatten()
trainingData.append(q)
labels.append(items)
######DEBUG######
#cv.namedWindow(path,cv.WINDOW_NORMAL)
#cv.imshow(path,img)
return trainingData, labels
svm_params = dict( kernel_type = cv.SVM_LINEAR,
svm_type = cv.SVM_C_SVC,
C=2.67, gamma=3 )
training, labels = getTrainingData()
train = np.asarray(training)
svm = cv.SVM()
svm.train(train, labels, params=svm_params)
svm.save('svm_data.dat')
But when i try to run i recieve the following error:
svm.train(train, labels, params=svm_params)
TypeError: trainData data type = 17 is not supported
What am i doing wrong?
Thanks A lot!
You should resize your input images. like this:
img = cv2.resize(img, (64,64))
Size is up to you.