We Keep Coding

Getting KMeans silhouette average score without using the Kmeans import

I have a KMeans function I made takes the input def kmeans(x,k, no_of_iterations): and returns the following return points, centroids it gets plotted perfectly, the code for that isn't very relevant. But I want to calculate for it, the silhouette score and graph this for each value.
#Load Data
data = load_digits().data
pca = PCA(2)
#Transform the data
df = pca.fit_transform(data)
X= df
#y = kmeans.fit_predict(X)
#Applying our function
label, centroids = kmeans(df,10,1000)#returns points value and centroids
y = label.fit_predict(data)
#Visualize the results
u_labels = np.unique(label)
for i in u_labels:
plt.scatter(df[label == i , 0] , df[label == i , 1] , label = i)
plt.scatter(centroids[:,0] , centroids[:,1] , s = 80, color = 'k')
plt.legend()
plt.show()
the above is code for running the KMeans plot
Below is my attempt to calculate silhouette. This is from an example that imports from KMeans but I don't really want to do that nor did it work with my code.
silhouette_avg = silhouette_score(X, y)
print("The average silhouette_score is :", silhouette_avg)
# Compute the silhouette scores for each sample
sample_silhouette_values = silhouette_samples(X, y)
You may notice that there is no value here for y, as I have found y is supposed to be the amount of clusters I think? So I had it as 10 at first and it give an error message. I don't know if from this code anyone could tell me what I do next to get this value?

Try this:
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from yellowbrick.cluster import KElbowVisualizer, SilhouetteVisualizer
mpl.rcParams["figure.figsize"] = (9,6)
# Generate synthetic dataset with 8 blobs
X, y = make_blobs(n_samples=1000, n_features=12, centers=8, shuffle=True, random_state=42)
# Instantiate the clustering model and visualizer
model = KMeans()
visualizer = KElbowVisualizer(model, k=(4,12))
visualizer.fit(X) # Fit the data to the visualizer
visualizer.poof()
# Instantiate the clustering model and visualizer
model = KMeans(8)
visualizer = SilhouetteVisualizer(model)
visualizer.fit(X) # Fit the data to the visualizer
visualizer.poof() # Draw/show/poof the data
Also, see this.
https://www.scikit-yb.org/en/latest/api/cluster/silhouette.html

How to add colour to graph to differentiate positive and negative-numbered data?

Plot of dataset showing banknote authentication
I don't know how to add colors to the different dots to differentiate between the positive and negative datasets. I tried following other examples, but I did not make any progress.
For the record, the Python coding I used is as follows:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
data = pd.read_csv('Banknote_authentication_dataset.csv')
from sklearn.cluster import KMeans
#V1 is the Variance of Wavelet Transformed image
#V2 is the Skewness of Wavelet Transformed image
V1 = data['V1']
V2 = data['V2']
V1_V2 = np.column_stack((V1, V2))
km_res = KMeans(n_clusters=2).fit(V1_V2)
clusters = km_res.cluster_centers_
plt.xlabel('Variance')
plt.ylabel('Skewness')
plt.scatter(V1, V2)
plt.scatter(clusters[:,0], clusters[:,1], s=1000, alpha = 0.50)
The link to the dataset is: https://d3c33hcgiwev3.cloudfront.net/1fXr31hcEemkYxLyQ1aU1g_50fc36ee697c4b158fe26ade3ec3bc24_Banknote-authentication-dataset-.csv?Expires=1613433600&Signature=PhnPBuxjL9TwNwXV2dmS7HN3YOtLJsJo3A26UID0CBBC13cxsBmRmpsyUVN7MXIcrte6oUCBeybrhveDMCb-6-nMsQ8JzSH8qxZgYR7mwfO32WZYDQ7S6qm2Z6hFnkw76NIeEdto5L9CDDFpKkF8OhLd81bjxnTictbS1UTOPXw_&Key-Pair-Id=APKAJLTNE6QMUY6HBC5A.

You can get the predictions by using km_res.predict(V1_V2) and then just pass that into your first call to plt.scatter. So your code would change to look like:
# ... code above
preds = km_res.predict(V1_V2)
plt.scatter(V1, V2, c=preds)
# ... code below
If you want control over what colors it uses just change the number predictions to colors (so you'd make all points that have prediction one turn to the string red for example)

Numpy append sometimes works, sometimes doesn't

so I've been working on this facial identification project. It's for my science fair and I'm in the phase where I'm trying to get data graphs, plots, and visualizations. I've got it to work to some extent, but it's not consistent (in terms of execution).
The thing is, sometimes the code works, sometimes it'll give me an error.
For some context, the error is with Numpy append(). I have a variable I want to append data to but when it doesn't work the error is AttributeError: 'numpy.ndarray' object has no attribute 'append'
#Although the results aren't as expected, this can make for a good demo in ISEF
#The whole refresh after a face is detected is cool and can be used to show how different faces cluster
# Numerical computation requirements
import numpy as np
from numpy import linalg, load, expand_dims, asarray, savez_compressed, append
from numpy.linalg import norm
import pandas as pd
# Plotting requirements
import matplotlib
from matplotlib import pyplot as plt
import matplotlib.patheffects as PathEffects
from matplotlib.animation import FuncAnimation as ani
import seaborn as sb
# Clustering requirements
import sklearn
from sklearn.cluster import KMeans
from sklearn.manifold import TSNE
from sklearn.preprocessing import scale
# Miscellaneous requirements
import os
import cv2
from PIL import Image
from mtcnn.mtcnn import MTCNN
from keras.models import load_model
from scipy.spatial.distance import squareform, pdist
# Initialize RNG seed and required size for Facenet
seed = 12345678
size = (160,160)
# Required networks
facenet = load_model('facenet_keras.h5')
fd = MTCNN()
# Initialize Seaborn plots
sb.set_style('darkgrid')
sb.set_palette('muted')
sb.set_context('notebook', font_scale=1.5, rc={'lines.linewidth': 2.5})
# Matplotlib animation requirements?
plt.style.use('fivethirtyeight')
fig = plt.figure()
# Load embeddings
data = load('jerome only npz/jerome embeddings.npz')
Data_1 = data['arr_0']
Dataset = []
for array in Data_1:
Dataset.append(np.expand_dims(array, axis=0))
# Create cluster
cluster = KMeans(n_clusters=2, random_state=0).fit(Data_1)
y = cluster.labels_
z = pd.DataFrame(y.tolist())
faces = list()
def scatter(x,colors):
palette = np.array(sb.color_palette('hls', 26))
plot = plt.figure()
ax = plt.subplot(aspect='equal')
# sc = ax.scatter(x[:,0],x[:,1], lw =0, s=120, c=palette[colors.astype(np.int)])
sc = ax.scatter(x[:,0],x[:,1], lw =0, s=120)
labels = []
return plot , ax, sc, labels
def detembed():
cam = cv2.VideoCapture(0)
_,frame = cam.read()
info = fd.detect_faces(frame)
if info != []:
for i in info:
print("***************** FACE DETECTED *************************************************")
x,yc,w,h = i['box']
x,y = abs(x), abs(yc)
w,h = abs(w), abs(h)
xx, yy = x+w, yc+h
#cv2.rectangle(frame, (x,y), (xx,yy), (0,0,255),2)
face = frame[yc:yy, x:xx]
image = Image.fromarray(face)
image = image.resize(size)
arr = asarray(image)
arr = arr.astype('float32')
mean, std = arr.mean(), arr.std()
arr = (arr - mean) / std
samples = expand_dims(arr, axis=0)
faces.append(samples)
#cv2.imshow('Camera Feed', frame)
while True:
detembed()
embeddings = Dataset
if not faces:
continue
else:
for face in faces:
embeds = facenet.predict(face)
#switch these if conflicts arise
embeddings.append(embeds)
embeddings = asarray(embeddings)
embeddings = embeddings[:,0,:]
cluster = KMeans(n_clusters=2, random_state=0).fit(Data_1)
y = cluster.labels_
points = TSNE(random_state=seed).fit_transform(embeddings)
# here "y" dictates the color of the plots depending on the kmeans algorithm
scatter(points,y)
graph = ani(fig, scatter, interval=20)
fcount = len(embeddings)
plt.text(0,0,'{} points'.format(fcount))
plt.show()
# reset embeddings var to initial dataset
Dataset = np.delete(Dataset, fcount - 1,0)
embeddings = Dataset
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cv2.release()
cv2.destroyAllWindows
Note that I am not a talented programmer; this code was botched from some example I found online. I had to pick up Python as I went along with this project. I do have a background in C, so I would say I get the basics of code logic.
Please help. I'm getting really desperate; the science fair is getting closer and I am a high schooler with no ML mentor. I live on an island (Guam) with no machine learning practitioners (not even in the university), so I turn to Stackoverflow.

There's no issue with NumPy's append(). Here(3rd statement) you're trying to append a value to Numpy array without using NumPy's np.append().
Dataset.append(np.expand_dims(array, axis=0))
embeddings = Dataset
embeddings.append(embeds)
Since Datasets contain Numpy array after running the first statement, embeddings will also be a NumPy array and hence the operation fails whenever the execution comes here.
A simple fix would be to use this:
np.append(embeddings, embeds)
Or this,
embeddings = list(Dataset)
Hope that helps.

Grad-cam not working properly in keras with inceptionv3

I am using visualise cam from keras-vis for creating guided-gradcam images.
The grad-cam is working perfectly well with vgg16. but when i used the same code for inceptionv3 it is not working properly.
from keras.applications.inception_v3 import InceptionV3
from vis.utils import utils
from keras.preprocessing import image
import numpy as np
from keras import activations
from matplotlib import pyplot as plt
%matplotlib inline
from vis.visualization import visualize_cam,overlay
#build the inceptionv3 model with imagenet weights
model = InceptionV3(weights='imagenet',include_top=True)
# Utility to search for layer index by name
layer_idx = utils.find_layer_idx(model,'predictions')
#swap with softmax with linear classifier for the reasons mentioned above
model.layers[layer_idx].activation = activations.linear
model = utils.apply_modifications(model)
from vis.utils import utils
from matplotlib import pyplot as plt
%matplotlib inline
plt.rcParams['figure.figsize']=(18,6)
img1 = utils.load_img('images/ouzel1.jpg',target_size=(299,299))
img2 = utils.load_img('images/ouzel2.jpg',target_size=(299,299))
f, ax = plt.subplots(1,2)
ax[0].imshow(img1)
ax[1].imshow(img2)
plt.show()
from vis.visualization import visualize_cam
for modifier in [None, 'guided', 'relu']:
plt.figure()
f, ax = plt.subplots(1, 2)
plt.suptitle("vanilla" if modifier is None else modifier)
for i, img in enumerate([img1, img2]):
# 20 is the imagenet index corresponding to `ouzel`
heatmap = visualize_cam(model, layer_idx, filter_indices=20,
seed_input=img, backprop_modifier=modifier,
#penultimate_layer_idx = 299 # corresponding to "conv2d_94"
)
# Lets overlay the heatmap onto original image.
ax[i].imshow(overlay(img, heatmap))
by commenting out the line #penultimate_layer also I am getting the same output which is not correct. can someone tell me what is the problem? The guided-grad cam result is given , followed by the original image is given.
The problem is the heatmap must be on the bird (ouzel).

I hit the very same problem, but then I discovered that InceptionV3 mis-classifies these images. Check:
>>> model.predict(np.stack([img1, img2], 0)).argmax(axis=1)
array([110, 725])
While with VGG it's:
>>> model.predict(np.stack([img1, img2], 0)).argmax(axis=1)
array([20, 20])

opencv knn TypeError: only length-1 arrays can be converted to Python scalars

I am trying to follow the tutorial http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_ml/py_knn/py_knn_opencv/py_knn_opencv.html and replaced KNearest with cv2.m1.KNearest_create() but i am getting TypeError: only length-1 arrays can be converted to Python scalars
import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread('digits.png')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Now we split the image to 5000 cells, each 20x20 size
cells = [np.hsplit(row,100) for row in np.vsplit(gray,50)]
# Make it into a Numpy array. It size will be (50,100,20,20)
x = np.array(cells)
# Now we prepare train_data and test_data.
train = x[:,:50].reshape(-1,400).astype(np.float32) # Size = (2500,400)
test = x[:,50:100].reshape(-1,400).astype(np.float32) # Size = (2500,400)
# Create labels for train and test data
k = np.arange(10)
train_labels = np.repeat(k,250)[:,np.newaxis]
test_labels = train_labels.copy()
# Initiate kNN, train the data, then test it with test data for k=1
cv2.m1.KNearest_create()
knn.train(train,train_labels)
ret,result,neighbours,dist = knn.find_nearest(test,k=5)
# Now we check the accuracy of classification
# For that, compare the result with test_labels and check which are wrong
matches = result==test_labels
correct = np.count_nonzero(matches)
accuracy = correct*100.0/result.size
print accuracy
(i am using a raspberry pi and followed this tutorial to install open cv http://www.pyimagesearch.com/2015/10/26/how-to-install-opencv-3-on-raspbian-jessie/ subsequently i pip installed matplotlib)

parameter cv2.ml.ROW_SAMPLE is missing and change knn.find_nearest(test,k=5) to below code.This is new in openCv3, please refer to openCv official site http://docs.opencv.org/3.0.0/dd/de1/classcv_1_1ml_1_1KNearest.html
` knn.train(train, cv2.ml.ROW_SAMPLE, train_labels)
ret, result, neighbours, dist = knn.findNearest(test, k=5)`

You're just missing one parameter, but I notice that a lot of people have questions about this section of the tutorial, so here's the whole final section adjusted to work with python3 and the modern openCV library.
knn = cv2.ml.KNearest_create()
knn.train(trainData, cv2.ml.ROW_SAMPLE, responses)
ret, results, neighbours, dist = knn.findNearest(newcomer, k=5)
print("result: ", results,"\n")
print("neighbours: ", neighbours,"\n")
print("distance: ", dist)
plt.show()

import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread('digits.png')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Now we split the image to 5000 cells, each 20x20 size
cells = [np.hsplit(row,100) for row in np.vsplit(gray,50)]
# Make it into a Numpy array. It size will be (50,100,20,20)
x = np.array(cells)
# Now we prepare train_data and test_data.
train = x[:,:50].reshape(-1,400).astype(np.float32) # Size = (2500,400)
test = x[:,50:100].reshape(-1,400).astype(np.float32) # Size = (2500,400)
# Create labels for train and test data
k = np.arange(10)
train_labels = np.repeat(k,250)[:,np.newaxis]
test_labels = train_labels.copy()
# Initiate kNN, train the data, then test it with test data for k=1
knn = cv2.ml.KNearest_create()
knn.train(train, cv2.ml.ROW_SAMPLE, train_labels)
ret, results, neighbours, dist = knn.findNearest(test, k=5)
#print("result: ", results,"\n")
#print("neighbours: ", neighbours,"\n")
#print("distance: ", dist)
matches = result=test_labels
correct = np.count_nonzero(matches)
accuracy = correct*100.0/result.size
print(accuracy)

doc of opencv said that:
findNearest(...) | findNearest(samples, k[, results[,
neighborResponses[, dist]]]) -> retval, results, neighborResponses,
...
not knn.find_nearest(test,k=5)
you can run
help(cv2.ml.KNearest_create())
then you will see.
by the way ,there are losts erros on opencv website