I'm using opencv to find contours of an object, the cuntours are in the the matrix of shape (7873, 1, 2)
(e.g. matrix zero below) in the form of [[[x1, y1]], [[x2, y2]], ...] whre x and y are indexes of pixels of an image.
Is it possible using numpy trickery to pass a list of all coordinates of the contour and change them to 1?
I'd like to avoid loops as this is time sensitive. Apart from numpy is there another time efficient way to do it?
zero = np.zeros((5, 5))
test = np.array([[[2,1]], [[3, 1]], [[1, 0]]])
zero[test] = 1
desired OUPUT (for this example):
x 0 1 2 3 4
y _____________
0| 0 1 0 0 0
1| 0 0 1 1 0
2| 0 0 0 0 0
3| 0 0 0 0 0
4| 0 0 0 0 0
You can do:
idx = test.reshape(-1,2).T
zero[idx[1], idx[0]] = 1
Related
I want to iterate through my datapoints and check whether they are in the same cluster, after using KMeans to cluster them.
And then I need to create a matrix for all the datapoints, and have 1 if they belong on the same cluster, and 0 if they don't.
After using Kmeans, I'm not sure how to retrieve which cluster every datapoint belongs to so I can create such matrix.
Do I do that using labels_ argument?
k_means = KMeans(n_clusters=5).fit(X)
labels_columns = k_means.labels_
labels_row = k_means.labels_
for row in labels_row:
for column in labels_columns:
if row == columns:
--add 1 in matrix position
else:
--add 0 in matrix position
How to best create this matrix? Or do they labels_ provide different information from what my understanding?
Any help is appreciated!
You are on the right track. Kmeans.labels_ returns a vector of n elements which tells you that the
cluster each point belongs to: [3, 4, 10, ...] tells you that point 0 belongs to cluster 3, point 1
belongs to cluster 4 and so on.
You can build the matrix you want in many ways. One possibility I thought which is a bit more elegant than
2 for loops would be the following:
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
n_samples, n_features = 10, 2
X, y = make_blobs(n_samples, n_features)
plt.scatter(X[:, 0], X[:, 1], c=y)
plt.show()
kmeans = KMeans(n_clusters=3).fit(X)
plt.scatter(X[:, 0], X[:, 1], c=kmeans.labels_)
plt.show()
neighbour_matrix = np.zeros(n_samples)
repeat_labels = np.repeat(kmeans.labels_.T, n_samples, axis=0).reshape(n_samples, n_samples)
print(kmeans.labels_)
print(repeat_labels)
proximity_matrix = (repeat_labels == repeat_labels.T).astype(int)
print(proximity_matrix)
I use the vector of labels as my starting point. Let's say that it is the following:
[1 0 0 1 1 2 2 2 2 0]
I transform it in a 2D matrix with np.repeat which has the following shape:
[[1 1 1 1 1 1 1 1 1 1]
[0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0]
[1 1 1 1 1 1 1 1 1 1]
.....
So I repeat the labels as many times as is the number of points n. Then I can just check where this
matrix and its transpose are equal. That will be true only if two points belong to the same cluster:
[[1 0 0 1 1 0 0 0 0 0]
[0 1 1 0 0 0 0 0 0 1]
[0 1 1 0 0 0 0 0 0 1]
[1 0 0 1 1 0 0 0 0 0]
.....
I casted the matrix to int, but mind you that the original output is actually a boolean array.
I left the print statements and the plots in the code to hopefully make it more clear.
Hope it helps!
Say I have an MxN matrix of 0's and 1's. It may or may not be sparse.
I want a function to efficiently find rectangles in the array, where by rectangle I mean:
a set of 4 elements that are all 1's that create the 4 corners of a
rectangle, such that the sides of the rectangle are orthogonal to the
array axes. In other words, a rectangle is a set of 4 1's elements
with coordinates [row index, column index] like so: [r1,c1], [r1,c2],
[r2,c2], [r2,c1].
E.g. this setup has one rectangle:
0 0 0 1 0 1 0
0 0 0 0 0 0 0
0 1 0 0 0 0 0
1 0 0 1 0 1 0
0 0 0 0 0 0 0
0 0 0 1 0 0 1
For a given MxN array, I want a Python function F(A) that returns an array L of subarrays, where each subarray is the coordinate pair of the corner of a rectangle (and includes all of the 4 corners of the rectangle). For the case where the same element of the array is the corner of multiple rectangles, it's ok to duplicate those coordinates.
My thinking so far is:
1) find the coordinates of the apex of each right triangle in the array
2) check each right triangle apex coordinate to see if it is part of a rectangle
Step 1) can be achieved by finding those elements that are 1's and are in a column with a column sum >=2, and in a row with a row sum >=2.
Step 2) would then iterate through each coordinate determined to be the apex of a right triangle. For a a given right triangle coordinate pair, it would iterate through that column, looking at every other right triangle coordinate from 1) that is in that column. For any pair of 2 right triangle points in a column, it would then check which row has a smaller row sum to know which row would be faster to iterate through. Then it would iterate through all of the right triangle column coordinates in that row and see if the other row also has a right triangle point in that column. If it does, those 4 points form a rectangle.
I think this will work, but there will be repetition, and overall this procedure seems like it would be reasonably computationally intensive. What are some better ways for detecting rectangle corners in 0-1 arrays?
This is from the top of my head and during 5 hrs layover at LAX. Following is my algorithm:
Step 1: Search all rows for at least two ones
| 0 0 0 1 0 1 0
| 0 0 0 0 0 0 0
| 0 1 0 0 0 0 0
\|/ 1 0 0 1 0 1 0
0 0 0 0 0 0 0
0 0 0 1 0 0 1
Output:
-> 0 0 0 1 0 1 0
0 0 0 0 0 0 0
0 1 0 0 0 0 0
-> 1 0 0 1 0 1 0
0 0 0 0 0 0 0
-> 0 0 0 1 0 0 1
Step 2: For each pair of ones at each row get the index for one's in the column corresponding to the ones, lets say for the first row:
-> 0 0 0 1 0 1 0
you check for ones in the following columns:
| |
\|/ \|/
0 0 0 1 0 1 0
0 0 0 0 0 0 0
0 1 0 0 0 0 0
1 0 0 1 0 1 0
0 0 0 0 0 0 0
0 0 0 1 0 0 1
Step 3: If both index match; return the indices of all four. This can be easily accessed as you know the row and index of ones at all steps. In our case the search at columns 3, 5 are going to return 3 assuming you start index from 0. So we get the indicies for the following:
0 0 0 ->1 0 ->1 0
0 0 0 0 0 0 0
0 1 0 0 0 0 0
1 0 0 ->1 0 ->1 0
0 0 0 0 0 0 0
0 0 0 1 0 0 1
Step 4: Repeat for all pairs
Algorithm Complexity
I know you need to search for columns * rows * number of pairs but you can always use hashmaps to optimize search O(1). Which will make over complexity bound to the number of pairs. Please feel free to comment with any questions.
Here's an Python implementation which is similar to #PseudoAj solution. It will process the rows starting from top while constructing a dict where keys are x coordinates and values are sets of respective y coordinates.
For every row following steps are done:
Generate a list of x-coordinates with 1s from current row
If length of list is less than 2 move to next row
Iterate over all coordinate pairs left, right where left < right
For every coordinate pair take intersection from dict containing processed rows
For every y coordinate in the intersection add rectangle to result
Finally update dict with coordinates from current row
Code:
from collections import defaultdict
from itertools import combinations
arr = [
[0, 0, 0, 1, 0, 1, 0],
[0, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0],
[1, 0, 0, 1, 0, 1, 0],
[0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 1]
]
# List corner coords
result = []
# Dict {x: set(y1, y2, ...)} of 1s in processed rows
d = defaultdict(set)
for y, row in enumerate(arr):
# Find indexes of 1 from current row
coords = [i for i, x in enumerate(row) if x]
# Move to next row if less than two points
if len(coords) < 2:
continue
# For every pair on this row find all pairs on previous rows
for left, right in combinations(coords, 2):
for top in d[left] & d[right]:
result.append(((top, left), (top, right), (y, left), (y, right)))
# Add coordinates on this row to processed rows
for x in coords:
d[x].add(y)
print(result)
Output:
[((0, 3), (0, 5), (3, 3), (3, 5))]
In numpy, we can do this:
x = np.random.random((10,10))
a = np.random.randint(0,10,5)
b = np.random.randint(0,10,5)
x[a,b] # gives 5 entries from x, indexed according to the corresponding entries in a and b
When I try something equivalent in TensorFlow:
xt = tf.constant(x)
at = tf.constant(a)
bt = tf.constant(b)
xt[at,bt]
The last line gives a "Bad slice index tensor" exception. It seems TensorFlow doesn't support indexing like numpy or Theano.
Does anybody know if there is a TensorFlow way of doing this (indexing a tensor by arbitrary values). I've seen the tf.nn.embedding part, but I'm not sure they can be used for this and even if they can, it's a huge workaround for something this straightforward.
(Right now, I'm feeding the data from x as an input and doing the indexing in numpy but I hoped to put x inside TensorFlow to get higher efficiency)
You can actually do that now with tf.gather_nd. Let's say you have a matrix m like the following:
| 1 2 3 4 |
| 5 6 7 8 |
And you want to build a matrix r of size, let's say, 3x2, built from elements of m, like this:
| 3 6 |
| 2 7 |
| 5 3 |
| 1 1 |
Each element of r corresponds to a row and column of m, and you can have matrices rows and cols with these indices (zero-based, since we are programming, not doing math!):
| 0 1 | | 2 1 |
rows = | 0 1 | cols = | 1 2 |
| 1 0 | | 0 2 |
| 0 0 | | 0 0 |
Which you can stack into a 3-dimensional tensor like this:
| | 0 2 | | 1 1 | |
| | 0 1 | | 1 2 | |
| | 1 0 | | 2 0 | |
| | 0 0 | | 0 0 | |
This way, you can get from m to r through rows and cols as follows:
import numpy as np
import tensorflow as tf
m = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])
rows = np.array([[0, 1], [0, 1], [1, 0], [0, 0]])
cols = np.array([[2, 1], [1, 2], [0, 2], [0, 0]])
x = tf.placeholder('float32', (None, None))
idx1 = tf.placeholder('int32', (None, None))
idx2 = tf.placeholder('int32', (None, None))
result = tf.gather_nd(x, tf.stack((idx1, idx2), -1))
with tf.Session() as sess:
r = sess.run(result, feed_dict={
x: m,
idx1: rows,
idx2: cols,
})
print(r)
Output:
[[ 3. 6.]
[ 2. 7.]
[ 5. 3.]
[ 1. 1.]]
LDGN's comment is correct. This is not possible at the moment, and is a requested feature. If you follow issue#206 on github you'll get updated if/when this is available. Many people would like this feature.
For Tensorflow 0.11, basic indexing has been implemented. More advanced indexing (like boolean indexing) is still missing but apparently is planned for future versions.
Advanced indexing can be tracked with https://github.com/tensorflow/tensorflow/issues/4638
I have a 2D labeled image (numpy array), each label represents an object. I have to find the object's center and its area. My current solution:
centers = [np.mean(np.where(label_2d == i),1) for i in range(1,num_obj+1)]
surface_area = np.array([np.sum(label_2d == i) for i in range(1,num_obj+1)])
Note that label_2d used for centers is not the same as the one for surface area, so I can't combine both operations. My current code is about 10-100 times to slow.
In C++ I would iterate through the image once (2 for loops) and fill the table (an array), from which I would than calculate centers and surface area.
Since for loops are quite slow in python, I have to find another solution. Any advice?
You could use the center_of_mass function present in scipy.ndimage.measurements for the first problem and then use np.bincount for the second problem. Because these are in the mainstream libraries, they will be heavily optimized, so you can expect decent speed gains.
Example:
>>> import numpy as np
>>> from scipy.ndimage.measurements import center_of_mass
>>>
>>> a = np.zeros((10,10), dtype=np.int)
>>> # add some labels:
... a[3:5, 1:3] = 1
>>> a[7:9, 0:3] = 2
>>> a[5:6, 4:9] = 3
>>> print(a)
[[0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0]
[0 1 1 0 0 0 0 0 0 0]
[0 1 1 0 0 0 0 0 0 0]
[0 0 0 0 3 3 3 3 3 0]
[0 0 0 0 0 0 0 0 0 0]
[2 2 2 0 0 0 0 0 0 0]
[2 2 2 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0]]
>>>
>>> num_obj = 3
>>> surface_areas = np.bincount(a.flat)[1:]
>>> centers = center_of_mass(a, labels=a, index=range(1, num_obj+1))
>>> print(surface_areas)
[4 6 5]
>>> print(centers)
[(3.5, 1.5), (7.5, 1.0), (5.0, 6.0)]
Speed gains depend on the size of your input data though, so I can't make any serious estimates on that. Would be nice if you could add that info (size of a, number of labels, timing results for the method you used and these functions) in the comments.
I'm trying to compute the tensor product (update: what I wanted was actually called the Kronecker product, and this naming confusion was why I couldn't find np.kron) of multiple matrices, so that I can apply transformations to vectors that are themselves the tensor product of multiple vectors. I'm running into trouble with flattening the result correctly.
For example, say I want to compute the tensor product of [[0,1],[1,0]] against itself. The result should be something like:
| 0*|0,1| 1*|0,1| |
| |1,0| |1,0| |
| |
| 1*|0,1| 0*|0,1| |
| |1,0| |1,0| |
which I then want to flatten to:
| 0 0 0 1 |
| 0 0 1 0 |
| 0 1 0 0 |
| 1 0 0 0 |
Unfortunately, the things I try all either fail to flatten the matrix or flatten it too much or permute the entries so that some columns are empty. More specifically, the output of the python program:
import numpy as np
flip = np.matrix([[0, 1], [1, 0]])
print np.tensordot(flip, flip, axes=0)
print np.reshape(np.tensordot(flip, flip, axes=0), (4, 4))
is
[[[[0 0]
[0 0]]
[[0 1]
[1 0]]]
[[[0 1]
[1 0]]
[[0 0]
[0 0]]]]
[[0 0 0 0]
[0 1 1 0]
[0 1 1 0]
[0 0 0 0]]
Neither of which is what I want.
There are a lot of other questions similar to this one, but the things suggested in them haven't worked (or maybe I missed the ones that work). Maybe "tensor product" means something slightly different than I thought; but the example above should make it clear.
From the answers to this and this question, I learned what you want is called the "Kronecker product". It's actually built into Numpy, so just do:
np.kron(flip, flip)
But if you want to make the reshape approach work, first rearrange the rows in the tensor:
flip = [[0,1],[1,0]]
tensor4d = np.tensordot(flip, flip, axes=0)
print tensor4d.swapaxes(2, 1).reshape((4,4))