I'm iterating over a NumPy (nd) array of OpenCV lines. I want to remove all lines which are outwith 8 degrees of vertical. I realise the numpy array is immutable and what I'm doing in the code is not right but it demonstrates the idea of what I'm trying to do;
index = 0
for line in self.lines[0]:
if (line[1]*180)/np.pi > 8:
self.lines[0] = np.delete(self.lines[0], index, axis=0)
index+=1
How can I go about removing these NumPy array indexes?
Thanks!
You cannot delete array indexes while iterating over that array. It will give wrong results. Say you are at iteration 5 and this index satisfies if condition and needs to be deleted, but if we delete this it will cause array element at index 6 to come at index 5 and next element selected will be 7 which comes to index 6 after deletion and thus condition is never checked for element initially at index 6 i.e., before deletion.
Basic idea to deal with this issue is to append these indices to a list and delete them outside lope. So your code is:
index = 0
idx = []
for line in self.lines[0]:
if (line[1]*180)/np.pi > 8:
idx.append(index)
index+=1
self.lines[0] = np.delete(self.lines[0], idx, axis=0)
I guess you are missing indentation in your code, so I interpret it like this:
index = 0
for line in self.lines[0]:
if (line[1]*180)/np.pi > 8:
self.lines[0] = np.delete(self.lines[0], index, axis=0)
index+=1
You can do that of course without a loop. First we do the check to build the boolean indexing array (True/False values for every item in the array):
index = self.lines[0,:,1]*180/np.pi > 8
Then we select the lines, where the condition is false:
a[~a%2==0]
Where np.nonzero converts from a boolean array to an array, where all indices with True-values are listed.
For example:
>>> a = np.arange(10)
>>> a[~a%2==0]
array([1, 3, 5, 7, 9])
Related
What I want is to repeat the 10x10 array by 3x3 array. For example, an array of 3x3 contains nine values of indexes [0][0:3], [1][0:3], [2][0:3], and I want to find the max value of these nine values and apply them to a new array. I will add a picture and what I tried.
enter image description here
[1
array_33 = []
new_list = []
for i in range(10):
for j in range(10):
array_33.append([i:i+3])
max_value = max(map(max, array_33) # to find a max_vlaue in 3x3 array
new_list.append(max_value)
One row succeeded in finding a value up to index [0:3], but the next row failed to find a way to get a value up to [0:3]. The value [8:10] is not divided by 3, so the value is added to the new array as it is. Then I want to do a repetitive task of finding a value of [0:3] from lines 4 to 6. I dont know how can i do this
you can use 2D slicing from numpy:
import numpy as np
a = np.array([[(i+1)%10 for i in range(10)]]*10)
print(a)
sz = 3
b = [[np.max(a[i:i+3, j:j+3])
for j in range(0,a.shape[1],sz)]
for i in range(0,a.shape[0],sz)]
print(b)
generally, a[row1:row2, col1:col2] will give you the submatrix in those indexes (not including the last index)
Let's say I have an array:
print(arr1.shape)
(188621, 10)
And in the nth column (let's say 4 for this example), I want to check when a value is above a threshold, t. I want to create a new list (of x instances) of the entire row of arr1 when the ith iteration of the 4th column is above threshold t. In other words, it is extracting the ith row from arr1 when the condition in the 4th column is met. So far I have:
arr2 = []
for i in range(0,len(arr1)):
if arr1[i,4] > t:
arr2.append(arr1[i,:])
I have also tried something along the lines of:
for i in range(0,len(arr1)):
if arr1[i,4] > t:
if len(arr2) == 0:
arr2 = arr1[i,:]
else:
arr2 = np.concatenate((arr2,arr1[i,:]))
However, both instances seem to be growing in 1D terms of x*10 instead of a 2D list of (x, 10) when the conditions are met. What am I missing here?
Well, it wasn't that difficult.
arr2 = arr1[np.logical_not(arr1[:,4] < t)]
I have a numpy array of numpy arrays (would be happy to work with a list of numpy arrays), and I want to edit the overall array. More specifically, I check if arrays (within the larger array) share values, and if they do, I remove the shared values from the smaller array.
The issue I'm having is that when I try to reinsert the modified arrays into the all encompassing array, the final output when the while loop is finished does not remember the updated modules.
I believe this is something to do with pythons nuances of copy/view items, and that when I access element i or j of the overall array, I'm making a new object within the while loop rather than editing the element within the larger array. However, I'm happy to admit I don't fully understand this and definitively can't think of an alternative despite hours of trying.
#Feature_Modules is an array (or list) of number arrays, each containing a set of integers
i = 0
j = 0
while i < Feature_Modules.shape[0]: # Check element i against every other element j
if i != j:
Ref_Module = Feature_Modules[i]
while j < Feature_Modules.shape[0]:
if i != j:
Query_Module = Feature_Modules[j]
if np.array_equal(np.sort(Ref_Module),np.sort(Query_Module)) == 1: # If modules contain exactly the same integers, delete one of this. This bit actually works and is outputted at the end.
Feature_Modules = np.delete(Feature_Modules,j)
Shared_Features = np.intersect1d(Ref_Module, Query_Module)
if Shared_Features.shape[0] > 0 and np.array_equal(np.sort(Ref_Module),np.sort(Query_Module)) == 0: # If the modules share elements, remove the shared elements from the smaller module. This is the bit that isn't outputted in the final Feature_Modules object.
Module_Cardinalities = np.array([Ref_Module.shape[0],Query_Module.shape[0]])
Smaller_Group = np.where(Module_Cardinalities == np.min(Module_Cardinalities))[0][0]
New_Groups = np.array([Ref_Module,Query_Module])
New_Groups[Smaller_Group] = np.delete(New_Groups[Smaller_Group],np.where(np.isin(New_Groups[Smaller_Group],Shared_Features) == 1))
Feature_Modules = Feature_Modules.copy()
Feature_Modules[i] = New_Groups[0] # Replace the current module of Feature_Modules with the new module (Isn't outputted at end of while loops)
Feature_Modules[j] = New_Groups[1] # Replace the current module of Feature_Modules with the new module (Isn't outputted at end of while loops)
else:
j = j + 1
else:
j = j + 1
else:
i = i + 1
i = i + 1
So if we use this small data set as an example,
Feature_Modules = np.array([np.array([1,2,3,4,5,6,7,8]),np.array([9,10,1,2,3,4]), np.array([20,21,22,23])])
The new Feature_Modules should be;
Feature_Modules = np.array([np.array([1,2,3,4,5,6,7,8]), np.array([9,10]), np.array([20,21,22,23])])
since the shared values in array's [0] and [1], were removed from the [1] as it was the smaller array.
I would suggest taking a more python X numpy approach to the code:
import numpy as np
Feature_Modules = np.array([np.array([1,2,3,4,5,6,7,8]), np.array([9,10,1,2,3,4]), np.array([20,21,22,23])])
for n1,arr1 in enumerate(Feature_Modules[:-1]):
l1 = len(arr1)
for n2,arr2 in enumerate(Feature_Modules[n1+1:]):
l2 = len(arr2)
intersect, ind1, ind2 = np.intersect1d(arr1, arr2, return_indices=True)
if len(intersect) == 0:
continue
if l1 > l2:
Feature_Modules[n2+n1+1] = np.delete(arr2, ind2)
else:
Feature_Modules[n1] = np.delete(arr1, ind1)
# [array([1, 2, 3, 4, 5, 6, 7, 8]) array([ 9, 10]) array([20, 21, 22, 23])]
EDIT:
This code will edit the original array in order to keep track of the list that already had element removed. If you want to leave the original array untached, just make a copy of it:
copy = np.array(original)
Ok so I have an array in python. This array holds indices to another array. I removed the indices I wanted to keep from this array.
stations = [1,2,3]
Let's call x the main array. It has 5 columns and I removed the 1st and 5th and put the rest in the array called stations.
I want to be able to create an if statement where the values from stations are excluded. So I'm just trying to find the number of instances (days) where the indices in the stations array are 0 and the other indices (0 and 4) are not 0.
How do I go about doing that? I have this so far, but it doesn't seem to be correct.
for j in range(len(x)):
if x[j,0] != 0 and x[j,4] != 0 and numpy.where(x[j,stations[0]:stations[len(stations)-1]]) == 0:
days += 1
return days
I don't think your problem statement is very clear, but if you want the x cols such that you exclude the indices contained in stations then do this.
excluded_station_x = [col for i, col in enumerate(x) if i not in stations]
This is a list comprehension, its a way for building a new list via transversing an iterable. Its the same as writing
excluded_station_x = []
for i, col in enumerate(x):
if i not in stations:
excluded_station_x.append(col)
enumerate() yields both the value and index of each element as we iterate through the list.
As requested, I will do it without enumerate.
You could also just del each of the bad indices, although I dislike this because it mutates the original list.
for i in stations:
del x[i]
Say I have an arbitrary array of variable order N. For example:
A is a 2x3x3 array is an order 3 array with 2,3, and 3 dimiensions along it's three indices.
I would like to efficiently loop through each element. If I knew a priori the order then I could do something like (in python),
#for order 3
import numpy as np
shape = np.shape(A)
i = 0
while i < shape[0]:
j = 0
while j < shape[1]:
k = 0
while k < shape[2]:
#code using i,j,k
k += 1
j += 1
i += 1
Now suppose I don't know the order of A, i.e. I don't know a priori the length of shape. How can I permute the quickest through all elements of the array?
There are many ways to do this, e.g. iterating over a.ravel() or a.flat. However, looping over every single element of an array in a Python loop will never be particularly efficient.
I don't think it matters which index you choose to permute over first, which index you choose to permute over second, etc. because your inner-most while statement will always be executed once per combination of i, j, and k.
If you need to keep the results of your operation (and assuming its a function of A and i,j,k) You'd want to use something like this:
import itertools
import numpy as np
results = ( (position, code(A,position))
for indices in itertools.product(*(range(i) for i in np.shape(A))))
Then you can iterate the results getting out the position and return value of code for each position. Or convert the generator expression to a list if you need to access the results multiple times.
If the array of of the format array = [[[1,2,3,4],[1,2]],[[1],[1,2,3]]]
You could use the following structure:
array = [[[1,2,3,4],[1,2]],[[1],[1,2,3]]]
indices = []
def iter_array(array,indices):
indices.append(0)
for a in array:
if isinstance(a[0],list):
iter_array(a,indices)
else:
indices.append(0)
for nonlist in a:
#do something using each element in indices
#print(indices)
indices.append(indices.pop()+1)
indices.pop()
indices.append(indices.pop()+1)
indices.pop()
iter_array(array,indices)
This should work for the usual nested list "arrays" I don't know if it would be possible to mimic this using numpy's array structure.