Python 3.x
I have for loop which is making some calculations and creating one Slice/2D Array lets say (x = 3, y = 3) per iteration and I want at the same time in the same for loop (append?/stack) them in a third dimension.
I have been trying with Numpy stack, vstack, hstack, dstack but I still don't get how to get them together in the 3rd dimension as I want.
So I would like to have at them end something like this:
(z = 10, x = 3, y = 3)
array([ [[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.]],
[[2., 2., 2.],
[2., 2., 2.],
[2., 2., 2.]],
.
.
.
])
Thanks,
you can do it like this
arrays = []
for i in range(5):
arr = np.full((3,3), i)
arrays.append(arr)
np.asarray(arrays)
If you want to you can do np.asarray(arrays) inside loop. But it will be not very efficient. Not that np.concatenate will also effectively creates new numpy array so efficiency will be similar. Doing these operation once outside the loop is better
Related
During initializing, I tried to reduce the repeats in my code, so instead of:
output= (torch.zeros(2, 3),
torch.zeros(2, 3))
I wrote:
z = torch.zeros(2, 3)
output= (z,z)
However, I find that the second method is wrong.
If I assign the data to variables h,c, any change on h would also be applied to c
h,c = output
print(h,c)
h +=torch.ones(2,3)
print('-----------------')
print(h,c)
results of the test above:
tensor([[0., 0., 0.],
[0., 0., 0.]]) tensor([[0., 0., 0.],
[0., 0., 0.]])
-----------------
tensor([[1., 1., 1.],
[1., 1., 1.]]) tensor([[1., 1., 1.],
[1., 1., 1.]])
Is there a more elegent way to initialize two indenpendent variables?
I agree that your initial line needs no modification but if you do want an alternative, consider:
z = torch.zeros(2, 3)
output= (z,z.clone())
The reason the other one (output = (z,z)) doesn't work, as you've correctly discovered is that no copy is made. You're only passing the same reference in each entry of the tuple to z
Assign it in a single statement but use separate reference for each, as below:
h, c = output, output
I have a list of numpy arrays and want to remove duplicates and also keep the order of my sorted data. This is my array with duplicates:
dup_arr=[np.array([[0., 10., 10.],\
[0., 2., 30.],\
[0., 3., 5.],\
[0., 3., 5.],\
[0., 3., 40.]]),\
np.array([[0., -1., -4.],\
[0., -2., -3.],\
[0., -3., -5.],\
[0., -3., -6.],\
[0., -3., -6.]])]
I tried to do it using the following code:
clean_arr=[]
for i in dup_arr:
new_array = [tuple(row) for row in i]
uniques = np.unique(new_array, axis=0)
clean_arr.append(uniques)
But the problem of this method is that it changes the sort of my data and I do not want to to sort them again because it is a tough task for my real data. I want to have the following result:
clean_arr=[np.array([[0., 10., 10.],\
[0., 2., 30.],\
[0., 3., 5.],\
[0., 3., 40.]]),\
np.array([[0., -1., -4.],\
[0., -2., -3.],\
[0., -3., -5.],\
[0., -3., -6.]])]
But the code shuffle it. I also tried the foolowing for loops but it was not also successful because I can not iterate until the end of my data and stop the second for loop before reaching to the end of each array of my list.
clean_arr=[]
for arrays in dup_arr:
for rows in range (len(arrays)-1):
if np.all(arrays [rows]== arrays [rows+1]):
continue
else:
dat= arrays [rows]
clean_arr.append(dat)
In advance, I do appreciate any help and contribution.
You can simply use np.unique with axis=0. If you want to keep the order from the original sequence try this -
[i[np.sort(np.unique(i, axis=0, return_index=True)[1])] for i in dup_arr]
[array([[ 0., 10., 10.],
[ 0., 2., 30.],
[ 0., 3., 5.],
[ 0., 3., 40.]]),
array([[ 0., -1., -4.],
[ 0., -2., -3.],
[ 0., -3., -5.],
[ 0., -3., -6.]])]
np.unique(i, axis=0, return_index=True)[1] returns the indexes of the unique elements.
np.sort() sorts these indexes back to original sequence in array.
[f(i) for i in dup_arr] applies the above 2 steps over each element in dup_arr.
NOTE: You will NOT be able to completely vectorize this operation (say by np.stack on this operations since it will may have variable duplicates removed from each matrix. This will cause the numpy array to have unequal shapes over an axis.
Breaking the steps as a function -
def f(a):
indexes = np.unique(a, axis=0, return_index=True)[1]
return a[np.sort(indexes)]
[f(i) for i in dup_arr]
In my humble opinion, every time you want to remove duplicates from an array or list in Python, you should consider using a set.
Also, try to avoid using multiple nested loops since errors easily occur and they're hard to find. I suggest you give the following code a try:
removed_duplicates=[]
for subarr in dup_arr:
removed_duplicates.append(np.array([list(item) for item in set(tuple(row) for row in subarr)]))
Basically what's happening is that you convert your array to a tuple, then to a set that removes all duplicates, and then to a list. Since your original data had an array of np.arrays, your convert the list back to a np.array before you append it to the new array.
Would this work?
I am trying to do multiple indices. However, it is showing the error of too many indices. Please help me and if you have some doubts or confusion in it then please leave comments in the comment box.
My result has (6561114,) shape and I want to indices the whole first row for instance ([array([-1., 1., 0., 0., 1.]), array([[43., 0., 43., 1., 2.]]), array([-43., 43., 0., 2., 3.]) and then from 3 columns I want to extract each value of array-like [-1,43,-43],[1,0,43], and so on.
This is Output:-
array([array([-1., 1., 0., 0., 1.]),
array([[43., 0., 43., 1., 2.]]),
array([-43., 43., 0., 2., 3.]), ...,
array([-1.406830e+01, 3.552240e+01, 2.145410e+01,
9.492236e+06,
9.492237e+06]),
array([[1.421949e+02, 2.145410e+01, 1.636490e+02, 9.492237e+06,
9.492238e+06],
[3.387300e+01, 1.636490e+02, 1.975220e+02, 9.492238e+06,
9.492239e+06]]),
array([-1.9052487e+02, 1.9752200e+02, 6.9971300e+00,
9.4922390e+06,
9.4922400e+06])], dtype=object)
This is what error looks like:-
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-29-537ba6ddfd42> in <module>
----> 1 result1[0,:]
IndexError: too many indices for array
Check in second array you are using 2 [] braces.
Here array([[43., 0., 43., 1., 2.]]) you are using 2 [] braces, remove [] braces, and use it
I am using this code, and its working fine
Check using print(a[0]) and post your output.
Code
import numpy as np
a=np.array([np.array([-1., 1., 0., 0., 1.]),np.array([43., 0., 43., 1., 2.]),np.array([-43., 43., 0.2., 3.])],dtype=object)
print(a[:,0])
I was wondering how I could name a matrix variable.
Let's say the matrix a
var=din3
a
array([[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.]])
I want to concatenate the name of the variable with a string.
I want to do something like this: a+"IM"+var
so that the resulting variable would be aIMdin3. So when I call aIMdin3 I get the matrix, instead of calling a.
How could I do this?
There is a way to dynamically create variable by playing with globals(), but I wouldn't recommend this type of "meta-programming"
In my humble opinion, it's best to use dictionary instead and generating your dynamic variables by creating dynamic keys to your dictionary.
var = "din3"
d = {}
d['aIM'+var] = a
For more details, visit How can you dynamically create variables via a while loop?
For example I need 30x30 numpy arrays created from images to be fed to a neural net. If I have a directory of images to predict, I should be able to loop through the directory, get image data and create an (n,30,30) shape np array
This is my current method, I intend to reshape each row before feeding to the model
def get_image_vectors(path):
img_list=os.listdir(path)
print(img_list)
X=np.empty((900,))
for img_file in img_list:
img= Image.open(os.path.join(path,img_file))
img_grey= img.convert("L")
resized = img_grey.resize((30,30))
flattened = np.array(resized.getdata())
# print(flattened.shape)
X=np.vstack((X,flattened))
print(img_file,'=>',X.shape)
return X[1:,:]
Instead of appending to an existing array, it will probably be better to use a list initially, appending to it, and converting to an array at the end. thus saving many redundant modifications of np arrays.
Here a toy example:
import numpy as np
def get_image_vectors():
X= [] #Create empty list
for i in range(10):
flattened = np.zeros(900)
X.append(flattened) #Append some np array to it
return np.array(X) #Create array from the list
With result:
array([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]])