I need to convert array like this:
[[1527 1369 86 86]
[ 573 590 709 709]
[1417 1000 68 68]
[1361 1194 86 86]]
to like this:
[(726, 1219, 1281, 664),
(1208, 1440, 1283, 1365),
(1006, 1483, 1069, 1421),
(999, 1414, 1062, 1351),]
I tried using convert diretly to tuple but got this:
( array([1527, 1369, 86, 86], dtype=int32),
array([573, 590, 709, 709], dtype=int32),
array([1417, 1000, 68, 68], dtype=int32),
array([1361, 1194, 86, 86], dtype=int32))
(array([701, 899, 671, 671], dtype=int32),)
The array method tolist is a easy and fast way of converting an array to a list. It handles multiple dimensions correctly:
In [92]: arr = np.arange(12).reshape(3,4)
In [93]: arr
Out[93]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
In [94]: arr.tolist()
Out[94]: [[0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10, 11]]
For most purposes such as list of lists is just as good as a list of tuples, or tuple of tuples. They differ only in mutability.
But if you must have a tuples, a list comprehension does the conversion nicely.
In [95]: [tuple(x) for x in arr.tolist()]
Out[95]: [(0, 1, 2, 3), (4, 5, 6, 7), (8, 9, 10, 11)]
An alternative [tuple(x) for x in arr] is a bit slower, because it is iterating on the array rather than on a list. It also produces a different result - though you have to examine the type of the tuple elements to see that.
I strongly recommend starting with the tolist method, and doing any list to tuple conversions after.
What about using tuble and map function like this:
import numpy
numpy_arr = numpy.array(((1527, 1369, 86, 86),(573 , 590 , 709, 709)))
converted_list = tuple(map(tuple,numpy_arr)) # as list
converted_arr = map(tuple,numpy_arr) #as array
print(converted_arr)
Here is the following function assuming you do not want the final object to be a numpy object.
def fun(var):
a=[]
for i in var:
a.append(tuple(i))
return a
if you want in one line
def fun(var):
return [tuple(i) for i in var]
If you prefer list comprehensions to map():
a = numpy.random.uniform(0,1,size=(4,4))
a_tuple_list = [tuple(row) for row in a]
Related
I want to modify rows of numpy arrays stored in a list. length of my numpy arrays are not the same. I have several huge numpy arrays stored as list. This is my data (for simplicity I copied only a small list of array):
elements= [array([[971, 466, 697, 1, 15, 18, 28],
[5445, 4, 301, 2, 12, 47, 5]]),
array([[5883, 316, 377, 2, 9, 87, 1]])]
Then, I want to replace the fourth column of each row with the last one and then delete the last column. I want to have the following result:
[array([[971, 466, 697, 1, 28, 18],
[5445, 4, 301, 2, 5, 47]]),
array([[5883, 316, 377, 2, 1, 87]])]
I tried the following code but it was not successful:
length=[len(i) for i in elements] # To find the length of each array
h=sum(length) # to find the total number of rows
for i in range (h):
elements[:,[4,-1]] = elements[:,[-1,4]]
elements=np.delete(elements,[-1],1)
I am facing the following error:
TypeError: list indices must be integers or slices, not tuple
I appreciate ay help in advance.
You can do it without loops but it's still slower (1.75 times on large data) than accepted solution:
counts = list(map(len, elements))
arr = np.concatenate(elements)
arr[:, 4] = arr[:, -1]
new_elements = np.split(arr[:,:-1], np.cumsum(counts)[:-1])
Concatenation is quite slow in numpy.
A simple inefficient solution:
import numpy as np
elements= [np.array([[971, 466, 697, 1, 15, 18, 28],
[5445, 4, 301, 2, 12, 47, 5]]),
np.array([[5883, 316, 377, 2, 9, 87, 1]])]
new_elements = list()
for arr in elements:
arr[:, 4] = arr[:, -1]
new_elements.append(arr[:, :-1])
The new list output is:
new_elements
Out[11]:
[array([[ 971, 466, 697, 1, 28, 18],
[5445, 4, 301, 2, 5, 47]]),
array([[5883, 316, 377, 2, 1, 87]])]
Try this one
p=[]
for x in range(len(elements)):
for y in range(len(elements[x])):
p.append(list(elements[x][y][:4])+[elements[x][y][-1]]+[elements[x][y][-2]])
print(p)
[[971, 466, 697, 1, 28, 18],
[5445, 4, 301, 2, 5, 47],
[5883, 316, 377, 2, 1, 87]]
I have a 6x6 matrix: e.g. matrix A
array([[ 0, 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35]])
I also have a 3x3x3 matrix: e.g. matrix B
array([[[ 1, 7, 2],
[ 5, 9, 3],
[ 2, 8, 6]],
[[ 3, 4, 6],
[ 6, 8, 9],
[ 4, 2, 8]],
[[ 6, 4, 7],
[ 8, 7, 8],
[ 4, 4, 7]]])
Finally, I have a 3x4x4 matrix C, (4 rows, 4 columns, 3 dimensions), that's empty (filled with 0s)
I want to multiply each "3rd dimension" of B (i.e. [1,:,:],[2,:,:],[3,:,:]) with A. However, for each dimension I want to multiply B in "windows", sliding by 1 each time across A till I cannot go further, at which point I move back to the beginning, slide 1 unit down and again sliding across one-by-one multiplying B with A, till the end, then move down and repeat till you don't go over the border. The results being stored in the respective "3rd dimension" of matrix C. So my result would be a [3x4x4] matrix.
Ex. (multiplication is dot product giving a scalar value, np.sum((np.multiply(x,y)))), so...
imagining B "overtop" of A, starting in the right corner, I multiply that 3x3 part of A with Bs [1x3x3] part storing the result in C...
referring to 1st unit (located in 1st row and 1st column) in the 1st dimension of C...
C[1,0,0] = 340. because [[0,1,2],[6,7,8],[12,13,4]] dot product [[1,7,2],[5,9,3],[2,8,6]]
sliding B matrix over by 1 on A, and storing my 2nd result in C...
C[1,0,1] = 383. because [[1,2,3],[7,8,9],[13,14,15]] dot product [[1,7,2],[5,9,3],[2,8,6]]
Then repeat this procedure of sliding across and down and across and ..., for B[2,:,:] and B[3,:,:] over A again, storing in C2,:,:] and C[3,:,:] respectively.
What is a good way to do this?
I think you're asking about 2D cross-correlation with three different kernels, rather than straightforward matrix multiplication.
The following piece of code is not the most efficient way to do this, but does this give you the answer you are looking for? I'm using scipy.signal.correlate2d to achieve 2D correlation here...
>>> from scipy.signal import correlate2d
>>> C = np.dstack([correlate2d(A, B[:, :, i], 'valid') for i in range(B.shape[2])])
>>> C.shape
(4, 4, 3)
>>> C
array([[[ 333, 316, 464],
[ 372, 369, 520],
[ 411, 422, 576],
[ 450, 475, 632]],
[[ 567, 634, 800],
[ 606, 687, 856],
[ 645, 740, 912],
[ 684, 793, 968]],
[[ 801, 952, 1136],
[ 840, 1005, 1192],
[ 879, 1058, 1248],
[ 918, 1111, 1304]],
[[1035, 1270, 1472],
[1074, 1323, 1528],
[1113, 1376, 1584],
[1152, 1429, 1640]]])
Here's a more "fun" way of doing this which doesn't use scipy, but using stride_tricks instead. I'm not sure if it's more efficient:
>>> import numpy.lib.stride_tricks as st
>>> s, t = A.strides
>>> i, j = A.shape
>>> k, l, m = B.shape
>>> D = st.as_strided(A, shape=(i-k+1, j-l+1, k, l), strides=(s, t, s, t))
>>> E = np.einsum('ijkl,klm->ijm', D, B)
>>> (E == C).all()
True
Hi let's say that I have two lists in python and I want to remove common values from both lists. A potential solution is:
x = [1, 2, 3, 4, 5, 6, 7, 8]
y = [43, 3123, 543, 76, 879, 32, 14241, 342, 2, 3, 4]
for i in x:
if i in y:
x.remove(i)
y.remove(i)
it seems correct but it is not. The reason, I guess, is because by removing an item from the list the index continues to iterate. Therefore, for two common values in the lists where the values are near each other we will be missing the later values (the code will not iterate through it).
The result would be:
>>> x
[1, 3, 5, 6, 8, 9, 10]
>>> y
[43, 3123, 543, 76, 879, 32, 14241, 342, 3]
So we are missing the value '3'.
Is the reason of that behaviour the one that I mentioned? or am I doing something else wrong?
Just slight change your code,Iterate through the copy of x it's x[:].You are modifying the list while iterating over it. So that's why you are missing value 3
for i in x[:]:
if i in y:
x.remove(i)
y.remove(i)
And alternative method
x,y = [i for i in x if i not in y],[j for j in y if j not in x]
You can also use difference of set objects.
a = list(set(y) - set(x))
b = list(set(x) - set(y))
z=[i for i in x if i not in y]
w=[i for i in y if i not in x]
x=z
y=w
That should do the trick? It's a bit less memory efficient.
If you use numpy, then all you need is:
x, y = np.setdiff1d(x, y), np.setdiff1d(y, x)
and if you don't want to use numpy:
x, y = list(set(x).difference(y)), list(set(y).difference(x))
I personally think Python's set data type is the way to go:
You can do something like this:
>>> x = [1, 2, 3, 4, 5, 6, 7, 8]
>>> y = [43, 3123, 543, 76, 879, 32, 14241, 342, 2, 3, 4]
>>> sx = set(x)
>>> sy = set(y)
>>> sx.union(sy)
set([1, 2, 3, 4, 5, 6, 7, 8, 32, 43, 76, 342, 543, 879, 3123, 14241])
Or you can reduce it to a one liner:
list(set(x).union(set(y)))
You can use set method to remove elements in list.
s1=input()
s2=input()
str=list(set(s1).symmetric_difference(set(s2)))
print(str)
I have a numpy array:
import numpy as np
a = np.array([2, 56, 4, 8, 564])
and I want to add two elements: one at the beginning of the array, 88, and one at the end, 77.
I can do this with:
a = np.insert(np.append(a, [77]), 0, 88)
so that a ends up looking like:
array([ 88, 2, 56, 4, 8, 564, 77])
The question: what is the correct way of doing this? I feel like nesting a np.append in a np.insert is quite likely not the pythonic way to do this.
Another way to do that would be to use numpy.concatenate . Example -
np.concatenate([[88],a,[77]])
Demo -
In [62]: a = np.array([2, 56, 4, 8, 564])
In [64]: np.concatenate([[88],a,[77]])
Out[64]: array([ 88, 2, 56, 4, 8, 564, 77])
You can use np.concatenate -
np.concatenate(([88],a,[77]))
You can pass the list of indices to np.insert :
>>> np.insert(a,[0,5],[88,77])
array([ 88, 2, 56, 4, 8, 564, 77])
Or if you don't know the length of your array you can use array.size to specify the end of array :
>>> np.insert(a,[0,a.size],[88,77])
array([ 88, 2, 56, 4, 8, 564, 77])
what about:
a = np.hstack([88, a, 77])
if I have a list of start and end indices for a given matrix in each dimension, is there any way to do slicing in an efficient way?
For example:
a=10
b=10
x_0=np.zeros((5,a,b))
i=[0, 5, 3, 5, 3]
j=[2, 4, 0, 6, 6]
x_1=x_0[:,i:i+3,j:j+3]
here x_1[k] would be (5,3,3). In the other words:
X_1[0,3,3]=x0[0,i[0]:i[0]+3,j[0]:j[0]+3]
X_1[1,3,3]=x0[1,i[1]:i[1]+3,j[1]:j[1]+3]
X_1[2,3,3]=x0[2,i[2]:i[2]+3,j[2]:j[2]+3]
X_1[3,3,3]=x0[3,i[3]:i[3]+3,j[3]:j[3]+3]
X_1[4,3,3]=x0[4,i[4]:i[4]+3,j[4]:j[4]+3]
or more genrally speaking:
for k in range(5):
x_1[k]=x_0[k,i[k]:i[k]+3,j[k]:j[k]+3]
Any help wold be appreciated.
i = 0
j = 2
x_0[:,i:i+3,j:j+3]
produces a (5,3,3) array.
But what do you want to do with?
i=[0, 5, 3, 5, 3]
j=[2, 4, 0, 6, 6]
Do you want, for example, to produce a (5,3,3) array for each pair of values from i and j?
Or do you want to stack?
x_0[0,0:3,2:5]
x_0[1,5:8,4:7]
...
which could be produced with:
np.array([x0[kk,ii:ii+3,jj:jj+3] for kk,(ii,jj) in enumerate(zip(i,j))])
Here I collect 5 (3,3) arrays and join them into one. It may be possible to join the indexes, and do the indexing once, but that will require some fiddling, and may not improve the speed.
Before worrying about doing things efficiently, lets be clear about what you want to do. My guess is that you will need to iterate in one way or other over the values of i and j, and concatenate values. It doesn't matter much whether you concatenate and then index or index and then concatenate.
If x0=np.arange(500).reshape(5,10,10), the above expression produces:
array([[[ 2, 3, 4],
[ 12, 13, 14],
[ 22, 23, 24]],
[[154, 155, 156],
[164, 165, 166],
[174, 175, 176]],
[[230, 231, 232],
[240, 241, 242],
[250, 251, 252]],
[[356, 357, 358],
[366, 367, 368],
[376, 377, 378]],
[[436, 437, 438],
[446, 447, 448],
[456, 457, 458]]])