Related
This is the array I have at hand:
[array([[[ 4, 9, 1, -3],
[-2, 0, 8, 6],
[ 1, 3, 7, 9 ],
[ 2, 5, 0, -7],
[-1, -6, -5, -8]]]),
array([[[ 0, 2, -1, 6 ],
[9, 8, 0, 3],
[ -1, 2, 5, -4],
[0, 5, 9, 6],
[ 6, 2, 9, 4]]]),
array([[[ 1, 2, 0, 9],
[3, 4, 8, -1],
[5, 6, 9, 0],
[ 7, 8, -3, -],
[9, 0, 8, -2]]])]
But the goal is obtain arrays A from first columns of nested arrays, B from second columns of nested arrays, Cfrom third columns of nested array etc.
Such that:
A = array([4, -2, 1, 2, -1, 0, 9, -1 ,0, 6, 1, 3, 5, 7, 9])
B = array([9, 0, 3, 5, -6, 2, 8, 2, 5, 2, 2,, 4, 6, 8, 0])
How should I do this?
You can do this with a single hstack() and use squeeze() to remove the extra dimension. With that you can use regular numpy indexing to pull out columns (or anything else you want):
import numpy as np
l = [np.array([[[ 4, 9, 1, -3],
[-2, 0, 8, 6],
[ 1, 3, 7, 9 ],
[ 2, 5, 0, -7],
[-1, -6, -5, -8]]]),
np.array([[[ 0, 2, -1, 6 ],
[9, 8, 0, 3],
[ -1, 2, 5, -4],
[0, 5, 9, 6],
[ 6, 2, 9, 4]]]),
np.array([[[ 1, 2, 0, 9],
[3, 4, 8, -1],
[5, 6, 9, 0],
[ 7, 8, -3, -1],
[9, 0, 8, -2]]])]
arr = np.hstack(l).squeeze()
A = arr[:,0]
print(A)
# [ 4 -2 1 2 -1 0 9 -1 0 6 1 3 5 7 9]
B = arr[:,1]
print(B)
#[ 9 0 3 5 -6 2 8 2 5 2 2 4 6 8 0]
# etc...
IIUC,
l = [np.array([[[ 4, 9, 1, -3],
[-2, 0, 8, 6],
[ 1, 3, 7, 9 ],
[ 2, 5, 0, -7],
[-1, -6, -5, -8]]]),
np.array([[[ 0, 2, -1, 6 ],
[9, 8, 0, 3],
[ -1, 2, 5, -4],
[0, 5, 9, 6],
[ 6, 2, 9, 4]]]),
np.array([[[ 1, 2, 0, 9],
[3, 4, 8, -1],
[5, 6, 9, 0],
[ 7, 8, -3, -9],
[9, 0, 8, -2]]])]
a = np.hstack([i[0][:, 0] for i in l])
b = np.hstack([i[0][:, 1] for i in l])
Output:
array([ 4, -2, 1, 2, -1, 0, 9, -1, 0, 6, 1, 3, 5, 7, 9])
array([ 9, 0, 3, 5, -6, 2, 8, 2, 5, 2, 2, 4, 6, 8, 0])
Consider two graphs, G1 = (V1, E1), G2 = (V2, E2)
V1 = {1,2,3,4,5,6}
V2 = {7,8,9,10,11,12}
In space, these vertices are connected by triangles faces (each with three vertices)
F1 = [[ 2, 1, 0], [ 0, 3, 2], [ 1, 4, 0], [ 0, 4, 3], [ 5, 1, 2], [ 3, 5, 2], [ 5, 4, 1], [ 4, 5, 3]]
F2 = [[ 8, 7, 6], [ 6, 9, 8], [ 7, 10, 6], [ 6, 10, 9], [11, 7, 8], [ 9, 11, 8], [11, 10, 7], [10, 11, 9]]
The above is what I am trying to find. If we are given the entire array of faces:
faces = [[ 2, 1, 0], [ 0, 3, 2], [ 1, 4, 0], [ 0, 4, 3], [ 5, 1, 2], [ 3, 5, 2],
[ 5, 4, 1], [ 4, 5, 3], [ 8, 7, 6], [ 6, 9, 8], [ 7, 10, 6], [ 6, 10, 9],
[11, 7, 8], [ 9, 11, 8], [11, 10, 7], [10, 11, 9]]
can we find the connected components, and separate into F1 and F2?
A version of this problem has been solved in Mathematica, but I cannot translate.
My work is found in this post.
Making a graph from your faces is pretty straightforward: each triplet yields 3 edges corresponding to all combinations of the triplet's members. Then it is just a matter of instantiating the networkx Graph object and calling networkx.algorithms.components.connected_components.
#!/usr/bin/env python
"""
Given a list of triangles, find the connected components.
https://stackoverflow.com/q/61584283/2912349
"""
import itertools
import networkx as nx
faces = [[ 2, 1, 0], [ 0, 3, 2], [ 1, 4, 0], [ 0, 4, 3], [ 5, 1, 2], [ 3, 5, 2],
[ 5, 4, 1], [ 4, 5, 3], [ 8, 7, 6], [ 6, 9, 8], [ 7, 10, 6], [ 6, 10, 9],
[11, 7, 8], [ 9, 11, 8], [11, 10, 7], [10, 11, 9]]
#create graph
edges = []
for face in faces:
edges.extend(list(itertools.combinations(face, 2)))
g = nx.from_edgelist(edges)
# compute connected components and print results
components = list(nx.algorithms.components.connected_components(g))
for component in components:
print(component)
# {0, 1, 2, 3, 4, 5}
# {6, 7, 8, 9, 10, 11}
# separate faces by component
component_to_faces = dict()
for component in components:
component_to_faces[tuple(component)] = [face for face in faces if set(face) <= component] # <= operator tests for subset relation
for component, component_faces in component_to_faces.items():
print(component, component_faces)
# (0, 1, 2, 3, 4, 5) [[2, 1, 0], [0, 3, 2], [1, 4, 0], [0, 4, 3], [5, 1, 2], [3, 5, 2], [5, 4, 1], [4, 5, 3]]
# (6, 7, 8, 9, 10, 11) [[8, 7, 6], [6, 9, 8], [7, 10, 6], [6, 10, 9], [11, 7, 8], [9, 11, 8], [11, 10, 7], [10, 11, 9]]
I have a pandas data frame that looks like this:
>>> df = pd.DataFrame({'a': list(range(10))})
>>> df['a'] = df.a.apply(lambda x: x*np.array([1,2,3]))
>>>df.head()
a
0 [0, 0, 0]
1 [1, 2, 3]
2 [2, 4, 6]
3 [3, 6, 9]
4 [4, 8, 12]
I would like to get column a from the df as a ndarray. But when I do that I get an array of arrays
>>> df.a.values
array([array([0, 0, 0]), array([1, 2, 3]), array([2, 4, 6]),
array([3, 6, 9]), array([ 4, 8, 12]), array([ 5, 10, 15]),
array([ 6, 12, 18]), array([ 7, 14, 21]), array([ 8, 16, 24]),
array([ 9, 18, 27])], dtype=object)
How can I get the returnd output to be
array([[ 0, 0, 0],
[ 1, 2, 3],
[ 2, 4, 6],
[ 3, 6, 9],
[ 4, 8, 12],
# ...
])
Using pandas,
df.a.apply(pd.Series).values
Using numpy,
np.vstack(df.a.values)
You get
array([[ 0, 0, 0],
[ 1, 2, 3],
[ 2, 4, 6],
[ 3, 6, 9],
[ 4, 8, 12],
[ 5, 10, 15],
[ 6, 12, 18],
[ 7, 14, 21],
[ 8, 16, 24],
[ 9, 18, 27]])
Check
np.array(df['a'].tolist())
array([[ 0, 0, 0],
[ 1, 2, 3],
[ 2, 4, 6],
[ 3, 6, 9],
[ 4, 8, 12],
[ 5, 10, 15],
[ 6, 12, 18],
[ 7, 14, 21],
[ 8, 16, 24],
[ 9, 18, 27]], dtype=int64)
I have a bunch of matrices that I stored in a big dataframe. Let's say here is my dataframe.
data = pd.DataFrame([[13, 1, 3, 4, 0, 0], [0, 2, 6, 2, 0, 0], [3, 1, 5, 2, 2, 0], [0, 0, 10, 11, 6, 0], [5, 5, 21, 25, 41, 0],
[11, 1, 3, 2, 0, 1], [3, 1, 7, 3, 1, 1], [1, 1, 6, 5, 3, 1], [1, 1, 6, 7, 6, 1], [6, 6, 21, 24, 42, 1],
[17, 1, 7, 0, 0, 2], [1, 1, 6, 1, 1, 2], [2, 4, 6, 2, 1, 2], [0, 2, 11, 7, 8, 2], [5, 6, 17, 16, 46, 2],
[11, 1, 10, 2, 1, 3], [2, 2, 7, 1, 1, 3], [0, 0, 14, 4, 1, 3], [0, 0, 7, 7, 5, 3], [5, 1, 20, 18, 48, 3],
[16, 3, 7, 1, 2, 4], [1, 2, 4, 1, 0, 4], [2, 4, 7, 5, 3, 4], [3, 0, 4, 4, 7, 4], [7, 2, 13, 12, 58, 4]],
columns=['1', '2', '3', '4', '5', 'iteration'])
print(pd.DataFrame(data))
Each data['iteration'] is a matrix on its own. So, as you can see there are 5 matrices here (iteration-0 to 4). I want to add them all, like in basic matrix addition, to get one single matrix.
I have tried the following, but there's something wrong with it. It doesn't work.
matrix = data[['1','2','3','4','5']]
print(np.sum([matrix[matrix_list['iteration']==i] for i in range(0,9)], axis=0))
How do I do this the right way?
You can use:
In [98]: d = data.set_index('iteration')
In [99]: np.sum(d.loc[i].values for i in d.index.drop_duplicates().values)
Out[99]:
array([[ 68, 7, 30, 9, 3],
[ 7, 8, 30, 8, 3],
[ 8, 10, 38, 18, 10],
[ 4, 3, 38, 36, 32],
[ 28, 20, 92, 95, 235]])
Or alternatively, use groupby():
np.sum(e[1].iloc[:, :-1].values for e in data.groupby('iteration'))
array([[ 68, 7, 30, 9, 3],
[ 7, 8, 30, 8, 3],
[ 8, 10, 38, 18, 10],
[ 4, 3, 38, 36, 32],
[ 28, 20, 92, 95, 235]])
Is there a fast way of getting the output of argwhere in the output of where format ?
Let me show you what I'm doing with a bit of code:
In [123]: filter = np.where(scores[:,:,:,4,:] > 21000)
In [124]: filter
Out[124]:
(array([ 2, 2, 4, 4, 4, 4, 4, 4, 4, 4, 4, 23, 23, 23, 23, 23]),
array([13, 13, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5]),
array([0, 1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2]),
array([44, 44, 0, 1, 2, 3, 6, 8, 12, 14, 22, 31, 58, 76, 82, 41]))
In [125]: filter2 = np.argwhere(scores[:,:,:,4,:] > 21000)
In [126]: filter2
Out[126]:
array([[ 2, 13, 0, 44],
[ 2, 13, 1, 44],
[ 4, 4, 3, 0],
[ 4, 4, 3, 1],
[ 4, 4, 3, 2],
[ 4, 4, 3, 3],
[ 4, 4, 3, 6],
[ 4, 4, 3, 8],
[ 4, 4, 3, 12],
[ 4, 4, 3, 14],
[ 4, 4, 3, 22],
[23, 4, 2, 31],
[23, 4, 2, 58],
[23, 4, 2, 76],
[23, 4, 2, 82],
[23, 5, 2, 41]])
In [150]: scores[:,:,:,4,:][filter]
Out[150]:
array([ 21344., 21344., 24672., 24672., 24672., 24672., 25232.,
25232., 25232., 25232., 24672., 21152., 21152., 21152.,
21152., 21344.], dtype=float16)
In [129]: filter2[np.argsort(scores[:,:,:,4,:][filter])]
Out[129]:
array([[23, 4, 2, 31],
[23, 4, 2, 58],
[23, 4, 2, 76],
[23, 4, 2, 82],
[ 2, 13, 0, 44],
[ 2, 13, 1, 44],
[23, 5, 2, 41],
[ 4, 4, 3, 0],
[ 4, 4, 3, 1],
[ 4, 4, 3, 2],
[ 4, 4, 3, 3],
[ 4, 4, 3, 22],
[ 4, 4, 3, 6],
[ 4, 4, 3, 8],
[ 4, 4, 3, 12],
[ 4, 4, 3, 14]])
129 is my desired output, so my code works, but I'm trying to make it as fast as possible. Should I get filter2 with np.array(filter).transpose() ? Is there something even better ?
Edit, trying to put it more clearly: I want a list of indices ordered by the value they return when applied to an array. To do that, I need both the output of np.where and np.argwhere, and I'm wondering what is the fastest way to switch from one output to the other, or if there's another of getting my result.
Look at the code for argwhere:
return transpose(asanyarray(a).nonzero())
while where docs say:
where(condition, [x, y])
If only condition is given, return condition.nonzero().
In effect, both use a.nonzero(). One uses it as is, the other transposes it.
In [933]: x=np.zeros((2,3),int)
In [934]: x[[0,1,0],[0,1,2]]=1
In [935]: x
Out[935]:
array([[1, 0, 1],
[0, 1, 0]])
In [936]: x.nonzero()
Out[936]: (array([0, 0, 1], dtype=int32), array([0, 2, 1], dtype=int32))
In [937]: np.where(x) # same as nonzero()
Out[937]: (array([0, 0, 1], dtype=int32), array([0, 2, 1], dtype=int32))
In [938]: np.argwhere(x)
Out[938]:
array([[0, 0],
[0, 2],
[1, 1]], dtype=int32)
In [939]: np.argwhere(x).T
Out[939]:
array([[0, 0, 1],
[0, 2, 1]], dtype=int32)
argwhere().T is the same as where except in a 2d rather than a tuple.
np.transpose(filter) and np.array(filter).T look equally good. For a large array the time spent in nonzero is much larger than the time spent on these transformations.