During communities' detection I am trying to remove duplicates nodes from lists of lists (aimed to calculate ARI).
What I have – few dozen lists inside one list with different dimensions:
lst_of_lts= [[5192, 32896, 34357, 34976, 36683, 43315], … ,[19, 92585, 94137, 98381, 99041, 100395, 101100, 109759]]
What I am running:
import itertools
Lst_of_lts.sort()
Lst_of_lts_2 = list(k for k,_ in itertools.groupby(Lst_of_lts))
Lst_of_lts_nodops= [list(i) for i in {tuple(sorted(i)) for i in Lst_of_lts_2}]
For some reason, it doesn’t remove duplicates.
The dimensions remain the same-
Any suggestions?
Also tried many options such as:
Remove duplicate items from lists in Python lists and
Remove duplicated lists in list of lists in Python
If you are removing duplicates just in the list itself, you can use set.
a = np.random.randint(0,5,(10,10)).tolist()
a
Out[128]:
[[0, 3, 0, 2, 4, 4, 0, 0, 3, 3],
[2, 4, 0, 2, 4, 2, 2, 4, 3, 1],
[3, 2, 0, 1, 2, 0, 2, 0, 2, 1],
[3, 1, 4, 1, 0, 1, 4, 4, 3, 4],
[2, 0, 1, 1, 0, 4, 1, 4, 2, 3],
[0, 0, 1, 3, 4, 3, 1, 3, 0, 1],
[1, 2, 0, 2, 1, 3, 4, 2, 2, 0],
[3, 3, 2, 2, 0, 4, 1, 1, 0, 0],
[0, 1, 3, 0, 4, 4, 2, 1, 1, 4],
[0, 1, 4, 4, 0, 1, 3, 2, 1, 1]]
[list(set(i)) for i in a]
Out[129]:
[[0, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3],
[0, 1, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]]
Or if you want to preserve the order of the element, you can use dict.fromkeys
[list(dict.fromkeys(i)) for i in a]
Out[133]:
[[0, 3, 2, 4],
[2, 4, 0, 3, 1],
[3, 2, 0, 1],
[3, 1, 4, 0],
[2, 0, 1, 4, 3],
[0, 1, 3, 4],
[1, 2, 0, 3, 4],
[3, 2, 0, 4, 1],
[0, 1, 3, 4, 2],
[0, 1, 4, 3, 2]]
Related
I need to create array 5x5, which looks like:
[[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]]
I want to do it with numpy using ndarray, I write np.ndarray((5, 5)) and then fill it as I need, but the answer is can I fill it during creation with this pattern?
Use numpy.tile (to repeat an array specified number of times):
np.tile(np.arange(5), (5,1))
array([[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]])
I'm working with lists and something came up, so let's say I have a 3D list and I want to create a 4D list, so every two positions in the inner lists split, here is what I tried
mylist = [[[0, 2, 1], [0, 3, 1], [0, 4, 3, 1], [0, 4, 3, 1], [0, 3, 2, 1], [0, 2, 3, 4, 1]],
[[0, 2, 1], [0, 4, 2, 3, 1], [0, 4, 3, 1], [0, 4, 3, 1], [0, 3, 2, 1], [0, 2, 3, 4, 1]]]
newlist = [mylist[i: i + 2] for i in range(0, len(mylist), 2)]
print(newlist)
newlist = [[[[0, 2, 1], [0, 3, 1], [0, 4, 3, 1], [0, 4, 3, 1], [0, 3, 2, 1], [0, 2, 3, 4, 1]],
[[0, 2, 1], [0, 4, 2, 3, 1], [0, 4, 3, 1], [0, 4, 3, 1], [0, 3, 2, 1], [0, 2, 3, 4, 1]]]]
but I was expecting something like:
newlist = [[[[0, 2, 1], [0, 3, 1]], [[0, 4, 3, 1], [0, 4, 3, 1]], [[0, 3, 2, 1], [0, 2, 3, 4, 1]]],
[[0, 2, 1], [0, 4, 2, 3, 1]], [[0, 4, 3, 1], [0, 4, 3, 1]], [[0, 3, 2, 1], [0, 2, 3, 4, 1]]]]
I believe I'm missing a for in my list comprehension something like:
newlist = [[mylist[j: j + 2] for j in i] for i in range(0, len(my list), 2)]
but I'm having an error and I can't figure it out what is the problem, so any help will appreciated, thank you so much!
Try this:
newlist=[[list(ls) for ls in zip(i[::2], i[1::2])] for i in mylist]
print(newlist)
Output:
[[[[0, 2, 1], [0, 3, 1]],
[[0, 4, 3, 1], [0, 4, 3, 1]],
[[0, 3, 2, 1], [0, 2, 3, 4, 1]]],
[[[0, 2, 1], [0, 4, 2, 3, 1]],
[[0, 4, 3, 1], [0, 4, 3, 1]],
[[0, 3, 2, 1], [0, 2, 3, 4, 1]]]]
Here is a possible solution. You were very close!
newlist = [[lst[i:i+2] for i in range(0, len(lst), 2)] for lst in mylist]
This is my current matrix:
[[0, 1, 2, 4],
[0, 3, 1, 3],
[0, 2, 3, 2],
[0, 2, 4, 1],
[0, 4, 1, 2],
[0, 3, 2, 2],
[1, 2, 2, 2]]
I want to transpose it and get this as output:
[[0, 0, 0, 0, 1],
[2, 2, 4, 3, 2],
[3, 4, 1, 2, 2],
[2, 1, 2, 2, 2]]
I used inverse = np.swapaxes(ate,0,7) but I am not sure what will be my axis2 value be. Here the axis2 is 7.
I think what you're looking for is np.transpose()
You can use np.swapaxes, however this swaps "dimensions", so for a matrix that's either 0 or 1 because you have two dimensions:
>>> np.swapaxes(arr, 0, 1) # assuming your matrix is called arr
array([[0, 0, 0, 0, 0, 0, 1],
[1, 3, 2, 2, 4, 3, 2],
[2, 1, 3, 4, 1, 2, 2],
[4, 3, 2, 1, 2, 2, 2]])
To get your desired output you'd need to remove the first two columns before the swapaxes:
>>> np.swapaxes(arr[2:], 0, 1)
array([[0, 0, 0, 0, 1],
[2, 2, 4, 3, 2],
[3, 4, 1, 2, 2],
[2, 1, 2, 2, 2]])
However generally you should use np.transpose or .T if you want to transpose the matrix/array:
>>> arr[2:].T
array([[0, 0, 0, 0, 1],
[2, 2, 4, 3, 2],
[3, 4, 1, 2, 2],
[2, 1, 2, 2, 2]])
I have a very big array with the shape = (32, 3, 1e6)
I need to reshape it to this shape = (3, 32e6)
On a snippet, how to go from this::
>>> m3_3_5
array([[[8, 4, 1, 0, 0],
[6, 8, 5, 5, 2],
[1, 1, 1, 1, 1]],
[[8, 7, 1, 0, 3],
[2, 8, 5, 5, 2],
[1, 1, 1, 1, 1]],
[[2, 4, 0, 2, 3],
[2, 5, 5, 3, 2],
[1, 1, 1, 1, 1]]])
to this::
>>> res3_15
array([[8, 4, 1, 0, 0, 8, 7, 1, 0, 3, 2, 4, 0, 2, 3],
[6, 8, 5, 5, 2, 2, 8, 5, 5, 2, 2, 5, 5, 3, 2],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])
I did try various combinations with reshape with no success::
>>> dd.T.reshape(3, 15)
array([[8, 8, 2, 6, 2, 2, 1, 1, 1, 4, 7, 4, 8, 8, 5],
[1, 1, 1, 1, 1, 0, 5, 5, 5, 1, 1, 1, 0, 0, 2],
[5, 5, 3, 1, 1, 1, 0, 3, 3, 2, 2, 2, 1, 1, 1]])
>>> dd.reshape(15, 3).T.reshape(3, 15)
array([[8, 0, 8, 2, 1, 8, 0, 8, 2, 1, 2, 2, 5, 2, 1],
[4, 0, 5, 1, 1, 7, 3, 5, 1, 1, 4, 3, 5, 1, 1],
[1, 6, 5, 1, 1, 1, 2, 5, 1, 1, 0, 2, 3, 1, 1]])
a.transpose([1,0,2]).reshape(3,15) will do what you want. (I am basically following comments by #hpaulj).
In [14]: a = np.array([[[8, 4, 1, 0, 0],
[6, 8, 5, 5, 2],
[1, 1, 1, 1, 1]],
[[8, 7, 1, 0, 3],
[2, 8, 5, 5, 2],
[1, 1, 1, 1, 1]],
[[2, 4, 0, 2, 3],
[2, 5, 5, 3, 2],
[1, 1, 1, 1, 1]]])
In [15]: a.transpose([1,0,2]).reshape(3,15)
Out[15]:
array([[8, 4, 1, 0, 0, 8, 7, 1, 0, 3, 2, 4, 0, 2, 3],
[6, 8, 5, 5, 2, 2, 8, 5, 5, 2, 2, 5, 5, 3, 2],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])
You can get the desired behavior with np.hstack
# g is your (3,3,5) array from above
reshaped = np.hstack(g[i,:,:] for i in range(3)) #uses a generator exp
reshaped_simpler = np.hstack(g) # this produces equivalent output to the above statmement
print reshaped # (3,30)
Output
array([[8, 4, 1, 0, 0, 8, 7, 1, 0, 3, 2, 4, 0, 2, 3],
[6, 8, 5, 5, 2, 2, 8, 5, 5, 2, 2, 5, 5, 3, 2],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])
How would I go about going converting a list of lists of ints into a matrix plot in Python?
The example data set is:
[[3, 5, 3, 5, 2, 3, 2, 4, 3, 0, 5, 0, 3, 2],
[5, 2, 2, 0, 0, 3, 2, 1, 0, 5, 3, 5, 0, 0],
[2, 5, 3, 1, 1, 3, 3, 0, 0, 5, 4, 4, 3, 3],
[4, 1, 4, 2, 1, 4, 5, 1, 2, 2, 0, 1, 2, 3],
[5, 1, 1, 1, 5, 2, 5, 0, 4, 0, 2, 4, 4, 5],
[5, 1, 0, 4, 5, 5, 4, 1, 3, 3, 1, 1, 0, 1],
[3, 2, 2, 4, 3, 1, 5, 5, 0, 4, 3, 2, 4, 1],
[4, 0, 1, 3, 2, 1, 2, 1, 0, 1, 5, 4, 2, 0],
[2, 0, 4, 0, 4, 5, 1, 2, 1, 0, 3, 4, 3, 1],
[2, 3, 4, 5, 4, 5, 0, 3, 3, 0, 2, 4, 4, 5],
[5, 2, 4, 3, 3, 0, 5, 4, 0, 3, 4, 3, 2, 1],
[3, 0, 4, 4, 4, 1, 4, 1, 3, 5, 1, 2, 1, 1],
[3, 4, 2, 5, 2, 5, 1, 3, 5, 1, 4, 3, 4, 1],
[0, 1, 1, 2, 3, 1, 2, 0, 1, 2, 4, 4, 2, 1]]
To give you an idea of what I'm looking for, the function MatrixPlot in Mathematica gives me this image for this data set:
Thanks!
You may try
from pylab import *
A = rand(5,5)
figure(1)
imshow(A, interpolation='nearest')
grid(True)
source
Perhaps matshow() from matplotlib is what you need.
You can also use pyplot from matplotlib, follows the code:
from matplotlib import pyplot as plt
plt.imshow(
[[3, 5, 3, 5, 2, 3, 2, 4, 3, 0, 5, 0, 3, 2],
[5, 2, 2, 0, 0, 3, 2, 1, 0, 5, 3, 5, 0, 0],
[2, 5, 3, 1, 1, 3, 3, 0, 0, 5, 4, 4, 3, 3],
[4, 1, 4, 2, 1, 4, 5, 1, 2, 2, 0, 1, 2, 3],
[5, 1, 1, 1, 5, 2, 5, 0, 4, 0, 2, 4, 4, 5],
[5, 1, 0, 4, 5, 5, 4, 1, 3, 3, 1, 1, 0, 1],
[3, 2, 2, 4, 3, 1, 5, 5, 0, 4, 3, 2, 4, 1],
[4, 0, 1, 3, 2, 1, 2, 1, 0, 1, 5, 4, 2, 0],
[2, 0, 4, 0, 4, 5, 1, 2, 1, 0, 3, 4, 3, 1],
[2, 3, 4, 5, 4, 5, 0, 3, 3, 0, 2, 4, 4, 5],
[5, 2, 4, 3, 3, 0, 5, 4, 0, 3, 4, 3, 2, 1],
[3, 0, 4, 4, 4, 1, 4, 1, 3, 5, 1, 2, 1, 1],
[3, 4, 2, 5, 2, 5, 1, 3, 5, 1, 4, 3, 4, 1],
[0, 1, 1, 2, 3, 1, 2, 0, 1, 2, 4, 4, 2, 1]], interpolation='nearest')
plt.show()
The output would be: