Any one know why I cannot use "|" to concatenate multiple numpy.flatiter object after converting it inot set? I try to look for all display number
from all row 11, all column 1 and section from (2,2) to (3,3) if I use np.concatenate I can get the right answer but after I use "|" I have empty set? or if there is a better way to write it?
import numpy as np
matrix = np.matrix(np.arange(36).reshape(6, 6))
rnum = matrix[1, :].flat
cnum = matrix[:, 1].flat
snum = matrix[2:4, 2:4].flat
print(matrix)
print(rnum)
print(set(rnum))
print(set(cnum))
print(set(snum))
print(set(np.concatenate((rnum, cnum, snum))))
print(set(rnum) | set(cnum) | set(snum))
#[[ 0 1 2 3 4 5]
# [ 6 7 8 9 10 11]
# [12 13 14 15 16 17]
# [18 19 20 21 22 23]
# [24 25 26 27 28 29]
# [30 31 32 33 34 35]]
#<numpy.flatiter object at 0x7faf52966c00>
#{6, 7, 8, 9, 10, 11}
#{1, 7, 13, 19, 25, 31}
#{20, 21, 14, 15}
#{1, 6, 7, 8, 9, 10, 11, 13, 14, 15, 19, 20, 21, 25, 31} => expect result
#set() => why?
The first call of set(rnum) in print(set(rnum)) consumes the iterator rnum. When you use set(rnum) again in set(rnum) | set(cnum) | set(snum), there are no more values left in the iterator rnum, so set(rnum) is the empty set.
Here's a more direct demonstration:
In [621]: matrix = np.matrix(np.arange(36).reshape(6, 6))
In [622]: rnum = matrix[1, :].flat
In [623]: set(rnum)
Out[623]: {6, 7, 8, 9, 10, 11}
In [624]: set(rnum)
Out[624]: set()
Instead of using rnum, you could create another iterator by repeating matrix[1, :].flat:
In [625]: set(matrix[1, :].flat)
Out[625]: {6, 7, 8, 9, 10, 11}
Alternatively, skip the use of numpy.matrix and iterators, and just index into a regular NumPy array:
In [639]: a = np.arange(36).reshape(6, 6)
In [640]: set(a[1,:])
Out[640]: {6, 7, 8, 9, 10, 11}
In [641]: set(a[:,1])
Out[641]: {1, 7, 13, 19, 25, 31}
In [642]: set(a[2:4, 2:4].ravel())
Out[642]: {20, 21, 14, 15}
Related
From a range of numbers [0:2407] I need to know what are the ones that are already being used.
arrray [0,1,2,..,2407]
To know the ones already used I have a file that I load with pandas.
example:
...| Index |...
...| 100 |...
...| 1572 |...
...| 2046 |...
...| 2045 |...
I need to remove from my initial list the ones coming from the file.
trying to do this in a clean and faster way since the files can be quite big.
Try this:
import pandas as pd
import random
## for demo purpose max number is changed from 2407 to 27
max = 27
## list containing range of numbers
unsed= list(range(max+1))
print(f'all_n : {unsed}')
## define dataFrame exaple
df = pd.DataFrame(random.sample(range(max+1), 10), columns=['index'])
# index
# 0 6
# 1 14
# 2 20
# 3 4
# 4 25
## convert used number to list
used = df['index'].tolist()
print(f'used : {sorted(used)}')
## unused
for n in used:
unused.remove(n)
print(f'unused : {unused}')
Result:
all_n : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27]
used : [4, 6, 14, 20, 25]
unused : [0, 1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 21, 22, 23, 24, 26, 27]
Create a list of flags of size 2408, initially setting all flags to false:
is_used = [False for i in range(2408)]
Iterate through your column and change the corresponding flag to True:
for entry in column:
is_used[entry] = True
Iterate through your list and append to a new list the elements that are not used:
new_list = []
for entry in l:
if not is_used[entry]:
new_l.append(entry)
Summarizing all in a single method:
def remove_used(l, column):
is_used = [False for i in range(2408)]
for entry in column:
is_used[entry] = True
new_list = []
for entry in l:
if not is_used[entry]:
new_l.append(entry)
return new_list
Also, it is worth mentioning that you can speed up by dividing the loops into blocks and putting threads/processes to act on each block.
seq = [11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1]
for i in range(len(seq)):
print(seq[i],end ="\t")
How do I get my output table to look like this?
11 34 17 52 26 13
40 20 10 5 16 8
4 2 1
one of many ways is this, you make iterate over the seq list by a step of 6 and print the element between those margins
seq = [11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1]
for i in range(0, len(seq), 6):
print(*seq[i:i+6], sep=' ')
output
11 34 17 52 26 13
40 20 10 5 16 8
4 2 1
You probably want to make use of string formatting. Below, f"{seq[i]:<4d}" means "A string of length 4, left-aligned, containing the string representation of seq[i]". If you want to right-align, just remove <.
seq = [11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1]
for i in range(len(seq)):
print(f"{seq[i]:<4d}", end = "")
if not (i+1) % 6:
print("")
print("")
Output:
11 34 17 52 26 13
40 20 10 5 16 8
4 2 1
The simplest relevant technique is padding
for i in range(0, len(seq), 6):
print(" ".join[str(k).ljust(2, " ") for k in seq[i: i + 6]]
but string formatting as in Printing Lists as Tabular Data will make is a more sophisticated solution
I have a pandas dataframe like:
I need to style it using a list of lists like:
[[3, 7, 4, 5],
[6, 17, 5, 10, 13, 16],
[7, 22, 6, 17, 19, 12],
[12, 26, 24, 25, 23, 18, 20],
[21, 20, 18, 27, 25]]
If R1 values are in first list color blue, if R2 values are in second list color blue and so on.
In other words color numbers of each column if value is in the correspondent list.
I have tried:
def posclass(val):
color = 'black'
for i in range(5):
if (val in list[i]):
color = 'blue'
return 'color: %s' % color
df.style.applymap(posclass, subset=['R1','R2','R3','R4','R5'])
But this is not working properly applying each list to each column.
The desired result is a dataframe with colored numbers (those that matches in each column with each list).
Try something like this:
df = pd.DataFrame(np.arange(40).reshape(-1,4), columns=[f'R{i}' for i in range(1,5)])
Input df:
R1 R2 R3 R4
0 0 1 2 3
1 4 5 6 7
2 8 9 10 11
3 12 13 14 15
4 16 17 18 19
5 20 21 22 23
6 24 25 26 27
7 28 29 30 31
8 32 33 34 35
9 36 37 38 39
and
list_l = [[3, 7, 4, 5],
[6, 17, 5, 10, 13, 16],
[7, 22, 6, 17, 19, 12],
[12, 26, 24, 25, 23, 18, 20],
[21, 20, 18, 27, 25]]
Then:
def f(x):
colpos = df.columns.get_loc(x.name)
return ['color: blue' if n in list_l[colpos] else '' for n in x]
df.style.apply(f)
Output:
What is the Pythonic way to get a list of diagonal elements in a matrix passing through entry (i,j)?
For e.g., given a matrix like:
[1 2 3 4 5]
[6 7 8 9 10]
[11 12 13 14 15]
[16 17 18 19 20]
[21 22 23 24 25]
and an entry, say, (1,3) (representing element 9) how can I get the elements in the diagonals passing through 9 in a Pythonic way? Basically, [3,9,15] and [5,9,13,17,21] both.
Using np.diagonal with a little offset logic.
import numpy as np
lst = np.array([[1, 2, 3, 4, 5],
[6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25]])
i, j = 1, 3
major = np.diagonal(lst, offset=(j - i))
print(major)
array([ 3, 9, 15])
minor = np.diagonal(np.rot90(lst), offset=-lst.shape[1] + (j + i) + 1)
print(minor)
array([ 5, 9, 13, 17, 21])
The indices i and j are the row and column. By specifying the offset, numpy knows from where to begin selecting elements for the diagonal.
For the major diagonal, You want to start collecting from 3 in the first row. So you need to take the current column index and subtract it by the current row index, to figure out the correct column index at the 0th row. Similarly for the minor diagonal, where the array is flipped (rotated by 90˚) and the process repeats.
As another alternative method, with raveling the array and for matrix with shape (n*n):
array = np.array([[1, 2, 3, 4, 5],
[6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25]])
x, y = 1, 3
a_mod = array.ravel()
size = array.shape[0]
if y >= x:
diag = a_mod[y-x:(x+size-y)*size:size+1]
else:
diag = a_mod[(x-y)*size::size+1]
if x-(size-1-y) >= 0:
reverse_diag = array[:, ::-1].ravel()[(x-(size-1-y))*size::size+1]
else:
reverse_diag = a_mod[x:x*size+1:size-1]
# diag --> [ 3 9 15]
# reverse_diag --> [ 5 9 13 17 21]
The correctness of the resulted arrays must be checked further. This can be developed to handle matrices with other shapes e.g. (n*m).
So I found this:
When converting MATLAB code it might be necessary to first reshape a
matrix to a linear sequence, perform some indexing operations and then
reshape back. As reshape (usually) produces views onto the same
storage, it should be possible to do this fairly efficiently.
Note that the scan order used by reshape in Numpy defaults to the 'C'
order, whereas MATLAB uses the Fortran order. If you are simply
converting to a linear sequence and back this doesn't matter. But if
you are converting reshapes from MATLAB code which relies on the scan
order, then this MATLAB code:
z = reshape(x,3,4);
should become
z = x.reshape(3,4,order='F').copy()
in Numpy.
I have a multidimensional 16*2 array called mafs, when I do in MATLAB:
mafs2 = reshape(mafs,[4,4,2])
I get something different than when in python I do:
mafs2 = reshape(mafs,(4,4,2))
or even
mafs2 = mafs.reshape((4,4,2),order='F').copy()
Any help on this? Thank you all.
Example:
MATLAB:
>> mafs = [(1:16)' (17:32)']
mafs =
1 17
2 18
3 19
4 20
5 21
6 22
7 23
8 24
9 25
10 26
11 27
12 28
13 29
14 30
15 31
16 32
>> reshape(mafs,[4 4 2])
ans(:,:,1) =
1 5 9 13
2 6 10 14
3 7 11 15
4 8 12 16
ans(:,:,2) =
17 21 25 29
18 22 26 30
19 23 27 31
20 24 28 32
Python:
>>> import numpy as np
>>> mafs = np.c_[np.arange(1,17), np.arange(17,33)]
>>> mafs.shape
(16, 2)
>>> mafs[:,0]
array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16])
>>> mafs[:,1]
array([17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32])
>>> r = np.reshape(mafs, (4,4,2), order="F")
>>> r.shape
(4, 4, 2)
>>> r[:,:,0]
array([[ 1, 5, 9, 13],
[ 2, 6, 10, 14],
[ 3, 7, 11, 15],
[ 4, 8, 12, 16]])
>>> r[:,:,1]
array([[17, 21, 25, 29],
[18, 22, 26, 30],
[19, 23, 27, 31],
[20, 24, 28, 32]])
I was having a similar issue myself, as I am also trying to make the transition from MATLAB to Python. I was finally able to convert a numpy matrix, given in depth, row, col, format to a single sheet of column vectors (per image).
In MATLAB I would have done something like:
output = reshape(imStack,[row*col,depth])
In Python this seems to translate to:
import numpy as np
output=np.transpose(imStack)
output=output.reshape((row*col, depth), order='F')