Related
I have a list of lists but because of the Int64Index I cannot access it. Is there a way to access individual values or make it into a normal list?
data_exp = pd.read_csv(path+'/exp.csv')
exp_list=[]
for i in range (1,n+1):
check=data_exp.apply(lambda x: True if x['Set No.']==i else False, axis=1)
temp=[data_exp[check==True].index+1]
exp_list.append(temp)
del temp
display(exp_list)
The for loop just sort values based on a condition. The output is good but it is the format which is problamatic.
Gives me out put as follows:-
[[Int64Index([8, 11, 17, 20, 21, 27, 29, 36, 37, 38], dtype='int64')],
[Int64Index([1, 3, 7, 10, 14, 31, 33, 34, 35], dtype='int64')],
[Int64Index([5, 9, 12, 15, 19, 23, 25, 26, 28, 32], dtype='int64')],
[Int64Index([2, 4, 6, 13, 16, 18, 22, 24, 30, 39, 40], dtype='int64')]]
Thanks in advance
I'm not quite sure what you're doing to get the list of Int64Indexes, but you can access the numpy array underlying the index with the values property:
from pandas import Int64Index
l = [[Int64Index([8, 11, 17, 20, 21, 27, 29, 36, 37, 38], dtype='int64')],
[Int64Index([1, 3, 7, 10, 14, 31, 33, 34, 35], dtype='int64')],
[Int64Index([5, 9, 12, 15, 19, 23, 25, 26, 28, 32], dtype='int64')],
[Int64Index([2, 4, 6, 13, 16, 18, 22, 24, 30, 39, 40], dtype='int64')]]
print(l[0][0].values[0])
Let's assume that I have input of 4x4 image with 3 channels with following pixel values:
And I want to make it to 12 x 9 matrix of image patches like this (using 2x2 kernel on a 4x4 image):
How can I achieve this using numpy?
Thank you for your help.
Assuming 4x4x3 as input and 12x9 as output
from scipy.signal import convolve
import numpy as np
# creating the 4x4x3 input image
a = np.arange( 1,16+1).reshape(4,4)
b = np.arange(17,32+1).reshape(4,4)
c = np.arange(33,48+1).reshape(4,4)
i_4x4x3 = np.dstack((a, b, c))
# creating four 2x2 kernels
mask_tl = np.array([0,0,0,1]).reshape(2,2)
mask_tr = np.array([0,0,1,0]).reshape(2,2)
mask_bl = np.array([0,1,0,0]).reshape(2,2)
mask_br = np.array([1,0,0,0]).reshape(2,2)
mask_tl = mask_tl[:,:,None]
mask_tr = mask_tr[:,:,None]
mask_bl = mask_bl[:,:,None]
mask_br = mask_br[:,:,None]
# convolving the input with all four kernels
tl = convolve(i_4x4x3, mask_tl, mode='valid')
tr = convolve(i_4x4x3, mask_tr, mode='valid')
bl = convolve(i_4x4x3, mask_bl, mode='valid')
br = convolve(i_4x4x3, mask_br, mode='valid')
i = np.dstack((
tl.reshape(-1,3),
tr.reshape(-1,3),
bl.reshape(-1,3),
br.reshape(-1,3)))
i=i.reshape(i.shape[0],-1).transpose()
display(a,b,c)
display(i)
Output:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
array([[17, 18, 19, 20],
[21, 22, 23, 24],
[25, 26, 27, 28],
[29, 30, 31, 32]])
array([[33, 34, 35, 36],
[37, 38, 39, 40],
[41, 42, 43, 44],
[45, 46, 47, 48]])
array([[ 1, 2, 3, 5, 6, 7, 9, 10, 11],
[ 2, 3, 4, 6, 7, 8, 10, 11, 12],
[ 5, 6, 7, 9, 10, 11, 13, 14, 15],
[ 6, 7, 8, 10, 11, 12, 14, 15, 16],
[17, 18, 19, 21, 22, 23, 25, 26, 27],
[18, 19, 20, 22, 23, 24, 26, 27, 28],
[21, 22, 23, 25, 26, 27, 29, 30, 31],
[22, 23, 24, 26, 27, 28, 30, 31, 32],
[33, 34, 35, 37, 38, 39, 41, 42, 43],
[34, 35, 36, 38, 39, 40, 42, 43, 44],
[37, 38, 39, 41, 42, 43, 45, 46, 47],
[38, 39, 40, 42, 43, 44, 46, 47, 48]])
How would one (efficiently) do the following:
x = np.arange(49)
x2 = np.reshape(x, (7,7))
x2
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34],
[35, 36, 37, 38, 39, 40, 41],
[42, 43, 44, 45, 46, 47, 48]])
From here I want to roll a couple of things.
I want to roll 0,7,14,21 etc so 14 comes to top.
Then the same with 4,11,18,25 etc so 39 comes to top.
Result should be:
x2
array([[14, 1, 2, 3, 39, 5, 6],
[21, 8, 9, 10, 46, 12, 13],
[28, 15, 16, 17, 4, 19, 20],
[35, 22, 23, 24, 11, 26, 27],
[42, 29, 30, 31, 18, 33, 34],
[ 0, 36, 37, 38, 25, 40, 41],
[ 7, 43, 44, 45, 32, 47, 48]])
I looked up numpy.roll, here and google but couldn't find how one would do this.
For horizontal rolls, I could do:
np.roll(x2[0], 3, axis=0)
x3
array([4, 5, 6, 0, 1, 2, 3])
But how do I return the full array with this roll change as a new copy?
Roll with a negative shift:
x2[:, 0] = np.roll(x2[:, 0], -2)
Roll with a positive shift:
x2[:, 4] = np.roll(x2[:, 4], 2)
gives:
>>>x2
array([[14, 1, 2, 3, 39, 5, 6],
[21, 8, 9, 10, 46, 12, 13],
[28, 15, 16, 17, 4, 19, 20],
[35, 22, 23, 24, 11, 26, 27],
[42, 29, 30, 31, 18, 33, 34],
[ 0, 36, 37, 38, 25, 40, 41],
[ 7, 43, 44, 45, 32, 47, 48]])
Here's a way to roll multiple columns in one go with advanced-indexing -
# Params
cols = [0,4] # Columns to be rolled
dirn = [2,-2] # Offset with direction as sign
n = x2.shape[0]
x2[:,cols] = x2[np.mod(np.arange(n)[:,None] + dirn,n),cols]
Sample run -
In [45]: x2
Out[45]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34],
[35, 36, 37, 38, 39, 40, 41],
[42, 43, 44, 45, 46, 47, 48]])
In [46]: cols = [0,4,5] # Columns to be rolled
...: dirn = [2,-2,4] # Offset with direction as sign
...: n = x2.shape[0]
...: x2[:,cols] = x2[np.mod(np.arange(n)[:,None] + dirn,n),cols]
...:
In [47]: x2 # Three columns rolled
Out[47]:
array([[14, 1, 2, 3, 39, 33, 6],
[21, 8, 9, 10, 46, 40, 13],
[28, 15, 16, 17, 4, 47, 20],
[35, 22, 23, 24, 11, 5, 27],
[42, 29, 30, 31, 18, 12, 34],
[ 0, 36, 37, 38, 25, 19, 41],
[ 7, 43, 44, 45, 32, 26, 48]])
You have to overwrite the column
e.g.:
x2[:,0] = np.roll(x2[:,0], 3)
See here a useful method for shifting a 2D array in all 4 directions (up, down, left, right):
def image_shift_roll(img, x_shift, y_roll):
img_roll = img.copy()
img_roll = np.roll(img_roll, -y_roll, axis = 0) # Positive y rolls up
img_roll = np.roll(img_roll, x_roll, axis = 1) # Positive x rolls right
return img_roll
I have a big two-dimensional array like this:
array([[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9,10,11,12,13,14,15,16],
[17,18,19,20,21,22,23,24],
[25,26,27,28,29,30,31,32],
[33,34,35,36,37,38,39,40],
[41,42,43,44,45,46,47,48],
....])
and I need to convert it into:
array([ 1, 9,17, 2,10,18, 3,11,19, 4,12,20, 5,13,21, 6,14,22, 7,15,23, 8,16,24],
[25,33,41,26,34,42,27,35,43,28,36,44,29,37,45,30,38,46,31,39,47,32,40,48],
...
Note that this should only be a demonstration what it should do.
The original array contains only boolean values and has the size of 512x8. In my example, I order only 3 rows with 8 elements into one row but what I really need are respectively 32 rows with 8 elements.
I am really sorry, but after 30 minutes of writing, this is the only description I got of my problem. I hope it is enough.
I think you can achieve your desired result using two reshape operations and a transpose:
x = np.array([[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9,10,11,12,13,14,15,16],
[17,18,19,20,21,22,23,24],
[25,26,27,28,29,30,31,32],
[33,34,35,36,37,38,39,40],
[41,42,43,44,45,46,47,48]])
y = x.reshape(2, 3, 8).transpose(0, 2, 1).reshape(2, -1)
print(repr(y))
# array([[ 1, 9, 17, 2, 10, 18, 3, 11, 19, 4, 12, 20, 5, 13, 21, 6, 14,
# 22, 7, 15, 23, 8, 16, 24],
# [25, 33, 41, 26, 34, 42, 27, 35, 43, 28, 36, 44, 29, 37, 45, 30, 38,
# 46, 31, 39, 47, 32, 40, 48]])
To break that down a bit:
#hpaulj's first reshape operation gives us this:
x1 = x.reshape(2, 3, 8)
print(repr(x1))
# array([[[ 1, 2, 3, 4, 5, 6, 7, 8],
# [ 9, 10, 11, 12, 13, 14, 15, 16],
# [17, 18, 19, 20, 21, 22, 23, 24]],
# [[25, 26, 27, 28, 29, 30, 31, 32],
# [33, 34, 35, 36, 37, 38, 39, 40],
# [41, 42, 43, 44, 45, 46, 47, 48]]])
print(x1.shape)
# (2, 3, 8)
In order to get the desired output we need to 'collapse' this array along the second dimension (with size 3), then along the third dimension (with size 8).
The easiest way to achieve this sort of thing is to first transpose the
array so that the dimensions you want to collapse along are ordered from first to last:
x2 = x1.transpose(0, 2, 1) # you could also use `x2 = np.rollaxis(x1, 1, 3)`
print(repr(x2))
# array([[[ 1, 9, 17],
# [ 2, 10, 18],
# [ 3, 11, 19],
# [ 4, 12, 20],
# [ 5, 13, 21],
# [ 6, 14, 22],
# [ 7, 15, 23],
# [ 8, 16, 24]],
# [[25, 33, 41],
# [26, 34, 42],
# [27, 35, 43],
# [28, 36, 44],
# [29, 37, 45],
# [30, 38, 46],
# [31, 39, 47],
# [32, 40, 48]]])
print(x2.shape)
# (2, 8, 3)
Finally I can use reshape(2, -1) to collapse the array over the last two dimensions. The -1 causes numpy to infer the appropriate size in the last dimension based on the number of elements in x.
y = x2.reshape(2, -2)
Looks like a starting point is to reshape it, for example
In [49]: x.reshape(2,3,8)
Out[49]:
array([[[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9, 10, 11, 12, 13, 14, 15, 16],
[17, 18, 19, 20, 21, 22, 23, 24]],
[[25, 26, 27, 28, 29, 30, 31, 32],
[33, 34, 35, 36, 37, 38, 39, 40],
[41, 42, 43, 44, 45, 46, 47, 48]]])
.ravel(order='F') doesn't get it right, so I think we need to swap some axes before flattening. It will need to be a copy.
Using #ali_m's transpose:
In [65]: x1=x.reshape(2,3,8)
In [66]: x1.transpose(0,2,1).flatten()
Out[66]:
array([ 1, 9, 17, 2, 10, 18, 3, 11, 19, 4, 12, 20, 5, 13, 21, 6, 14,
22, 7, 15, 23, 8, 16, 24, 25, 33, 41, 26, 34, 42, 27, 35, 43, 28,
36, 44, 29, 37, 45, 30, 38, 46, 31, 39, 47, 32, 40, 48])
oops - there's an inner layer of nesting that's easy to miss
array([1,9,17,2,10,18,3,11,19,4,12,20,5,13,21,6,14,22,7,15,23,8,16,24],
[25,33,41,26,34,42,27,35,43,28,36,44,29,37,45,30,38,46,31,39,47,32,40,4],
You are missing a [] set. So #ali_m got it right.
I'm tempted to delete this, but my trial and error might be instructive.
travel_mat1= numpy.array([[23,23,20,24,28,12,17,10],[11,27,17,19,24,18,23,7],
[17,26,22,13,18,29,30,18],[22,21,28,7,18,29,30,18],[27,16,33,36,10,23,26,25],
[31,13,36,14,26,23,20,27],[34,7,33,20,35,17,14,24],[28,13,27,26,37,11,10,18],
[25,17,33,28,34,10,12,15]])
I need to change the size of array dynamically with no loss of actual data in the array. Means, I need to have a virtual dynamic array.
The above array Travel_mat1 is a 9X8 matrix. So if i need a 8X7 size matrix from Travel_mat1, it should look like:
([[23,23,20,24,28,12,17],[11,27,17,19,24,18,23],[17,26,22,13,18,29,30],
[22,21,28,7,18,29], [27,16,33,36,10,23,26],[31,13,36,14,26,23,20],
[34,7,33,20,35,17,14],[28,13,27,26,37,11,10]]).
Means, I need to reduce a row and a column in this case. How can I do this in python?
You can use numpy.delete:
>>> numpy.delete(numpy.delete(travel_mat1, 8, 0), 7, 1)
array([[23, 23, 20, 24, 28, 12, 17],
[11, 27, 17, 19, 24, 18, 23],
[17, 26, 22, 13, 18, 29, 30],
[22, 21, 28, 7, 18, 29, 30],
[27, 16, 33, 36, 10, 23, 26],
[31, 13, 36, 14, 26, 23, 20],
[34, 7, 33, 20, 35, 17, 14],
[28, 13, 27, 26, 37, 11, 10]])