I have a 3d array that has shape (2, 10, 3) and a 2d array that has shape (2, 3) like this:
print(t) #2d array
Output:
[[1.003 2.32 3.11 ]
[1.214 5.32 2.13241]]
print(normal) #3d array
Output:
[[[0.69908573 0.0826756 0.84485978]
[0.51058213 0.4052637 0.5068118 ]
[0.45974276 0.25819549 0.10780089]
[0.27484999 0.33367648 0.128262 ]
[0.35963389 0.77600065 0.89393939]
[0.46937506 0.59291623 0.06620307]
[0.87603987 0.44414505 0.83394174]
[0.83186093 0.62491876 0.38160734]
[0.96819897 0.80183442 0.75102768]
[0.54182908 0.19403844 0.07925769]]
[[2.82248573 3.2341756 0.96825978]
[2.63398213 3.5567637 0.6302118 ]
[2.58314276 3.40969549 0.23120089]
[2.39824999 3.48517648 0.251662 ]
[2.48303389 3.92750065 1.01733939]
[2.59277506 3.74441623 0.18960307]
[2.99943987 3.59564505 0.95734174]
[2.95526093 3.77641876 0.50500734]
[3.09159897 3.95333442 0.87442768]
[2.66522908 3.34553844 0.20265769]]]
How can I get each row in the 2d array t to get the corresponding dot product in the 3d array normal such that the array I end up with a shape (2, 10) where each contains all 10 dot products between the nth row in 2d array and nth matrix in 3d array?
[0.62096458 0.62618459 0.37528887 0.5728386 1.19634398 0.79620507
1.997884 0.75229492 1.2236496 0.4210626 ]
[2.96347746 3.30738892 3.50596579 4.93082295 5.33811805 4.44872493
7.33480393 4.19173472 4.7406248 7.83229689]
You can use numpy.einsum (np.einsum('ijk,ik->ij', t, normal)) to get this result:
import numpy as np
normal = np.array([
[1.003,2.32,3.11],
[1.214,5.32,2.13241]
])
t = np.array([
[
[0.69908573, 0.0826756, 0.84485978],
[0.51058213, 0.4052637, 0.5068118 ],
[0.45974276, 0.25819549, 0.10780089],
[0.27484999, 0.33367648, 0.128262 ],
[0.35963389, 0.77600065, 0.89393939],
[0.46937506, 0.59291623, 0.06620307],
[0.87603987, 0.44414505, 0.83394174],
[0.83186093, 0.62491876, 0.38160734],
[0.96819897, 0.80183442, 0.75102768],
[0.54182908, 0.19403844, 0.07925769]
],
[
[2.82248573, 3.2341756, 0.96825978],
[2.63398213, 3.5567637, 0.6302118 ],
[2.58314276, 3.40969549, 0.23120089],
[2.39824999, 3.48517648, 0.251662 ],
[2.48303389, 3.92750065, 1.01733939],
[2.59277506, 3.74441623, 0.18960307],
[2.99943987, 3.59564505, 0.95734174],
[2.95526093, 3.77641876, 0.50500734],
[3.09159897, 3.95333442, 0.87442768],
[2.66522908, 3.34553844, 0.20265769]
]
])
np.einsum('ijk,ik->ij', t, normal)
This results in
array([[ 3.52050429, 3.02851036, 1.39539629, 1.44869879, 4.9411858 ,
2.05224039, 4.50264332, 3.47096686, 5.16705551, 1.24011516],
[22.69703871, 23.46350713, 21.76853041, 21.98926093, 26.07809129,
23.47223475, 24.81159677, 24.75511727, 26.64957859, 21.46600189]])
Which is the same as doing the two multiplications in order:
t[0] # normal[0]
t[1] # normal[1]
Gives the two:
array([3.52050429, 3.02851036, 1.39539629, 1.44869879, 4.9411858 ,
2.05224039, 4.50264332, 3.47096686, 5.16705551, 1.24011516])
array([22.69703871, 23.46350713, 21.76853041, 21.98926093, 26.07809129,
23.47223475, 24.81159677, 24.75511727, 26.64957859, 21.46600189])
Related
I have an array of values A, an X array and a Y array:
A=np.random.rand(100)
X=np.linspace(0,9.9,100).astype('int')
Y=np.mod(np.linspace(0,99,100).astype('int'),10)
from sklearn.utils import shuffle
X, Y = shuffle(X, Y, random_state=0)
Is there a fast way without using a for loop, to build the array B so that B[X[i],Y[i]]=A[i]?
If you don't mind using additional libraries, you could use Scipy to create a sparse matrix and convert it to numpy array.
import scipy
B = scipy.sparse.coo_matrix((A, (X,Y))).toarray()
Gives you:
array([[0.72165902, 0.52990391, 0.09793563, 0.50304214, 0.1220594 ,
0.21866079, 0.21402 , 0.10083007, 0.02413508, 0.47135903],
[0.66857096, 0.53983909, 0.77737883, 0.51930043, 0.82780692,
0.49366086, 0.00985082, 0.69271283, 0.05050557, 0.13781565],
[0.55825277, 0.23288426, 0.32808517, 0.04021886, 0.62240996,
0.35775835, 0.18166671, 0.72845399, 0.82937063, 0.44513946],
[0.43459875, 0.91125147, 0.0997432 , 0.27235194, 0.51298843,
0.51844005, 0.24192413, 0.87119785, 0.35759919, 0.97350304],
[0.76596234, 0.0454183 , 0.7452377 , 0.99293005, 0.46079918,
0.77791926, 0.01981786, 0.79152577, 0.29018054, 0.28989354],
[0.21862015, 0.55044776, 0.41118987, 0.28499734, 0.83900364,
0.25426192, 0.98742121, 0.43742215, 0.08900043, 0.19032915],
[0.34900342, 0.34423692, 0.25362518, 0.14360852, 0.22580151,
0.29163466, 0.30428173, 0.84232247, 0.04396114, 0.3534083 ],
[0.65250231, 0.7090245 , 0.30458176, 0.51372318, 0.89439072,
0.74326644, 0.28444398, 0.2566198 , 0.70270672, 0.40769932],
[0.66205028, 0.20791073, 0.99532769, 0.69981262, 0.0782375 ,
0.24892482, 0.29280819, 0.3020104 , 0.62367672, 0.98863622],
[0.13194516, 0.8298722 , 0.31361851, 0.98307841, 0.08061206,
0.1239905 , 0.03836026, 0.52931279, 0.5748074 , 0.69816216]])
I have the following 3D Numpy matrix:
import numpy as np
arr = np.random.random([4, 3, 4])
array([[[0.86061437, 0.28274671, 0.08120691, 0.07529454],
[0.93281252, 0.28959613, 0.89955385, 0.23104958],
[0.70399225, 0.78649787, 0.65668005, 0.1078731 ]],
[[0.2604536 , 0.74093858, 0.71550647, 0.07096532],
[0.49281007, 0.04934752, 0.2316176 , 0.8452892 ],
[0.5559128 , 0.89977194, 0.60539768, 0.88640264]],
[[0.21532865, 0.02557637, 0.70641993, 0.86614863],
[0.26946359, 0.00956061, 0.91330073, 0.0074185 ],
[0.79044557, 0.50265835, 0.70721046, 0.69482905]],
[[0.15602922, 0.65337023, 0.44756636, 0.97871331],
[0.60633134, 0.93488194, 0.53871744, 0.48607869],
[0.39678678, 0.02369235, 0.42945214, 0.48460456]]])
If we splice arr with v = arr[:,0,0], we get:
array([0.86061437, 0.26045360, 0.21532865, 0.15602922])
I would like to create a matrix like the one in yellow below that results from subtracting each element from every other element:
My initial thought was to take arr[:,0,0] and the transpose arr[:,0,0].T, then try to do the subtraction this way. But, not sure how to do it.
Thanks in advance for your help!
You were very close:
arr.T[..., None, :] - arr.T[..., None]
I have a 2-d matrix. For the purposes of this example, let's say it is a random matrix
>>> a = np.random.randn(5, 7)
>>> a
array([[-0.37279322, 0.28619523, -0.05309901, 0.26010327, 0.1846693 , 0.33112176, 0.75814911],
[ 1.57001151, -0.86831693, -0.20576395, 1.46450855, -0.01631132, 3.02790403, -0.65313017],
[ 0.2362675 , -1.52190536, 0.04687194, 2.01618876, 0.03780218, -0.53041096, -0.30104844],
[-0.5504834 , 1.04286156, 1.12863785, 0.89583492, 0.28607363, 1.42858007, 0.28582572],
[-0.768464 , 0.31952554, 0.81129581, 0.26239668, -0.23242878, -1.01584339, 0.39573906]])
and two vectors of labels:
label_y = np.array([23, 984, 123, 9321, 121238])
label_x = np.array([121, 31312, 9123131, 1111, 1231441, 1929313, 192312312361])
I'd like to flatten the elements of a and output their label indeces and values. For example:
23,121,-0.37279322
23,31312,0.28619523
23,9123131,-0.05309901
23,1111,0.26010327
23,1231441,0.1846693
23,1929313,0.33112176
23,192312312361,0.75814911
984,121,...
...
is there an easy way of doing it in numpy without for loops?
Use np.meshgrid to create 2D meshes corresponding to X and Y labels and then stack them as columns alongwith the 2D input array a, like so -
X,Y = np.meshgrid(label_x,label_y)
out = np.column_stack((Y.ravel(),X.ravel(),a.ravel()))
Good day to everyone! I'm currently converting a MATLAB project to Python 2.7. I am trying to convert the line
h = [ im(:,2:cols) zeros(rows,1) ] - [ zeros(rows,1) im(:,1:cols-1) ];
When I try to convert it
h = np.concatenate((im[1,range(2,cols)], np.zeros((rows, 1)))) -
np.concatenate((np.zeros((rows, 1)),im[1,range(2,cols - 1)] ))
IDLE returns different errors like
ValueError: all the input arrays must have same number of dimensions
I'm very new to Python and I would appreciate it if you would suggest other methods. Thank you so much! Here's the function I am trying to convert.
function [gradient, or] = canny(im, sigma, scaling, vert, horz)
xscaling = vert; yscaling = horz;
hsize = [6*sigma+1, 6*sigma+1]; % The filter size.
gaussian = fspecial('gaussian',hsize,sigma);
im = filter2(gaussian,im); % Smoothed image.
im = imresize(im, scaling, 'AntiAliasing',false);
[rows, cols] = size(im);
h = [ im(:,2:cols) zeros(rows,1) ] - [ zeros(rows,1) im(:,1:cols-1) ];
And I also would ask the equivalent of ':' operator that is used mainly in indeces and arrays in Python. Is there any equivalent for the : operator?
The Python converted code I started:
def canny(im=None, sigma=None, scaling=None, vert=None, horz=None):
xscaling = vert
yscaling = horz
hsize = (6 * sigma + 1), (6 * sigma + 1) # The filter size.
gaussian = gauss2D(hsize, sigma)
im = filter2(gaussian, im) # Smoothed image.
print("This is im")
print(im)
print("This is hsize")
print(hsize)
print("This is scaling")
print(scaling)
#scaling = 0.4
#scaling = tuple(scaling)
im = cv2.resize(im,None, fx=scaling, fy=scaling )
[rows, cols] = np.shape(im)
Say your data is in a list of lists. Try this:
a = [[2, 9, 4], [7, 5, 3], [6, 1, 8]]
im = np.array(a, dtype=float)
rows = 3
cols = 3
h = (np.hstack([im[:, 1:cols], np.zeros((rows, 1))])
- np.hstack([np.zeros((rows, 1)), im[:, :cols-1]]))
The equivalent of MATLAB's horzcat (that is, [A B]) is np.hstack and the equivalent of vertcat ([A; B]) is np.vstack.
Array indexing in numpy is very close to MATLAB, except that indexes start at 0 in numpy, and the range p:q means "p to q-1".
Also, the storage order of arrays is row-major by default, and you can use column-major order if you want (see this). In MATLAB, arrays are stored in column-major order. To check in Python, type for instance np.isfortran(im). If it returns true, the array has the same order as MATLAB (Fortran order), otherwise it's row-major (C order). It's important when you want to optimize loops, or when you pass an array to a C or Fortran routine.
Ideally, try to put everything in an np.array as soon as possible, and don't use lists (they take much more space and processing is much slower). There are also some quirks: for instance, 1.0 / 0.0 throws an exception, but np.float64(1.0) / np.float64(0.0) returns inf, like in MATLAB.
Another example from the comments:
d1 = [ im(2:rows,2:cols) zeros(rows-1,1); zeros(1,cols) ] - ...
[ zeros(1,cols); zeros(rows-1,1) im(1:rows-1,1:cols-1) ];
d2 = [ zeros(1,cols); im(1:rows-1,2:cols) zeros(rows-1,1); ] - ...
[ zeros(rows-1,1) im(2:rows,1:cols-1); zeros(1,cols) ];
For this one, rather than np.vstack and np.hstack, you can use np.block.
im = np.ones((10, 15))
rows, cols = im.shape
d1 = (np.block([[im[1:rows, 1:cols], np.zeros((rows-1, 1))],
[np.zeros((1, cols))]]) -
np.block([[np.zeros((1, cols))],
[np.zeros((rows-1, 1)), im[:rows-1, :cols-1]]]))
d2 = (np.block([[np.zeros((1, cols))],
[im[:rows-1, 1:cols], np.zeros((rows-1, 1))]]) -
np.block([[np.zeros((rows-1, 1)), im[1:rows, :cols-1]],
[np.zeros((1, cols))]]))
With np.zeros((Nrows,1)) you are generating a 2D array containing Nrows 1D arrays with 1 element. Then, with im[1,2:cols] your are getting a 1D array of cols-2 elements. You should change np.zeros((rows,1)) by np.zeros(rows).
Moreover, at the second np.concatenate, when you get a subarray from 'im' you should take the same number of elements than in the first concatenate. Note that you are taking one element less: range(2,cols) VS range(2,cols-1).
I want a tensorflow function, which accepts a 3D matrix and an array ( shape of the array is similar to the first dimension of a 3D matrix ) and I want to slice the elements from each 2D matrix inside the 3D matrix based on the given array. The equivalent numpy looks like as follows. The basic idea is to picking all hidden states of each input in a batch ( avoid the padded ) in a dynamic rnn
import numpy as np
a = np.random.uniform(-1,1,(3,5,7))
a_length = np.random.randint(5,size=(3))
a_tf = tf.convert_to_tensor(a)
a_length_tf = tf.convert_to_tensor(a_length)
res = []
for index, length_ in enumerate(a_length):
res.extend(a[index,:length_,:])
res = np.array(res)
Output
print(a_length)
array([1, 4, 4])
print(res)
array([[-0.060161 , 0.36000953, 0.46160677, -0.66576281, 0.28562044,
-0.60026872, 0.08034777],
[ 0.04776443, 0.38018207, -0.73352382, 0.61847258, -0.89731857,
0.57264147, -0.88192537],
[ 0.92657628, 0.6236141 , 0.41977008, 0.88720247, 0.44639323,
0.26165976, 0.2678753 ],
[-0.78125831, 0.76756136, -0.05716537, -0.64696257, 0.48918477,
0.15376225, -0.41974593],
[-0.625326 , 0.3509537 , -0.7884495 , 0.11773297, 0.23713942,
0.30296786, 0.12932378],
[ 0.88413986, -0.10958306, 0.9745586 , 0.8975006 , 0.23023047,
-0.89991669, -0.60032688],
[ 0.33462775, 0.62883724, -0.81839566, -0.70312966, -0.00246936,
-0.95542994, -0.33035891],
[-0.26355579, -0.58104982, -0.54748412, -0.30236209, -0.74270132,
0.46329941, 0.34277915],
[ 0.92837516, -0.06748299, 0.32837354, -0.62863672, 0.86226447,
0.63604586, 0.0905248 ]])
print(a)
array([[[-0.060161 , 0.36000953, 0.46160677, -0.66576281,
0.28562044, -0.60026872, 0.08034777],
[ 0.26379226, 0.67066755, -0.90139221, -0.86862163,
0.36405595, 0.71342926, -0.1265208 ],
[ 0.15007877, 0.82065234, 0.03984378, -0.20038364,
-0.09945102, 0.71605241, -0.55865999],
[ 0.27132257, -0.84289149, -0.15493576, 0.74683429,
-0.71159896, 0.50397217, -0.99025404],
[ 0.51546368, 0.45460343, 0.87519031, 0.0332339 ,
-0.53474897, -0.01733648, -0.02886814]],
[[ 0.04776443, 0.38018207, -0.73352382, 0.61847258,
-0.89731857, 0.57264147, -0.88192537],
[ 0.92657628, 0.6236141 , 0.41977008, 0.88720247,
0.44639323, 0.26165976, 0.2678753 ],
[-0.78125831, 0.76756136, -0.05716537, -0.64696257,
0.48918477, 0.15376225, -0.41974593],
[-0.625326 , 0.3509537 , -0.7884495 , 0.11773297,
0.23713942, 0.30296786, 0.12932378],
[ 0.44550219, -0.38828221, 0.35684203, 0.789946 ,
-0.8763921 , 0.90155917, -0.75549455]],
[[ 0.88413986, -0.10958306, 0.9745586 , 0.8975006 ,
0.23023047, -0.89991669, -0.60032688],
[ 0.33462775, 0.62883724, -0.81839566, -0.70312966,
-0.00246936, -0.95542994, -0.33035891],
[-0.26355579, -0.58104982, -0.54748412, -0.30236209,
-0.74270132, 0.46329941, 0.34277915],
[ 0.92837516, -0.06748299, 0.32837354, -0.62863672,
0.86226447, 0.63604586, 0.0905248 ],
[ 0.70272633, 0.17122912, -0.58209965, 0.55557024,
-0.46295566, -0.33845157, -0.62254313]]])
Here is a way to do that using tf.boolean_mask:
import tensorflow as tf
import numpy as np
# NumPy/Python implementation
a = np.random.uniform(-1,1,(3,5,7)).astype(np.float32)
a_length = np.random.randint(5,size=(3)).astype(np.int32)
res = []
for index, length_ in enumerate(a_length):
res.extend(a[index,:length_,:])
res = np.array(res)
# TensorFlow implementation
a_tf = tf.convert_to_tensor(a)
a_length_tf = tf.convert_to_tensor(a_length)
# Make a mask for all wanted elements
mask = tf.range(tf.shape(a)[1]) < a_length_tf[:, tf.newaxis]
# Apply mask
res_tf = tf.boolean_mask(a_tf, mask)
# Test
with tf.Session() as sess:
print(np.allclose(sess.run(res_tf), res))
Output:
True