Let's consider a function of two variables f(x1, x2) , where x1 spans over a vector v1 and x2 spans over a vector v2.
If f(x1, x2) = np.exp(x1 + x2), we can represent this function in Python as a matrix by means of the command numpy.meshgrid like this:
xx, yy = numpy.meshgrid(v1, v2)
M = numpy.exp(xx + yy)
This way, M is a representation of the function f over the cartesian product "v1 x v2", since M[i,j] = f(v1[i],v2[j]).
But this works because both sums and exponential work in parallel componentwise. My question is:
if my variable is x = numpy.array([x1, x2]) and f is a quadratic function f(x) = x.T # np.dot(Q, x), where Q is a 2x2 matrix, how can I do the same thing with the meshgrid function (i.e. calculating all the values of the function f on "v1 x v2" at once)?
Please let me know if I should include more details!
def quad(x, y, q):
"""Return an array A of a shape (len(x), len(y)) with
values A[i,j] = [x[i],y[j]] # q # [x[i],y[j]]
x, y: 1d arrays,
q: an array of shape (2,2)"""
from numpy import array, meshgrid, einsum
a = array(meshgrid(x, y)).transpose()
return einsum('ijk,kn,ijn->ij', a, q, a)
Notes
meshgrid produces 2 arrays of a shape (len(y), len(x)), where first one is with x values along the second dimension. If we apply to this pair np.array then a 3d array of shape (2, len(y), len(x)) will be produced. With transpose we obtain an array, where an element indexed by [i,j,k] is x[i] if k==0 else y[j], where k is 0 or 1, i.e. first or second array from meshgrid.
With 'ijk,kn,ijn->ij' we tell einsum to return the sum written bellow for each i, j:
sum(a[i,j,k]*q[k,n]*a[i,j,n] for k in range(2) for n in range(2))
Note, that a[i,j] == [x[i], y[j]].
I'm looking to find a vectorised way to create ndarrays from formulae, using the indices (or coordinates) of the value being calculated.
For example, if I want a 4x5x3 array filled by the formula 3x+y^z, I have no current way to reference x, y, or z directly. The closest I can come to that is wasting memory by creating incrementing arrays through arange().
Here are my current methods:
import numpy as np
# for loops
arr = np.empty((4, 5, 3))
for x in range(4):
for y in range(5):
for z in range(3):
arr[x, y, z] = 3 * x + y ** z
print(arr)
# separate arange()-created arrays for x, y, and z
x = np.arange(4)[:, None, None]
y = np.arange(5)[None, :, None]
z = np.arange(3)[None, None, :]
arr = 3 * x + y ** z
print(arr)
# same thing but written in a different way (I think)
all = np.arange(5)
arr = 3 * all[:4, None, None] + all[None, :, None] ** all[None, None, :3]
print(arr)
Is there any more efficient way to do this? I'm assuming there is, because numpy's underlying C code must use an iterator to find the memory addresses of each item, so surely that iterator could be repurposed as a faster alternative to using indices stored in memory?
I'd like to calculate a dot product of two matrices, where one of them is a diagonal matrix. However, I don't want to use np.diag or np.diagflat in order to create the full matrix, but instead use the 1D array directly filled with the diagonal values. Is there any way or numpy operation which I can use for this kind of problem?
x = np.arange(9).reshape(3,3)
y = np.arange(3) # diagonal elements
z = np.dot(x, np.diag(y))
and the solution I'm looking for should be without np.diag
z = x ??? y
Directly multiplying the ndarray by your vector will work. Numpy conveniently assumes that you want to multiply the nth column of x by the nth element of your y.
x = np.random.random((5, 5)
y = np.random.random(5)
diagonal_y = np.diag(y)
z = np.dot(x, diagonal_y)
np.allclose(z, x * y) # Will return True
The Einstein summation is an elegant solution to these kind of problems:
import numpy as np
x = np.random.uniform(0,1, size=5)
w = np.random.uniform(0,1, size=(5, 3))
diagonal_x = np.diagflat(x)
z = np.dot(diagonal_x, w)
zz = np.einsum('i,ij->ij',x , w)
np.allclose(z, zz) # Will return True
See: https://docs.scipy.org/doc/numpy/reference/generated/numpy.einsum.html#numpy.einsum
What is a good way to produce a numpy array containing the values of a function evaluated on an n-dimensional grid of points?
For example, suppose I want to evaluate the function defined by
def func(x, y):
return <some function of x and y>
Suppose I want to evaluate it on a two dimensional array of points with the x values going from 0 to 4 in ten steps, and the y values going from -1 to 1 in twenty steps. What's a good way to do this in numpy?
P.S. This has been asked in various forms on StackOverflow many times, but I couldn't find a concisely stated question and answer. I posted this to provide a concise simple solution (below).
shorter, faster and clearer answer, avoiding meshgrid:
import numpy as np
def func(x, y):
return np.sin(y * x)
xaxis = np.linspace(0, 4, 10)
yaxis = np.linspace(-1, 1, 20)
result = func(xaxis[:,None], yaxis[None,:])
This will be faster in memory if you get something like x^2+y as function, since than x^2 is done on a 1D array (instead of a 2D one), and the increase in dimension only happens when you do the "+". For meshgrid, x^2 will be done on a 2D array, in which essentially every row is the same, causing massive time increases.
Edit: the "x[:,None]", makes x to a 2D array, but with an empty second dimension. This "None" is the same as using "x[:,numpy.newaxis]". The same thing is done with Y, but with making an empty first dimension.
Edit: in 3 dimensions:
def func2(x, y, z):
return np.sin(y * x)+z
xaxis = np.linspace(0, 4, 10)
yaxis = np.linspace(-1, 1, 20)
zaxis = np.linspace(0, 1, 20)
result2 = func2(xaxis[:,None,None], yaxis[None,:,None],zaxis[None,None,:])
This way you can easily extend to n dimensions if you wish, using as many None or : as you have dimensions. Each : makes a dimension, and each None makes an "empty" dimension. The next example shows a bit more how these empty dimensions work. As you can see, the shape changes if you use None, showing that it is a 3D object in the next example, but the empty dimensions only get filled up whenever you multiply with an object that actually has something in those dimensions (sounds complicated, but the next example shows what i mean)
In [1]: import numpy
In [2]: a = numpy.linspace(-1,1,20)
In [3]: a.shape
Out[3]: (20,)
In [4]: a[None,:,None].shape
Out[4]: (1, 20, 1)
In [5]: b = a[None,:,None] # this is a 3D array, but with the first and third dimension being "empty"
In [6]: c = a[:,None,None] # same, but last two dimensions are "empty" here
In [7]: d=b*c
In [8]: d.shape # only the last dimension is "empty" here
Out[8]: (20, 20, 1)
edit: without needing to type the None yourself
def ndm(*args):
return [x[(None,)*i+(slice(None),)+(None,)*(len(args)-i-1)] for i, x in enumerate(args)]
x2,y2,z2 = ndm(xaxis,yaxis,zaxis)
result3 = func2(x2,y2,z2)
This way, you make the None-slicing to create the extra empty dimensions, by making the first argument you give to ndm as the first full dimension, the second as second full dimension etc- it does the same as the 'hardcoded' None-typed syntax used before.
Short explanation: doing x2, y2, z2 = ndm(xaxis, yaxis, zaxis) is the same as doing
x2 = xaxis[:,None,None]
y2 = yaxis[None,:,None]
z2 = zaxis[None,None,:]
but the ndm method should also work for more dimensions, without needing to hardcode the None-slices in multiple lines like just shown. This will also work in numpy versions before 1.8, while numpy.meshgrid only works for higher than 2 dimensions if you have numpy 1.8 or higher.
import numpy as np
def func(x, y):
return np.sin(y * x)
xaxis = np.linspace(0, 4, 10)
yaxis = np.linspace(-1, 1, 20)
x, y = np.meshgrid(xaxis, yaxis)
result = func(x, y)
I use this function to get X, Y, Z values ready for plotting:
def npmap2d(fun, xs, ys, doPrint=False):
Z = np.empty(len(xs) * len(ys))
i = 0
for y in ys:
for x in xs:
Z[i] = fun(x, y)
if doPrint: print([i, x, y, Z[i]])
i += 1
X, Y = np.meshgrid(xs, ys)
Z.shape = X.shape
return X, Y, Z
Usage:
def f(x, y):
# ...some function that can't handle numpy arrays
X, Y, Z = npmap2d(f, np.linspace(0, 0.5, 21), np.linspace(0.6, 0.4, 41))
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot_wireframe(X, Y, Z)
The same result can be achieved using map:
xs = np.linspace(0, 4, 10)
ys = np.linspace(-1, 1, 20)
X, Y = np.meshgrid(xs, ys)
Z = np.fromiter(map(f, X.ravel(), Y.ravel()), X.dtype).reshape(X.shape)
In the case your function actually takes a tuple of d elements, i.e. f((x1,x2,x3,...xd)) (for example the scipy.stats.multivariate_normal function), and you want to evaluate f on N^d combinations/grid of N variables, you could also do the following (2D case):
x=np.arange(-1,1,0.2) # each variable is instantiated N=10 times
y=np.arange(-1,1,0.2)
Z=f(np.dstack(np.meshgrid(x,y))) # result is an NxN (10x10) matrix, whose entries are f((xi,yj))
Here np.dstack(np.meshgrid(x,y)) creates an 10x10 "matrix" (technically a 10x10x2 numpy array) whose entries are the 2-dimensional tuples to be evaluated by f.
My two cents:
import numpy as np
x = np.linspace(0, 4, 10)
y = np.linspace(-1, 1, 20)
[X, Y] = np.meshgrid(x, y, indexing = 'ij', sparse = 'true')
def func(x, y):
return x*y/(x**2 + y**2 + 4)
# I have defined a function of x and y.
func(X, Y)
I have three separate 1d arrays of a list of numbers, their squares and cubes (created by a 'for' loop).
I would like these arrays to appear in three corresponding columns, however I have tried the column_stack function and python says its not defined. I have read about the vstack and hstack functions but am confused about which to use and what exactly they do.
My code so far reads;
import numpy
makearange = lambda a: numpy.arange(int(a[0]),int(a[1]),int(a[2]))
x = makearange(raw_input('Enter start,stop,increment: ').split(','))
y = numpy.zeros(len(x), dtype=int)
z = numpy.zeros(len(x), dtype=int)
for i in range(len(x)):
y[i] = x[i]**2
for i in range(len(x)):
z[i] = x[i]**3
print 'original array: ',x
print 'squared array: ',y
print 'cubed array: ', z
I would appreciate any advice
Why don't you define y and z directly?
y = x**2
z = x**3
and then simply:
stacked = np.column_stack((x,y,z))
which gives you a 2D array of shape len(x) * 3
import numpy
makearange = lambda a: numpy.arange(int(a[0]),int(a[1]),int(a[2]))
x = makearange(raw_input('Enter start,stop,increment: ').split(','))
a = np.zeros((len(x),3))
a[:,0] = x
a[:,1] = x**2
a[:,2] = x**3
When using arrays you should avoid for loops as much as possible, that's kind of the point of arrays.
a = np.zeros((len(x),3)) creates an array of length same as x and with 3 columns
a[:,i] is a reference to the 'i'th column of this array (i.e. select all values (denoted by :) along this (i) column)
I would strongly recommend you look at the Numpy Tutorial.
You do want column_stack. Have you tried:
w = numpy.column_stack((x,y,z))
print(w)