I'll try to explain my issue here without going into too much detail on the actual application so that we can stay grounded in the code. Basically, I need to do operations to a vector field. My first step is to generate the field as
x,y,z = np.meshgrid(np.linspace(-5,5,10),np.linspace(-5,5,10),np.linspace(-5,5,10))
Keep in mind that this is a generalized case, in the program, the bounds of the vector field are not all the same. In the general run of things, I would expect to say something along the lines of
u,v,w = f(x,y,z).
Unfortunately, this case requires so more difficult operations. I need to use a formula similar to
where the vector r is defined in the program as np.array([xgrid-x,ygrid-y,zgrid-z]) divided by its own norm. Basically, this is a vector pointing from every point in space to the position (x,y,z)
Now Numpy has implemented a cross product function using np.cross(), but I can't seem to create a "meshgrid of vectors" like I need.
I have a lambda function that is essentially
xgrid,ygrid,zgrid=np.meshgrid(np.linspace(-5,5,10),np.linspace(-5,5,10),np.linspace(-5,5,10))
B(x,y,z) = lambda x,y,z: np.cross(v,np.array([xgrid-x,ygrid-y,zgrid-z]))
Now the array v is imported from another class and seems to work just fine, but the second array, np.array([xgrid-x,ygrid-y,zgrid-z]) is not a proper shape because it is a "vector of meshgrids" instead of a "meshgrid of vectors". My big issue is that I cannot seem to find a method by which to format the meshgrid in such a way that the np.cross() function can use the position vector. Is there a way to do this?
Originally I thought that I could do something along the lines of:
x,y,z = np.meshgrid(np.linspace(-2,2,5),np.linspace(-2,2,5),np.linspace(-2,2,5))
A = np.array([x,y,z])
cross_result = np.cross(np.array(v),A)
This, however, returns the following error, which I cannot seem to circumvent:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\numpy\core\numeric.py", line 1682, in cross
raise ValueError(msg)
ValueError: incompatible dimensions for cross product
(dimension must be 2 or 3)
There's a work around with reshape and broadcasting:
A = np.array([x_grid, y_grid, z_grid])
# A.shape == (3,5,5,5)
def B(v, p):
'''
v.shape = (3,)
p.shape = (3,)
'''
shape = A.shape
Ap = A.reshape(3,-1) - p[:,None]
return np.cross(v[None,:], Ap.reshape(3,-1).T).reshape(shape)
print(B(v,p).shape)
# (3, 5, 5, 5)
I think your original attempt only lacks the specification of the axis along which the cross product should be executed.
x, y, z = np.meshgrid(np.linspace(-2, 2, 5),np.linspace(-2, 2, 5), np.linspace(-2, 2, 5))
A = np.array([x, y, z])
cross_result = np.cross(np.array(v), A, axis=0)
I tested this with the code below. As an alternative to np.array([x, y, z]), you can also use np.stack(x, y, z, axis=0), which clearly shows along which axis the meshgrids are stacked to form a meshgrid of vectors, the vectors being aligned with axis 0. I also printed the shape each time and used random input for testing. In the test, the output of the formula is compared at a random index to the cross product of the input-vector at the same index with vector v.
import numpy as np
x, y, z = np.meshgrid(np.linspace(-5, 5, 10), np.linspace(-5, 5, 10), np.linspace(-5, 5, 10))
p = np.random.rand(3) # random reference point
A = np.array([x-p[0], y-p[1], z-p[2]]) # vectors from positions to reference
A_bis = np.stack((x-p[0], y-p[1], z-p[2]), axis=0)
print(f"A equals A_bis? {np.allclose(A, A_bis)}") # the two methods of stacking yield the same
v = -1 + 2*np.random.rand(3) # random vector v
B = np.cross(v, A, axis=0) # cross-product for all points along correct axis
print(f"Shape of v: {v.shape}")
print(f"Shape of A: {A.shape}")
print(f"Shape of B: {B.shape}")
print("\nComparison for random locations: ")
point = np.random.randint(0, 9, 3) # generate random multi-index
a = A[:, point[0], point[1], point[2]] # look up input-vector corresponding to index
b = B[:, point[0], point[1], point[2]] # look up output-vector corresponding to index
print(f"A[:, {point[0]}, {point[1]}, {point[2]}] = {a}")
print(f"v = {v}")
print(f"Cross-product as v x a: {np.cross(v, a)}")
print(f"Cross-product from B (= v x A): {b}")
The resulting output looks like:
A equals A_bis? True
Shape of v: (3,)
Shape of A: (3, 10, 10, 10)
Shape of B: (3, 10, 10, 10)
Comparison for random locations:
A[:, 8, 1, 1] = [-4.03607312 3.72661831 -4.87453077]
v = [-0.90817859 0.10110274 -0.17848181]
Cross-product as v x a: [ 0.17230515 -3.70657882 -2.97637688]
Cross-product from B (= v x A): [ 0.17230515 -3.70657882 -2.97637688]
Related
I have two numpy arrays (4x4 each). I would like to concatenate them to a tensor of (4x4x2) in which the first 'sheet' is the first array, second 'sheet' is the second array, etc. However, when I try np.stack the output of d[1] is not showing the correct values of the first matrix.
import numpy as np
x = array([[ 3.38286851e-02, -6.11905173e-05, -9.08147798e-03,
-2.46860166e-02],
[-6.11905173e-05, 1.74237508e-03, -4.52140165e-04,
-1.22904439e-03],
[-9.08147798e-03, -4.52140165e-04, 1.91939979e-01,
-1.82406361e-01],
[-2.46860166e-02, -1.22904439e-03, -1.82406361e-01,
2.08321422e-01]])
print(np.shape(x)) # 4 x 4
y = array([[ 6.76573701e-02, -1.22381035e-04, -1.81629560e-02,
-4.93720331e-02],
[-1.22381035e-04, 3.48475015e-03, -9.04280330e-04,
-2.45808879e-03],
[-1.81629560e-02, -9.04280330e-04, 3.83879959e-01,
-3.64812722e-01],
[-4.93720331e-02, -2.45808879e-03, -3.64812722e-01,
4.16642844e-01]])
print(np.shape(y)) # 4 x 4
d = np.dstack((x,y))
np.shape(d) # indeed it is 4,4,2... but if I do d[1] then it is not the first x matrix.
d[1] # should be y
If you do np.dstack((x, y)), which is the same as the more explicit np.stack((x, y), axis=-1), you are concatenating along the last, not the first axis (i.e., the one with size 2):
(x == d[..., 0]).all()
(y == d[..., 1]).all()
Ellipsis (...) is a python object that means ": as many times as necessary" when used in an index. For a 3D array, you can equivalently access the leaves as
d[:, :, 0]
d[:, :, 1]
If you want to access the leaves along the first axis, your array must be (2, 4, 4):
d = np.stack((x, y), axis=0)
(x == d[0]).all()
(y == d[1]).all()
Use np.stack instead of np.dstack:
>>> d = np.stack([y, x])
>>> np.all(d[0] == y)
True
>>> np.all(d[1] == x)
True
>>> d.shape
(2, 4, 4)
I'm having an issue where I'm adding two 4x1 arrays and the result is a 4x4 array where the first column is repeated 4 times. The result I need is a 4x1 array.
I've initialized an array as such (m = 4): z = np.zeros((m, len(t))
Later in my code I pass this array into a function as z[:,k+1] so the dimensionality becomes a 4x1 array. (Note that when I print this array to my terminal is shows up as a row vector and not a column vector: [0. 0. 0. 0.], I'm not sure why this is either). The array that I'm trying to add to z has the following structure when printed to my terminal:
[[#]
[#]
[#]
[#]]
Clearly the addition is pulling the above array into each element of z instead of adding their respective components together, but I'm not sure why as they should both be column vectors. I'd appreciate any help with this.
EDIT: I have a lot of code so I've included a condensed version that hopefully gets the idea accross.
n = 4 # Defines number of states
m = 4 # Defines number of measurements
x = np.zeros((n, len(t)), dtype=np.float64) # Initializes states
z = np.zeros((m, len(t)), dtype=np.float64) # Initializes measurements
u = np.zeros((1, len(t)), dtype=np.float64) # Initializes input
...
C = np.eye(m) # Defines measurement matrix
...
for k in range(len(t)-1):
...
x_ukf[:,k+1], P_ukf[k+1,:,:] = function_call(x_ukf[:,k], z[:,k+1], u[:,k], P_ukf[k,:,:], C, Q, R, T) # Calls UKF function
This then leads to the function where the following occurrs (note that measurement_matrix = C (4x4 matrix), X is a 4x9 matrix, and W a 1x9 row vector):
Z = measurement_matrix # X # Calculates measurements based on sigma points
zhat = Z # W.T
...
state_vec = state_vec + K # (measurement_vec - zhat) # Updates state estimates
The issue I'm having is with the expression (measurement_vec - zhat). This is where the result should be a 4x1 vector but I'm getting a 4x4 matric.
This is sometimes called broadcasting:
a, b = np.arange(4), np.arange(8,12)
c = a + b[:,None]
Output:
array([[ 8, 9, 10, 11],
[ 9, 10, 11, 12],
[10, 11, 12, 13],
[11, 12, 13, 14]])
I have a problem showing data in a graph. The graph frame appears, but no graph is to be seen. Can you please help ?
I made sure that the dimension of the x axis and the data is the same ... I simply cannot find out why I do not get a graph in return.
Thank you very much in advance.
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm
n = 1000
theta = 0.8
d = np.sqrt(1-theta**2)
def p(x,y):
"Stochastic kernel for the TAR model"
return norm().pdf((y-theta*np.abs(x))/d)/d
Z = norm().rvs(n)
X = np.empty(n)
for t in range(n-1):
X[t+1] = theta*np.abs(X[t])+d*Z[t+1]
n = len(X)
X = X.reshape((n, 1))
ys = np.linspace(-3,3,200)
k = len(ys)
ys = ys.reshape((1,k))
v = p(X,ys)
kernel = np.mean(v, axis=0)
h = len(kernel)
kernel = kernel.reshape((1,h))
fig, ax = plt.subplots(figsize=(10,7))
ax.plot(ys,kernel, 'b-', lw=2,alpha=0.6, label='look ahead estimate')
plt.show()
The problem is, that through reshaping the two 1-dimensional arrays ys and kernel to a 1xk or 1xh array respectively you get 2-dimensional arrays, where the first dimension is 1. The plot function apparently only iterates through the first dimension, which is why the plot doesn't show anything.
I can think of two easy options to fix that:
Do not reshape the variables kernel and ys:
# ... continuing your code ...
ys = np.linspace(-3,3,200)
k = len(ys)
#ys = ys.reshape((1,k))
v = p(X,ys)
kernel = np.mean(v, axis=0)
h = len(kernel)
#kernel = kernel.reshape((1,h))
fig, ax = plt.subplots(figsize=(10,7))
ax.plot(ys,kernel, 'b-', lw=2,alpha=0.6, label='look ahead estimate')
plt.show()
Call your plot function like this:
ax.plot(ys[0],kernel[0], 'b-', lw=2, alpha=0.6, label='look ahead estimate')
I hope this solves your problem.
To understand why you still have to reshape X:
Let's first understand your function p(x,y) in terms of dimensions:
def p(x,y):
"Stochastic kernel for the TAR model"
"""If x is not reshaped, you substract two one-dimensional arrays from each other,
which have not the same dimensions (dim(x) == 1000, dim(y) == 200 in your case).
This throws an error.
If you reshape X before passing to this function, the y array is substracted
element-wise by each of the values of X, which gives you a matrix with dimension
dim(x) x dim(y).
"""
return norm().pdf((y-theta*np.abs(x))/d)/d
For illustration what happens here dimension-wise:
>>> X = np.array([[1], [2], [3], [4]])
>>> Y = np.array([1, 2, 3])
>>> Y-X
array([[ 0, 1, 2],
[-1, 0, 1],
[-2, -1, 0],
[-3, -2, -1]])
Now we take a look what happens with the matrix returned by p(x,y):
The calculation of the kernel with np.mean(v, axis=0), where v is the returned matrix from p(X,ys), works such, that np.mean iterates over the lines of the matrix v and calculates the mean value of each "line vector" in the matrix. This gives you an one dimensional array (dimension of ys) which you can plot over ys.
I've been oddly bashing my head against this problem for several hours, and would appreciate any help!
I would like to create a (for example) 100x100 array in which each index is a (x,y) coordinate. The overall goal here is the following:
I have x,y coordinates and would like to arrange them in a 2D space so that I can use the np.diagonal function to return the (x,y) coordinates along a line. I'll then use those (x,y) points to compare particular values.
The first step here is actually creating the array and I just can't seem to do it.
I'm not sure about the numpy part of your request, but you can create the array like so:
coords = [[(y,x) for x in range(100)] for y in range(100)]
>>> coords[50][2]
(50,2)
If you just want the values along the diagonal, why dont you just create a 1D list?
import numpy as np
xs = np.linspace(1,10,100) # assuming x goes form 1 to 10
ys = np.linspace(2,3, 100) # assuming y goes from 2 to 3
xy = zip(xs, ys)
You no longer need the 2d array and then call the diagonal.
Working on Jaime's suggestion:
>>> x, y = numpy.mgrid[0:100, 0:100]
>>> z = numpy.array([x, y]).transpose([1,2,0])
>>> z[50, 2]
array([50, 2])
EDIT: Given an array of points p, of the shape (2, P), this is how you would find out which of these points are underneath diagonal n:
>>> d = numpy.diagonal(z, n)
>>> cond0 = p[0, None] < d[0, :, None]
>>> cond1 = p[1, None] < d[1, :, None]
>>> good_indices_full = numpy.where(numpy.logical_and(cond0, cond1))
>>> good_indices = good_indices_full[1]
(I prefer to work with "good_indices", i.e. write stuff like p[:, good_indices], rather than the full tuple of arrays that numpy.where gives back).
I have a point set which I have stored its coordinates in three different arrays (xa, ya, za). Now, I want to calculate the euclidean distance between each point of this point set (xa[0], ya[0], za[0] and so on) with all the points of an another point set (xb, yb, zb) and every time store the minimum distance in a new array.
Let's say that xa.shape = (11,), ya.shape = (11,), za.shape= (11,). Respectively, xb.shape = (13,), yb.shape = (13,), zb.shape = (13,). What I want to do is to take each time one xa[],ya[],za[], and calculate its distance with all the elements of xb, yb, zb, and at the end store the minimum value into an xfinal.shape = (11,) array.
Do you think that this would be possible with numpy?
A different solution would be to use the spatial module from scipy, the KDTree in particular.
This class learn from a set of data and can be interrogated given a new dataset:
from scipy.spatial import KDTree
# create some fake data
x = arange(20)
y = rand(20)
z = x**2
# put them togheter, should have a form [n_points, n_dimension]
data = np.vstack([x, y, z]).T
# create the KDTree
kd = KDTree(data)
now if you have a point you can ask the distance and the index of the closet point (or the N closest points) simply by doing:
kd.query([1, 2, 3])
# (1.8650720813822905, 2)
# your may differs
or, given an array of positions:
#bogus position
x2 = rand(20)*20
y2 = rand(20)*20
z2 = rand(20)*20
# join them togheter as the input
data2 = np.vstack([x2, y2, z2]).T
#query them
kd.query(data2)
#(array([ 14.96118553, 9.15924813, 16.08269197, 21.50037074,
# 18.14665096, 13.81840533, 17.464429 , 13.29368755,
# 20.22427196, 9.95286671, 5.326888 , 17.00112683,
# 3.66931946, 20.370496 , 13.4808055 , 11.92078034,
# 5.58668204, 20.20004206, 5.41354322, 4.25145521]),
#array([4, 3, 2, 4, 2, 2, 4, 2, 3, 3, 2, 3, 4, 4, 3, 3, 3, 4, 4, 4]))
You can calculate the difference from each xa to each xb with np.subtract.outer(xa, xb). The distance to the nearest xb is given by
np.min(np.abs(np.subtract.outer(xa, xb)), axis=1)
To extend this to 3D,
distances = np.sqrt(np.subtract.outer(xa, xb)**2 + \
np.subtract.outer(ya, yb)**2 + np.subtract.outer(za, zb)**2)
distance_to_nearest = np.min(distances, axis=1)
If you actually want to know which of the b points is the nearest, you use argmin in place of min.
index_of_nearest = np.argmin(distances, axis=1)
There is more than one way of doing this. Most importantly, there's a trade-off between memory-usage and speed. Here's the wasteful method:
s = (1, -1)
d = min((xa.reshape(s)-xb.reshape(s).T)**2
+ (ya.reshape(s)-yb.reshape(s).T)**2
+ (za.reshape(s)-zb.reshape(s).T)**2), axis=0)
The other method would be to iterate over the point set in b to avoid the expansion to the full blown matrix.