I have a set of data loaded in from a csv file, with 1D arrays representing the x,y,z coords of the data points, and another 1D array, T, representing the value of a field at the corresponding points. The points are not uniform in space.
I am struggling to interpolate T a given point xi,yi,zi. scipy's interpn seems to want to accept T only as a 3D array, which doesn't make sense to me as T is simply 1D data?
Any advice would be appreciated.
Edit:
Example:
import numpy as np
x = np.array([1.0,1.5,1.1,1.3,1.4])
y = np.array([1.1,1.3,1.2,1.4,1.45])
z = np.array([1.0,1.1,1.4,1.2,1.0])
T = np.array([5.0,5.1,5.4,4.6,4.9])
point = ([1.2,1.1,1.25])
from scipy.interpolate import interpn
out = interpn((x,y,z),T,point)
print(out)
Cheers
I want to generate a dataset with m random data points of k dimensions each. Thus resulting in data size of shape (m, k). These points should be i.i.d. from a normal distribution with mean 0 and standard deviation 1. There are 2 ways of generating these points.
First way:
import numpy as np
# Initialize the array
Z = np.zeros((m, k))
# Generate each point of each dimension independent of each other
for datapoint in range(m):
z = [np.random.standard_normal() for _ in range(k)]
Z[datapoint] = z[:]
Second way:
import numpy as np
# Directly sample the points
Z = np.random.normal(0, 1, (m, k))
What I think is the 2nd way gives a resulting dataset not independent of each other but the 1st one gives i.i.d dataset of points. Is this the difference between the 2 pieces of code?
My assumption would be that standard_normal is just normal with "standard" parameters (mean=0 and std=1).
Let's test that:
import numpy as np
rng0 = np.random.default_rng(43210)
rng1 = np.random.default_rng(43210)
print(rng0.standard_normal(10))
print(rng1.normal(0, 1, 10))
which gives:
[ 0.62824213 -1.18535536 -1.18141382 -0.74127753 -0.41945915 1.02656223 -0.64935657 1.70859865 0.47731614 -1.12700957]
[ 0.62824213 -1.18535536 -1.18141382 -0.74127753 -0.41945915 1.02656223 -0.64935657 1.70859865 0.47731614 -1.12700957]
So I think that assumption was correct.
Given data with shape = (t,m,n), I need to find a vector variable of shape (n,) that minimizes a convex function of the data and vector. I've used cvxopt (and cvxpy) to perform convex optimizations using 2D input, but it seems like they don't support 3D arrays. Is there a way to implement this convex optimization using these or other similar packages?
Given data with shape (t,m,n) and (t,m) and var with shape (n,), here's a simplification of the type of function I need to minimize:
import numpy as np
obj_func(var,data1,data2):
#data1.shape = (t,m,n)
#data2.shape = (t,m)
#var.shape = (n,)
score = np.sum(data1*var,axis=2) #dot product along axis 2
time_series = np.sum(score*data2,axis=1) #weighted sum along axis 1
return np.sum(time_series)-np.sum(time_series**2) #some function
This seems like it should be a simple convex optimization, but unfortunately these functions aren't supported on N-dimensional arrays in cvxopt/cvxpy. Is there a way to implement this?
I think if you simply reshape data1 to be 2d temporarily you'll be fine, e.g.
import numpy as np
import cvxpy as cp
t, m, n = 10, 8, 6
data1 = np.ones((t, m, n))
data2 = np.ones((t, m))
x = cp.Variable(n)
score = cp.reshape(data1.reshape(-1, n) * x, (t, m))
time_series = cp.sum(cp.multiply(score, data2), axis=1)
expr = cp.sum(time_series) - cp.sum(time_series ** 2)
print(repr(expr))
Outputs:
Expression(CONCAVE, UNKNOWN, ())
I have a 2d array/matrix like this, how would I randomly pick the value from this 2D matrix, for example getting value like [-62, 29.23]. I looked at the numpy.choice but it is built for 1d array.
The following is my example with 4 rows and 8 columns
Space_Position=[
[[-62,29.23],[-49.73,29.23],[-31.82,29.23],[-14.2,29.23],[3.51,29.23],[21.21,29.23],[39.04,29.23],[57.1,29.23]],
[[-62,11.28],[-49.73,11.28],[-31.82,11.28],[-14.2,11.28],[3.51,11.28],[21.21,11.28] ,[39.04,11.28],[57.1,11.8]],
[[-62,-5.54],[-49.73,-5.54],[-31.82,-5.54] ,[-14.2,-5.54],[3.51,-5.54],[21.21,-5.54],[39.04,-5.54],[57.1,-5.54]],
[[-62,-23.1],[-49.73,-23.1],[-31.82,-23.1],[-14.2,-23.1],[3.51,-23.1],[21.21,-23.1],[39.04,-23.1] ,[57.1,-23.1]]
]
In the answers the following solution was given:
random_index1 = np.random.randint(0, Space_Position.shape[0])
random_index2 = np.random.randint(0, Space_Position.shape[1])
Space_Position[random_index1][random_index2]
this indeed works to give me one sample, how about more than one sample like what np.choice() does?
Another way I am thinking is to tranform the matrix into a array instead of matrix like,
Space_Position=[
[-62,29.23],[-49.73,29.23],[-31.82,29.23],[-14.2,29.23],[3.51,29.23],[21.21,29.23],[39.04,29.23],[57.1,29.23], ..... ]
and at last use np.choice(), however I could not find the ways to do the transformation, np.flatten() makes the array like
Space_Position=[-62,29.23,-49.73,29.2, ....]
Just use a random index (in your case 2 because you have 3 dimensions):
import numpy as np
Space_Position = np.array(Space_Position)
random_index1 = np.random.randint(0, Space_Position.shape[0])
random_index2 = np.random.randint(0, Space_Position.shape[1])
Space_Position[random_index1, random_index2] # get the random element.
The alternative is to actually make it 2D:
Space_Position = np.array(Space_Position).reshape(-1, 2)
and then use one random index:
Space_Position = np.array(Space_Position).reshape(-1, 2) # make it 2D
random_index = np.random.randint(0, Space_Position.shape[0]) # generate a random index
Space_Position[random_index] # get the random element.
If you want N samples with replacement:
N = 5
Space_Position = np.array(Space_Position).reshape(-1, 2) # make it 2D
random_indices = np.random.randint(0, Space_Position.shape[0], size=N) # generate N random indices
Space_Position[random_indices] # get N samples with replacement
or without replacement:
Space_Position = np.array(Space_Position).reshape(-1, 2) # make it 2D
random_indices = np.arange(0, Space_Position.shape[0]) # array of all indices
np.random.shuffle(random_indices) # shuffle the array
Space_Position[random_indices[:N]] # get N samples without replacement
Refering to numpy.random.choice:
Sampling random rows from a 2-D array is not possible with this function, but is possible with Generator.choice through its axis keyword.
The genrator documentation is linked here numpy.random.Generator.choice.
Using this knowledge. You can create a generator and then "choice" from your array:
rng = np.random.default_rng() #creates the generator ==> Generator(PCG64) at 0x2AA703BCE50
N = 3 #Number of Choices
a = np.array(Space_Position) #makes sure, a is an ndarray and numpy-supported
s = a.shape #(4,8,2)
a = a.reshape((s[0] * s[1], s[2])) #makes your array 2 dimensional keeping the last dimension seperated
a.shape #(32, 2)
b = rng.choice(a, N, axis=0, replace=False) #returns N choices of a in array b, e.g. narray([[ 57.1 , 11.8 ], [ 21.21, -5.54], [ 39.04, 11.28]])
#Note: replace=False prevents having the same entry several times in the result
Space_Position[np.random.randint(0, len(Space_Position))]
[np.random.randint(0, len(Space_Position))]
gives you what you want
I am using the numpy's random.normal routine to create a Gaussian with a given mean and standard deviation.
array_a = an array of len(100)
gaussian = np.random.normal(loc=array_a,scale=0.1,size=len(2*array_a))
So I expect the gaussian to have a mean=array_a and stddev=0.1 and the size of the gaussian array to be 2 times array_a.
However the above returns me an array with the same size as that of array_a !
How do I get the len(gaussian) to be 2 times len(array_a) with the given mean and standard deviation?
you have to multiplicate len(array_a) * 2 instead of len(array_a * 2) and loc=array_a.mean()
Try:
import numpy as np
array_a = np.arange(100)
gaussian = np.random.normal(loc=array_a.mean(), scale=0.1, size=2 * len(array_a))
Now gaussian.size is 200 and gaussian.mean() is equal to array_a.mean().