A bijective function from N*N*N to N - python

Can anyone help me in finding a bijective mathematical function from N * N * N → N that takes three parameters x, y, and z and returns a number n?
I would like to know the function f and its inverse f' in a way that if I have n I will be able to determine x, y, z by applying f'(n).


Defining f as a composition of a simpler function g
Suppose g is a bijection from N × N to N and let g-1 be its inverse. Then we can define f in terms of g as follows.
f(x, y, z) = g(g(x, y), z) = n
f-1(n) = (x, y, z) where g-1(n) = (w, z) and g-1(w) = (x, y)
Defining g as a bijection from N × N to N
We now have the much simpler problem of defining g.
g(x, y) = (x + y)(x + y + 1) / 2 + y = n
g-1(n) = (x, y) where m = ⌊(2n)1/2⌋ and exactly one of the following two conditions hold.
x + y = m and y = n - m(m + 1) / 2
x + y = m - 1 and y = n - m(m - 1) / 2
Python implementation
def f(x, y, z):
return g(g(x, y), z)
def f_inv(n):
w, z = g_inv(n)
x, y = g_inv(w)
return (x, y, z)
def g(x, y):
return (x + y) * (x + y + 1) / 2 + y
def g_inv(n):
m = math.floor(math.sqrt(2 * n))
while True:
y = n - m * (m + 1) / 2
if y >= 0:
break
m -= 1
x = m - y
return x, y


your function is not surjective, let p is a prime number, we can't find any x,y,z in N such that p=2^x3^y5^z...


F(x,y,z) = 2^x*3^y*5^z
In fact you can choose any distinct set of prime numbers. And inverse is simply by factorizing to corresponding prime factors.

Related

Numpy - vectorize the bivariate poisson pmf equation

I'm trying to write a function to evaluate the probability mass function for the bivariate poisson distribution.
This is easy when all of the parameters (x, y, theta1, theta2, theta0) are scalars, but tricky to scale up without loops to allow these parameters to be vectors. I need it to scale such that, for:
theta0 being a scalar - the "correlation parameter" in the equation
theta1 and theta2 having length l
x, y both having length n
the output array would have shape (l, n, n). For example, a slice [j, :, :] from the output array would look like:
The first part (the constant, before the summation) I think i've figured out:
import numpy as np
from scipy.special import factorial
def constant(theta1, theta2, theta0, x, y):
exponential_part = np.exp(-(theta1 + theta2 + theta0)).reshape(-1, 1, 1)
x = np.tile(x, (len(x), 1)).transpose()
y = np.tile(y, (len(y), 1))
double_factorial = (np.power(np.array(theta1).reshape(-1, 1, 1), x)/factorial(x)) * \
(np.power(np.array(theta2).reshape(-1, 1, 1), y)/factorial(y))
return exponential_part * double_factorial
But I'm struggling with the summation part. How can I vectorize a summation where the limits depend on variable arrays?

I think I have this figured out, based on the approach that #w-m suggests: calculate every possible summation term which could appear, based on the maximum x or y value which appears, and use a mask to get rid of the ones you don't want. Assuming you have your x and y terms go from 0 to N, in consecutive order, this is calculating up to three times more terms than are actually required, but this is offset by getting to use vectorization.
Reference implementation
I wrote this by first writing a pure-Python reference implementation, which just implements your problem using loops. With 4 nested loops, it's not exactly fast, but it's handy to have while testing the numpy version.
import numpy as np
from scipy.special import factorial, comb
import operator as op
from functools import reduce
def choose(n, r):
# https://stackoverflow.com/a/4941932/530160
r = min(r, n-r)
numer = reduce(op.mul, range(n, n-r, -1), 1)
denom = reduce(op.mul, range(1, r+1), 1)
return numer // denom # or / in Python 2
def reference_impl_constant(s_theta1, s_theta2, s_theta0, s_x, s_y):
# Cast to float to prevent overflow
s_theta1 = float(s_theta1)
s_theta2 = float(s_theta2)
s_theta0 = float(s_theta0)
s_x = float(s_x)
s_y = float(s_y)
term1 = np.exp(-(s_theta1 + s_theta2 + s_theta0))
term2 = (s_theta1 ** s_x / factorial(s_x))
term3 = (s_theta2 ** s_y / factorial(s_y))
assert term1 >= 0
assert term2 >= 0
assert term3 >= 0
return term1 * term2 * term3
def reference_impl_constant_loop(theta1, theta2, theta0, x, y):
theta_len = theta1.shape[0]
xy_len = x.shape[0]
constant_array = np.zeros((theta_len, xy_len, xy_len))
for i in range(theta_len):
for j in range(xy_len):
for k in range(xy_len):
s_theta1 = theta1[i]
s_theta2 = theta2[i]
s_theta0 = theta0
s_x = x[j]
s_y = y[k]
constant_term = reference_impl_constant(s_theta1, s_theta2, s_theta0, s_x, s_y)
assert constant_term >= 0
constant_array[i, j, k] = constant_term
return constant_array
def reference_impl_summation(s_theta1, s_theta2, s_theta0, s_x, s_y):
sum_ = 0
for i in range(min(s_x, s_y) + 1):
sum_ += choose(s_x, i) * choose(s_y, i) * factorial(i) * ((s_theta0/s_theta1/s_theta2) ** i)
assert sum_ >= 0
return sum_
def reference_impl_summation_loop(theta1, theta2, theta0, x, y):
theta_len = theta1.shape[0]
xy_len = x.shape[0]
summation_array = np.zeros((theta_len, xy_len, xy_len))
for i in range(theta_len):
for j in range(xy_len):
for k in range(xy_len):
s_theta1 = theta1[i]
s_theta2 = theta2[i]
s_theta0 = theta0
s_x = x[j]
s_y = y[k]
summation_term = reference_impl_summation(s_theta1, s_theta2, s_theta0, s_x, s_y)
assert summation_term >= 0
summation_array[i, j, k] = summation_term
return summation_array
def reference_impl(theta1, theta2, theta0, x, y):
# all array inputs must be 1D
assert len(theta1.shape) == 1
assert len(theta2.shape) == 1
assert len(x.shape) == 1
assert len(y.shape) == 1
# theta vectors must have same length
theta_len = theta1.shape[0]
assert theta2.shape[0] == theta_len
# x and y must have same length
xy_len = x.shape[0]
assert y.shape[0] == xy_len
# theta0 is scalar
assert isinstance(theta0, (int, float))
constant_array = np.zeros((theta_len, xy_len, xy_len))
output = np.zeros((theta_len, xy_len, xy_len))
constant_array = reference_impl_constant_loop(theta1, theta2, theta0, x, y)
summation_array = reference_impl_summation_loop(theta1, theta2, theta0, x, y)
output = constant_array * summation_array
return output
Numpy implementation
I split the implementation of this across two functions.
The fast_constant() function calculates everything to the left of the summation symbol. The fast_summation() function calculates everything inside the summation symbol.
import numpy as np
from scipy.special import factorial, comb
def fast_summation(theta1, theta2, theta0, x, y):
x = np.tile(x, (len(x), 1)).transpose()
y = np.tile(y, (len(y), 1))
sum_limit = np.minimum(x, y)
max_sum_limit = np.max(sum_limit)
i = np.arange(max_sum_limit + 1).reshape(-1, 1, 1)
summation_mask = (i <= sum_limit)
theta_ratio = (theta0 / (theta1 * theta2)).reshape(-1, 1, 1, 1)
theta_to_power = np.power(theta_ratio, i)
terms = comb(x, i) * comb(y, i) * factorial(i) * theta_to_power
# mask out terms which aren't part of sum
terms *= summation_mask
# axis 0 is theta
# axis 1 is i
# axis 2 & 3 are x and y
# so sum across axis 1
terms = terms.sum(axis=1)
return terms
def fast_constant(theta1, theta2, theta0, x, y):
theta1 = theta1.astype('float64')
theta2 = theta2.astype('float64')
exponential_part = np.exp(-(theta1 + theta2 + theta0)).reshape(-1, 1, 1)
# x and y must be 1D
assert len(x.shape) == 1
assert len(y.shape) == 1
# x and y must have same shape
assert x.shape == y.shape
x_len, y_len = x.shape[0], y.shape[0]
x = x.reshape((x_len, 1))
y = y.reshape((1, y_len))
double_factorial = (np.power(np.array(theta1).reshape(-1, 1, 1), x)/factorial(x)) * \
(np.power(np.array(theta2).reshape(-1, 1, 1), y)/factorial(y))
return exponential_part * double_factorial
def fast_impl(theta1, theta2, theta0, x, y):
return fast_summation(theta1, theta2, theta0, x, y) * fast_constant(theta1, theta2, theta0, x, y)
Benchmarking
Assuming that X and Y range from 0 to 20, and that theta is centered somewhere inside that range, I get the result that the numpy version is roughly 280 times faster than the pure python reference.
Numerical stability
I'm unsure how numerically stable this is. For example, when I center theta at 100, I get a floating-point overflow. Typically, when computing an expression which has lots of choose and factorial expressions inside it, you'll use some mathematical equivalent which results in smaller intermediate sums. In this case I have so little understanding of the math that I don't know how you'd do that.

How to generate a polynomial dataset

I am trying to generate a polynomial dataset. I wrote a code
def generate_dataset1():
n = 500
X = 2 - 3 * np.random.normal(0, 1, n)
y = X - 2 * (X ** 2) + 0.5 * ( X ** 3) + np.random.normal(-3, 3, n)
m = np.random.uniform(0.3, 0.5, (n, ))
b = np.random.uniform(5, 10, (n, ))
plt.scatter(X, y, s=10)
plt.show()
Now, if I want to generate a dataset using the given formula (from Wikipedia), could you tell me what I have to change?
y = B_0 + B_1*x, B_2*x2 + B_3*x3 + ... + e
Here, x2 means x (square), x3 means x (cube), so on and e is the unobserved random error with mean zero.

There are many ways to multiply x with B, such as dot product. But I think for loop should be good enough. Just loop through element of B and x:
def generate_dataset(B, n):
# B is beta, n is number of sample
e = np.random.normal(-3, 3, n)
X = 2 - 3 * np.random.normal(0, 1, n)
y = 0
for i in range(len(B)):
y += B[i] * X**i
y += e
return X, y
def plot_dataset(X, y):
#m = np.random.uniform(0.3, 0.5, (n, )) # not sure why you need this
#b = np.random.uniform(5, 10, (n, )) # not sure why you need this
plt.scatter(X, y, s=10)
plt.show()
n = 500
B = [0, 1, -2, 0.5] # [beta0, beta1, beta2, beta3]
X, y = generate_dataset(B, 500)
plot_dataset(X, y)

Numerical radial derivative of function evaluated on Cartesian grid

I have a radially symmetric function evaluated on a 3D Cartesian grid. How can I numerically calculate the radial derivative of the function?
For a simple example (spherical Gaussian), calculate derivatives df/dx, df/dy and df/dz:
# Parameters
start = 0
end = 5
n = 20
# Variables
x = np.linspace(start, end, num=n)
y = np.linspace(start, end, num=n)
z = np.linspace(start, end, num=n)
dx = (end - start) / n
dy = (end - start) / n
dz = (end - start) / n
x_grid, y_grid, z_grid = np.meshgrid(x, y, z)
eval_xyz = np.exp(-(x_grid ** 2 + y_grid ** 2 + z_grid ** 2))
# Allocate
df_dx = np.zeros((n, n, n))
df_dy = np.zeros((n, n, n))
df_dz = np.zeros((n, n, n))
# Calculate Cartesian gradient numerically
for x in range(eval_xyz.shape[0] - 1):
for y in range(eval_xyz.shape[1] - 1):
for z in range(eval_xyz.shape[2] - 1):
df_dx[x, y, z] = (eval_xyz[x + 1, y, z] - eval_xyz[x, y, z]) / dx
df_dy[x, y, z] = (eval_xyz[x, y + 1, z] - eval_xyz[x, y, z]) / dy
df_dz[x, y, z] = (eval_xyz[x, y, z + 1] - eval_xyz[x, y, z]) / dz
Is it then possible to easily calculate the radial derivative df/dr from the Cartesian derivatives?

The trick is to express the radial derivatives as sum of Cartesian derivatives, taking into account theta and phi at each point which can be expressed in Cartesian coordiantes as:
The code therefore becomes:
theta_val = theta(i * dx, j * dy, k * dz)
phi_val = phi(i * dx, j * dy)
df_dr[i, j, k] = df_dx[i, j, k] * np.sin(theta_val) * np.cos(phi_val) \
+ df_dy[i, j, k] * np.sin(theta_val) * np.sin(phi_val) \
+ df_dz[i, j, k] * np.cos(theta_val)
Where theta and phi are calculated carefully to deal with divide by zero
def theta(x, y, z):
if x == 0 and y == 0 and z == 0:
return 0
elif z == 0:
return np.pi / 2
elif x == 0 and y == 0:
return 0
else:
return np.arctan(np.sqrt(x ** 2 + y ** 2) / z)
def phi(x, y):
if x == 0 and y == 0:
return 0
elif x == 0:
return np.pi / 2
elif y == 0:
return 0
else:
return math.atan2(y, x)

Your own answer is a step in the right direction, but there are some issues both in the answer and in the code generating the Cartesian derivatives.
These lines have a problem:
x = np.linspace(start, end, num=n)
dx = (end - start) / n
The step size is actually (end-start)/(n-1).
Here:
x_grid, y_grid, z_grid = np.meshgrid(x, y, z)
df_dx[x, y, z] = (eval_xyz[x + 1, y, z] - eval_xyz[x, y, z]) / dx
you fell in the trap of meshgrid's default setting: meshgrid(np.arange(n1), np.arange(n2)) will return arrays in the shape (n2, n1) unless you add the parameter indexing='ij'. Because you have size n in all dimensions, you will not get indexing errors to alert you, but you might be spending a lot of time trying to debug why the numbers make no sense.
When you manipulate multidimensional arrays, it's a good idea to set the sizes in different directions to slightly different values, so that you can easily check that the array shapes are what you want them to be.
Also, you should generally evaluate the derivative as (f[i+1]-f[i-1])/(2*dx), which is correct up to the second order in x.
for x in range(eval_xyz.shape[0] - 1):
for y in range(eval_xyz.shape[1] - 1):
for z in range(eval_xyz.shape[2] - 1):
When working with numpy, you should always try to vectorize operations rather than writing out for loops that potentially need to iterate over thousands of elements.
Here is code that calculates the Cartesian derivative and then the radial derivative.
import numpy as np
def get_cartesian_gradient(f, xyzsteps):
"""For f shape (nx, ny, nz), return gradient as (3, nx, ny, nz) shape.
xyzsteps is a (3,) array.
Note: edge points of the gradient array are set to NaN.
(Exercise for the reader to implement those).
"""
fshape = f.shape
grad = np.full((3,) + fshape, np.nan, dtype=np.float64)
sl, sm, sr = slice(0, -2), slice(1, -1), slice(2, None)
# Note: multiplying is faster than dividing.
grad[0, sm, sm, sm] = (f[sr, sm, sm] - f[sl, sm, sm]) * (0.5/xyzsteps[0])
grad[1, sm, sm, sm] = (f[sm, sr, sm] - f[sm, sl, sm]) * (0.5/xyzsteps[1])
grad[2, sm, sm, sm] = (f[sm, sm, sr] - f[sm, sm, sl]) * (0.5/xyzsteps[2])
return grad
def get_dfdr_from_cartesian(grad, x1s, y1s, z1s):
"""Return df/dr array from gradient(f).
grad.shape must be (3, nx, ny, nz)
return shape (nx, ny, nz).
"""
_, nx, ny, nz = grad.shape
# we need sin(theta), cos(theta), sin(phi), and cos(phi)
# rxy: shape (nx, ny, 1)
rxy = np.sqrt(x1s.reshape(-1, 1, 1)**2 + y1s.reshape(1, -1, 1)**2)
# r: shape (nx, ny, nz)
r = np.sqrt(rxy**2 + z1s.reshape(1, 1, -1)**2)
# change zeros to NaN
r = np.where(r==0, np.nan, r)
rxy = np.where(rxy==0, np.nan, rxy)
cos_theta = z1s.reshape(1, 1, -1) / r
sin_theta = rxy / r
cos_phi = x1s.reshape(-1, 1, 1) / rxy
sin_phi = y1s.reshape(1, -1, 1) / rxy
# and the derivative
dfdr = (grad[0]*cos_phi + grad[1]*sin_phi)*sin_theta + grad[2]*cos_theta
return dfdr
x1s = np.linspace(-1, 1, 19)
y1s = np.linspace(-1, 1, 21)
z1s = np.linspace(-1, 1, 23)
xs, ys, zs = np.meshgrid(x1s, y1s, z1s, indexing='ij')
xyzsteps = [x1s[1]-x1s[0], y1s[1]-y1s[0], z1s[1]-z1s[0]]
def func(x, y, z):
return x**2 + y**2 + z**2
def dfdr_analytical(x, y, z):
r = np.sqrt(x**2 + y**2 + z**2)
return 2*r
# grad has shape (3, nx, ny, nz)
grad = get_cartesian_gradient(func(xs, ys, zs), xyzsteps)
dfdr = get_dfdr_from_cartesian(grad, x1s, y1s, z1s)
# test
diff = dfdr - dfdr_analytical(xs, ys, zs)
assert np.nanmax(np.abs(diff)) < 1e-14
Note that I've chosen to return NaN values for points on the z-axis, because df/dr is not defined there unless f(x,y,z) is rotationally symmetric around the z-axis and has df/dr=0 in all directions. This is something that is not guaranteed for an arbitrary dataset.
The reason for replacing zeros in the denominators by np.nan using np.where is because dividing by zero will give warning messages, whereas dividing by nan won't.

Think Python exercise 6.1

This is an exercise 6.1 in the book "Think Python". The question is to find the print result.
This is what I can get so far.
x = 1, y = 2
bring to a(x, y), return 4
b(z), return z**2 + z
I couldn't find the valve z from c(x, y, z) function.
def b(z):
prod = a(z, z)
print(z, prod)
return prod
def a(x, y):
x = x + 1
return x * y
def c(x, y, z):
total = x + y + z
square = b(total)**2
return square
x = 1
y = x + 1
print(c(x, y+3, x+y))

The x and y in def a(x, y): are not the same x and y defined elsewhere in the script. It might as well say def a(j, k):. When you see prod = a(z, z), you need to know what the value of z is, and then go to the definition of def a(j, k): and think j = z and k = z.
If we just tell you what the output is, then you wouldn't learn to "Think Python"

In Sympy, how do I define a generic function like f(x), such that sympy.diff(f(x), x) returns f' rather than 0.

I am trying to take derivatives of this function
x, y, z, P, k, q = sp.symbols('x y z P k q')
expr = sp.exp(-sp.I*(P+k/(2*q)*(x**2 + y**2)))
where P and q are functions of z. How can I define P and q such that sp.diff(P, z) returns P' rather than 0?

From what you wrote, sympy can't know P and q are functions of z, can it? So it's treating them as constants - just like all the other variables except z. Your expression does not mention z at all, so it is all a constant expression - and the derivation of a constant is 0, no exceptions.
Make sure sympy knows P and q are functions of z. And obviously, it matters what those functions are - you can't just leave them blank. A square differentiates differently than a square root. If you don't know, sympy will do the best it can:
x, y, z, k = sp.symbols('x y z k')
P = sp.Function('P')
q = sp.Function('q')
expr = sp.exp(-sp.I*(P(z)+k/(2*q(z))*(x**2 + y**2)))
sp.diff(expr, z)
# => -I*(-k*(x**2 + y**2)*Derivative(q(z), z)/(2*q(z)**2) + Derivative(P(z), z))*
# exp(-I*(k*(x**2 + y**2)/(2*q(z)) + P(z)))
but if you do know, it can calculate it exactly:
x, y, z, k = sp.symbols('x y z k')
P = sp.Lambda(z, z * z)
q = sp.Lambda(z, sp.sqrt(z))
expr = sp.exp(-sp.I*(P(z)+k/(2*q(z))*(x**2 + y**2)))
sp.diff(expr, z)
# => -I*(-k*(x**2 + y**2)/(4*z**(3/2)) + 2*z)*
# exp(-I*(k*(x**2 + y**2)/(2*sqrt(z)) + z**2))
Similarly, I don't think you can differentiate P, but this works:
sp.diff(P(z), z)
# => 2*z

You can use idiff to get a fragile result with unevaluated derivatives: dPdz = idiff(expr, (P, q), z). It is fragile in the sense that dPdz.doit() will give 0 because there is no explicit dependence on z for q.
>>> idiff(expr,(P,q),z)
k*(x**2 + y**2)*Derivative(q, z)/(2*q**2)
>>> _.doit()
0

Categories