I'm stuck with one task on my learning path.
For the binomial distribution X∼Bp,n with mean μ=np and variance σ**2=np(1−p), we would like to upper bound the probability P(X≥c⋅μ) for c≥1.
Three bounds introduced:
Formulas
The task is to write three functions respectively for each of the inequalities. They must take n , p and c as inputs and return the upper bounds for P(X≥c⋅np) given by the above Markov, Chebyshev, and Chernoff inequalities as outputs.
And there is an example of IO:
Code:
print Markov(100.,0.2,1.5)
print Chebyshev(100.,0.2,1.5)
print Chernoff(100.,0.2,1.5)
Output
0.6666666666666666
0.16
0.1353352832366127
I'm completely stuck. I just can't figure out how to plug in all that math into functions (or how to think algorithmically here). If someone could help me out, that would be of great help!
p.s. and all libs are not allowed by task conditions except math.exp
Ok, let's look at what's given:
Input and derived values:
n = 100
p = 0.2
c = 1.5
m = n*p = 100 * 0.2 = 20
s2 = n*p*(1-p) = 16
s = sqrt(s2) = sqrt(16) = 4
You have multiple inequalities of the form P(X>=a*m) and you need to provide bounds for the term P(X>=c*m), so you need to think how a relates to c in all cases.
Markov inequality: P(X>=a*m) <= 1/a
You're asked to implement Markov(n,p,c) that will return the upper bound for P(X>=c*m). Since from
P(X>=a*m)
= P(X>=c*m)
it's clear that a == c, you get 1/a = 1/c. Well, that's just
def Markov(n, p, c):
return 1.0/c
>>> Markov(100,0.2,1.5)
0.6666666666666666
That was easy, wasn't it?
Chernoff inequality states that P(X>=(1+d)*m) <= exp(-d**2/(2+d)*m)
First, let's verify that if
P(X>=(1+d)*m)
= P(X>=c *m)
then
1+d = c
d = c-1
This gives us everything we need to calculate the uper bound:
def Chernoff(n, p, c):
d = c-1
m = n*p
return math.exp(-d**2/(2+d)*m)
>>> Chernoff(100,0.2,1.5)
0.1353352832366127
Chebyshev inequality bounds P(X>=m+k*s) by 1/k**2
So again, if
P(X>=c*m)
= P(X>=m+k*s)
then
c*m = m+k*s
m*(c-1) = k*s
k = m*(c-1)/s
Then it's straight forward to implement
def Chebyshev(n, p, c):
m = n*p
s = math.sqrt(n*p*(1-p))
k = m*(c-1)/s
return 1/k**2
>>> Chebyshev(100,0.2,1.5)
0.16
Related
I have the following polynomial equation that I would like to find the local minima and maxima for.
I defined the function as follows. It uses a flatten function to flatten the nested list, I'll include it for testing purposes (found it here http://rightfootin.blogspot.com/2006/09/more-on-python-flatten.html)
flatten list
from itertools import combinations
import math
def flatten(l, ltypes=(list, tuple)):
ltype = type(l)
l = list(l)
i = 0
while i < len(l):
while isinstance(l[i], ltypes):
if not l[i]:
l.pop(i)
i -= 1
break
else:
l[i:i + 1] = l[i]
i += 1
return ltype(l)
my polynomial
def poly(coefficients, factors):
#quadratic terms
constant = 1
singles = factors
products = [math.prod(c) for c in combinations(factors, 2)]
squares = [f**2 for f in factors]
sequence = flatten([constant, singles, products, squares])
z = sum([math.prod(i) for i in zip(coefficients, sequence)])
return z
The arguments it takes is a list of coefficients, for example:
coefs = [12.19764959, -1.8233151, 2.50952816,-1.56344375, 1.00003828, -1.72128301, -2.54254877, -1.20377309, 5.53510616, 2.94755653, 4.83759279, -0.85507208, -0.48007208, -3.70507208, -0.27007208]
And a list of factor or variable values:
factors = [0.4714, 0.4714, -0.4714, 0.4714]
Plug these in and it calculates the result of the polynomial. The reason I wrote it like this is because the number of variables (factors) changes from fit to fit, so I wanted to keep it flexible. I now want to find the combination of "factors" values within a certain range (let's say between -1 and 1) where the function reaches its maximum and minimum values. If the function was "hard coded" I could use scipy.optimize, but I can't figure out how to make it works as is.
Another option is a brute force grid search (which I use at the moment), but it's very slow as soon as you have more than 2 variables, especially with small step sizes. There may be no true minima/maxima where slope == 0 within the bounds, but as long as I can get the maximum and minimum values that is OK.
Ok, I figured it out. It was two really silly things:
the order of the arguments in the function had to be reversed, so that the first argument (the one I wanted to optimize for) were the "factors" or the X values, followed by the coefficients. That way an array of the same size could be used as the X0 and the coefficients could be used as args.
That wassn't enough, as the function would return an array if an array was the input. I just added a factors = list(factors) to the function itself to put it into the correct shape.
The new function:
def poly(factors, coefficients):
factors = list(factors)
#quadratic terms
constant = 1
singles = factors
products = [math.prod(c) for c in combinations(factors, 2)]
squares = [f**2 for f in factors]
sequence = flatten([constant, singles, products, squares])
z = sum([math.prod(i) for i in zip(coefficients, sequence)])
return z
And the optimization:
coefs = [4.08050532, -0.47042713, -0.08200181, -0.54184481, -0.18515675,
-0.96751856, -1.10814625, -1.7831592, 5.2763512, 2.83505438, 4.7082153,
0.22988773, 1.06488773, -0.70011227, 1.42988773]
x0 = [0.1, 0.1, 0.1, 0.1]
minimize(poly,x0 = x0, args = coefs, bounds = ((-1,1),(-1,1),(-1,1),(-1,1)))
Which returns:
fun: -1.6736636102536673
hess_inv: <4x4 LbfgsInvHessProduct with dtype=float64>
jac: array([-2.10611305e-01, 2.19138777e+00, -8.16990766e+00, -1.11022302e-07])
message: 'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'
nfev: 85
nit: 12
njev: 17
status: 0
success: True
x: array([1., -1.,1., 0.03327357])
Consider the following problem: Given a set of n intervals and a set of m floating-point numbers, determine, for each floating-point number, the subset of intervals that contain the floating-point number.
This problem has been addressed by constructing an interval tree (or called range tree or segment tree). Implementations have been done for the one-dimensional case, e.g. python's intervaltree package. Usually, these implementations consider one or few floating-point numbers, namely a small "m" above.
In my problem setting, both n and m are extremely large numbers (from solving an image processing problem). Further, I need to consider the N-dimensional intervals (called cuboid when N=3, because I was modeling human brains with the Finite Element Method). I have implemented a simple N-dimensional interval tree in python, but it run in a loop and can only take one floating-point number at a time. Can anyone help improve the implementation in terms of efficiency? You can change data structure freely.
import sys
import time
import numpy as np
# find the index of a satisfying x > a in one dimension
def find_index_smaller(a, x):
idx = np.argsort(a)
ss = np.searchsorted(a, x, sorter=idx)
res = idx[0:ss]
return res
# find the index of a satisfying x < a in one dimension
def find_index_larger(a, x):
return find_index_smaller(-a, -x)
# find the index of a satisfing amin < x < amax in one dimension
def find_intv_at(amin, amax, x):
idx = find_index_smaller(amin, x)
idx2 = find_index_larger(amax[idx], x)
res = idx[idx2]
return res
# find the index of a satisfying amin < x < amax in N dimensions
def find_intv_at_nd(amin, amax, x):
dim = amin.shape[0]
res = np.arange(amin.shape[-1])
for i in range(dim):
idx = find_intv_at(amin[i, res], amax[i, res], x[i])
res = res[idx]
return res
I also have two test examples for sanity check and performance testing:
def demo1():
print ("By default, we do a correctness test")
n_intv = 2
n_point = 2
# generate the test data
point = np.random.rand(3, n_point)
intv_min = np.random.rand(3, n_intv)
intv_max = intv_min + np.random.rand(3, n_intv)*8
print ("point ")
print (point)
print ("intv_min")
print (intv_min)
print ("intv_max")
print (intv_max)
print ("===Indexes of intervals that contain the point===")
for i in range(n_point):
print (find_intv_at_nd(intv_min,intv_max, point[:, i]))
def demo2():
print ("Performance:")
n_points=100
n_intv = 1000000
# generate the test data
points = np.random.rand(n_points, 3)*512
intv_min = np.random.rand(3, n_intv)*512
intv_max = intv_min + np.random.rand(3, n_intv)*8
print ("point.shape = "+str(points.shape))
print ("intv_min.shape = "+str(intv_min.shape))
print ("intv_max.shape = "+str(intv_max.shape))
starttime = time.time()
for point in points:
tmp = find_intv_at_nd(intv_min, intv_max, point)
print("it took this long to run {} points, with {} interva: {}".format(n_points, n_intv, time.time()-starttime))
My idea would be:
Remove np.argsort() from the algo, because the interval tree does not change, so sorting could have been done in pre-processing.
Vectorize x. The algo runs a loop for each x. It would be nice if we can get rid of the loop over x.
Any contribution would be appreciated.
I'm a beginner to Python and I'm trying to calculate the angles (-26.6 &18.4) for this figure below and so on for the rest of the squares by using Python code.
I have found the code below and I'm trying to understand very well. How could it work here? Any clarification, please?
Python Code:
def computeDegree(a,b,c):
babc = (a[0]-b[0])*(c[0]-b[0])+(a[1]-b[1])*(c[1]-b[1])
norm_ba = math.sqrt((a[0]-b[0])**2 + (a[1]-b[1])**2)
norm_bc = math.sqrt((c[0]-b[0])**2 + (c[1]-b[1])**2)
norm_babc = norm_ba * norm_bc
radian = math.acos(babc/norm_babc)
degree = math.degrees(radian)
return round(degree, 1)
def funcAngle(p, s, sn):
a = (s[0]-p[0], s[1]-p[1])
b = (sn[0]-p[0], sn[1]-p[1])
c = a[0] * b[1] - a[1] * b[0]
if p != sn:
d = computeDegree(s, p, sn)
else:
d = 0
if c > 0:
result = d
elif c < 0:
result = -d
elif c == 0:
result = 0
return result
p = (1,4)
s = (2,2)
listSn= ((1,2),(2,3),(3,2),(2,1))
for sn in listSn:
func(p,s,sn)
The results
I expected to get the angles in the picture such as -26.6, 18.4 ...
Essentially, this uses the definition of dot products to solve for the angle. You can read more it at this link (also where I found these images).
To solve for the angle you first need to convert your 3 input points into two vectors.
# Vector from b to a
# BA = (a[0] - b[0], a[1] - b[1])
BA = a - b
# Vector from b to c
# BC = (a[0] - c[0], a[1] - c[1])
BC = c - b
Using the two vectors you can then find the angle between them by first finding the value of the dot product with the second formula.
# babc = (a[0]-b[0])*(c[0]-b[0])+(a[1]-b[1])*(c[1]-b[1])
dot_product = BA[0] * BC[0] + BA[1] * BC[1]
Then by going back to the first definition, you can divide off the lengths of the two input vectors and the resulting value should be the cosine of the angle between the vectors. It may be hard to read with the array notation but its just using the Pythagoras theorem.
# Length/magnitude of vector BA
# norm_ba = math.sqrt((a[0]-b[0])**2 + (a[1]-b[1])**2)
length_ba = math.sqrt(BA[0]**2 + BA[1]**2)
# Length/magnitude of vector BC
# norm_bc = math.sqrt((c[0]-b[0])**2 + (c[1]-b[1])**2)
length_bc = math.sqrt(BC[0]**2 + BC[1]**2)
# Then using acos (essentially inverse of cosine), you can get the angle
# radian = math.acos(babc/norm_babc)
angle = Math.acos(dot_product / (length_ba * length_bc))
Most of the other stuff is just there to catch cases where the program might accidentally try to divide by zero. Hopefully this helps to explain why it looks the way it does.
Edit: I answered this question because I was bored and didn't see harm in explaining the math behind that code, however in the future try to avoid asking questions like 'how does this code work' in the future.
Let's start with funcAngle since it calls computeDegree later.
The first thing it does is define a as a two item tuple. A lot of this code seems to use two item tuples, with the two parts referenced by v[0] and v[1] or similar. These are almost certainly two dimensional vectors of some sort.
I'm going to write these as 𝐯 for the vector and vₓ and vᵧ since they're probably the two components.
[don't look too closely at that second subscript, it's totally a y and not a gamma...]
a is the vector difference between s and p: i.e.
a = (s[0]-p[0], s[1]-p[1])
is aₓ=sₓ-pₓ and aᵧ=sᵧ-pᵧ; or just 𝐚=𝐬-𝐩 in vector.
b = (sn[0]-p[0], sn[1]-p[1])
again; 𝐛=𝐬𝐧-𝐩
c = a[0] * b[1] - a[1] * b[0]
c=aₓbᵧ-aᵧbₓ; c is the cross product of 𝐚 and 𝐛 (and is just a number)
if p != sn:
d = computeDegree(s, p, sn)
else:
d = 0
I'd take the above in reverse: if 𝐩 and 𝐬𝐧 are the same, then we already know the angle between them is zero (and it's possible the algorithm fails badly) so don't compute it. Otherwise, compute the angle (we'll look at that later).
if c > 0:
result = d
elif c < 0:
result = -d
elif c == 0:
result = 0
If c is pointing in the normal direction (via the left hand rule? right hand rule? can't remember) that's fine: if it isn't, we need to negate the angle, apparently.
return result
Pass the number we've just worked out to some other code.
You can probably invoke this code by adding something like:
print (funcangle((1,0),(0,1),(2,2))
at the end and running it. (Haven't actually tested these numbers)
So this function works out a and b to get c; all just to negate the angle if it's pointing the wrong way. None of these variables are actually passed to computeDegree.
so, computeDegree():
def computeDegree(a,b,c):
First thing to note is that the variables from before have been renamed. funcAngle passed s, p and sn, but now they're called a, b and c. And the note the order they're passed in isn't the same as they're passed to funcAngle, which is nasty and confusing.
babc = (a[0]-b[0])*(c[0]-b[0])+(a[1]-b[1])*(c[1]-b[1])
babc = (aₓ-bₓ)(cₓ-bₓ)+(aᵧ-bᵧ)(cᵧ-bᵧ)
If 𝐚' and 𝐜' are 𝐚-𝐛 and 𝐜-𝐛 respectively, this is just
a'ₓc'ₓ+a'ᵧc'ᵧ, or the dot product of 𝐚' and 𝐜'.
norm_ba = math.sqrt((a[0]-b[0])**2 + (a[1]-b[1])**2)
norm_bc = math.sqrt((c[0]-b[0])**2 + (c[1]-b[1])**2)
norm_ba = √[(aₓ-bₓ)² + (aᵧ-bᵧ)²] (and norm_bc likewise).
This looks like the length of the hypotenuse of 𝐚' (and 𝐜' respectively)
norm_babc = norm_ba * norm_bc
which we then multiply together
radian = math.acos(babc/norm_babc)
We use the arccosine (inverse cosine, cos^-1) function, with the length of those multiplied hypotenuses as the hypotenuse and that dot product as the adjacent length...
degree = math.degrees(radian)
return round(degree, 1)
but that's in radians, so we convert to degrees and round it for nice formatting.
Ok, so now it's in maths, rather than Python, but that's still not very easy to understand.
(sidenote: this is why descriptive variable names and documentation is everyone's friend!)
I am using sympy to solve some equations and I am running into a problem. I have this issue with many equations but I will illustrate with an example. I have an equation with multiple variables and I want to solve this equation in terms of all variables but one is excluded. For instance the equation 0 = 2^n*(2-a) - b + 1. Here there are three variables a, b and n. I want to get the values for a and b not in terms of n so the a and b may not contain n.
2^n*(2-a) - b + 1 = 0
# Since we don't want to solve in terms of n we know that (2 - a)
# has to be zero and -b + 1 has to be zero.
2 - a = 0
a = 2
-b + 1 = 0
b = 1
I want sympy to do this. Maybe I'm just not looking at the right documentation but I have found no way to do this. When I use solve and instruct it to solve for symbols a and b sympy returns to me a single solution where a is defined in terms of n and b. I assume this means I am free to choose b and n, However I don't want to fix n to a specific value I want n to still be a variable.
Code:
import sympy
n = sympy.var("n", integer = True)
a = sympy.var("a")
b = sympy.var("b")
f = 2**n*(2-a) - b + 1
solutions = sympy.solve(f, [a,b], dict = True)
# this will return: "[{a: 2**(-n)*(2**(n + 1) - b + 1)}]".
# A single solution where b and n are free variables.
# However this means I have to choose an n I don't want
# to that I want it to hold for any n.
I really hope someone can help me. I have been searching google for hours now...
Ok, here's what I came up with. This seems to solve the type of equations you're looking for. I've provided some tests as well. Of course, this code is rough and can be easily caused to fail, so i'd take it more as a starting point than a complete solution
import sympy
n = sympy.Symbol('n')
a = sympy.Symbol('a')
b = sympy.Symbol('b')
c = sympy.Symbol('c')
d = sympy.Symbol('d')
e = sympy.Symbol('e')
f = sympy.sympify(2**n*(2-a) - b + 1)
g = sympy.sympify(2**n*(2-a) -2**(n-1)*(c+5) - b + 1)
h = sympy.sympify(2**n*(2-a) -2**(n-1)*(e-1) +(c-3)*9**n - b + 1)
i = sympy.sympify(2**n*(2-a) -2**(n-1)*(e+4) +(c-3)*9**n - b + 1 + (d+2)*9**(n+2))
def rewrite(expr):
if expr.is_Add:
return sympy.Add(*[rewrite(f) for f in expr.args])
if expr.is_Mul:
return sympy.Mul(*[rewrite(f) for f in expr.args])
if expr.is_Pow:
if expr.args[0].is_Number:
if expr.args[1].is_Symbol:
return expr
elif expr.args[1].is_Add:
base = expr.args[0]
power = sympy.solve(expr.args[1])
sym = expr.args[1].free_symbols.pop()
return sympy.Mul(sympy.Pow(base,-power[0]), sympy.Pow(base,sym))
else:
return expr
else:
return expr
else:
return expr
def my_solve(expr):
if not expr.is_Add:
return None
consts_list = []
equations_list = []
for arg in expr.args:
if not sympy.Symbol('n') in arg.free_symbols:
consts_list.append(arg)
elif arg.is_Mul:
coeff_list = []
for nested_arg in arg.args:
if not sympy.Symbol('n') in nested_arg.free_symbols:
coeff_list.append(nested_arg)
equations_list.append(sympy.Mul(*coeff_list))
equations_list.append(sympy.Add(*consts_list))
results = {}
for eq in equations_list:
var_name = eq.free_symbols.pop()
val = sympy.solve(eq)[0]
results[var_name] = val
return results
print(my_solve(rewrite(f)))
print(my_solve(rewrite(g)))
print(my_solve(rewrite(h)))
print(my_solve(rewrite(i)))
The book Calculus and Pizza by Clifford Pickover has a few code examples here and there, all written in some dialect of BASIC.
I wrote a Python version of the code example covering integration. His BASIC example goes like:
10 REM Integration
20 DEF FNY(X) = X*X*X
30 A = 0
40 B = 1
50 N = 10
55 R = 0
60 H = (B-A)/N
70 FOR X = A TO B - H/2 STEP H
80 R = R + FNY(X)
90 NEXT X
100 R = R * H
110 PRINT *INTEGRATION ESTIMATE*: R
I changed a few things here and there, allowing the user to specify the interval over which to take the integral, specify the function to be integrated as a lambda, and so forth. I knew right off the bat that the for loop wouldn't work as I have written it below. I'm just wondering if there's some direct or idiomatic translation of the BASIC for to a Python for.
def simpleintegration():
f = eval(input("specify the function as a lambda\n:%"))
a = int(input("take the integral from x = a = ...\n:%"))
b = int(input("to x = b = ...\n:%"))
n = 10
r = 0
h = (b-a)/n
for x in range(a,b-h/2,h):
r = r + f(x)
r = r * h
print(r)
Your translation isn't far off. The only difference between the for loop in other languages and Python's "loop-over-a-range" pattern is that the "stop" value is usually inclusive in other languages, but is exclusive in Python.
Thus, in most other languages, a loop including a and b looks like
for i = a to b step c
' Do stuff
next i
In Python, it would be
for i in range(a, b + 1, c):
# Do stuff
The formula is computing the Riemann sums using the values at the left end of the subdivision intervals. Thus the last used value for X should be B-H.
Due to floating point errors, stepping from A by H can give a last value that is off by some small amount, thus B-H is not a good bound (in the BASIC code) and B-H/2 is used to stop before X reaches B.
The Python code should work in the presented form for the same reasons, since the bound B-H/2 is unreachable, thus the range should stop with B-H or a value close by.
Using a slight modification you can actually compute the trapezoidal approximation, where you initialize with R=f(A)/2, step X from A+H to including B-H adding f(X) to R and then finish by adding f(B)/2 (which could already be done in the initialization). As before, the approximation of the integral is then R*H.
You can do as below, just changing iteration of 'i' in for loop.
def simpleintegration():
f = eval(input("specify the function as a lambda\n:%"))
a = int(input("take the integral from x = a = ...\n:%"))
b = int(input("to x = b = ...\n:%"))
n = 10
r = 0
h = (b-a)/n
for x = a to b-h/2 step h:
r = r + f(x)
r = r * h
print(r)