We Keep Coding

Function does not return the correct value unless I include a print statement

I am working through translating some Python code which makes use of a golden search algorithm to find a minimizer of a function. The issue is that the algorithm does not return the correct minimizer unless I uncomment both print functions, print('in gold', obj(c,*args), yc, 'in gold') and print('in gold', obj(d,*args), yd, 'in gold'). This is exclusive to Python as I have translated this exact algorithm to Fortran and it works beautifully.
def optimizer(obj, a, b, args=(), tol=1e-6):
""" golden section search optimizer
Args:
obj (callable): 1d function to optimize over
a (double): minimum of starting bracket
b (double): maximum of starting bracket
args (tuple): additional arguments to the objective function
tol (double,optional): tolerance
Returns:
(float): optimization result
"""
inv_phi = (np.sqrt(5) - 1) / 2 # 1/phi
inv_phi_sq = (3 - np.sqrt(5)) / 2 # 1/phi^2
# a. distance
dist = b - a
if dist <= tol:
return (a+b)/2
# b. number of iterations
n = int(np.ceil(np.log(tol/dist)/np.log(inv_phi)))
# c. potential new mid-points
c = a + inv_phi_sq * dist
d = a + inv_phi * dist
yc = obj(c,*args)
#print('in gold', obj(c,*args), yc, 'in gold')
yd = obj(d,*args)
#print('in gold', obj(d,*args), yd, 'in gold')
# d. loop
for _ in range(n-1):
if yc < yd:
b = d
d = c
yd = yc
dist = inv_phi*dist
c = a + inv_phi_sq * dist
yc = obj(c,*args)
else:
a = c
c = d
yc = yd
dist = inv_phi*dist
d = a + inv_phi * dist
yd = obj(d,*args)
# e. return
if yc < yd:
return (a+d)/2
else:
return (c+b)/2
Normally, I would just keep the print statements in there, but this is called throughout a larger set of code that I am trying to debug, making the output very costly to sort through. I am using VScode if that helps.
The objective function (obj) is obj_last_period, which is defined as:
def obj_last_period(d,x,par):
""" objective function in last period """
# implied consumption (rest)
c = x-d
return -utility.func(c, d, par)
where utility.func(c, d, par) is defined as:
def func(c, d, par):
return func_nopar(c, d, par.d_ubar, par.alpha, par.rho)
where func_nopar is defined as:
def func_nopar(c, d, d_ubar, alpha, rho):
dtot = d + d_ubar
c_total = c**alpha*dtot**(1.0-alpha)
return c_total**(1-rho)/(1-rho)
par.d_ubar, par.alpha, and par.rho are all global parameters, which have values of 0.01, 0.9, and 2, respectively.
As for sample output, for a value of x = 1.1595155892092217, d_low =1e-08, d_high = 1.1595155892092217, the object is called as
d_adj = golden_section_search.optimizer(obj_last_period,d_low,d_high,args=(x,par),tol=par.tol)
and returns a value of d_adj = 0.7166200494040755 (i.e., the minimizer) with no print statements (the associated value of the objective function is 2.1488103454406895 ), which is not the minimum (it actually happens to be the initial value of "c" on line 25 in the def for optimizer, which makes me think the program exits early for some reason?)
When I uncomment the print statements, the program returns d_adj = 0.10695155964918086 (value of objective is 1.1835203405861294), which is the correct minimum. The correct minimum corresponds to the results with my Fortran code so it has to be something specific to Python. I am not an experienced Python user and have really been learning as I go along translating everything into Fortran.
The source code is found at https://github.com/NumEconCopenhagen/ConsumptionSavingNotebooks/tree/master/02.%20DurableConsumptionModel
The specific part of the code that is causing me trouble is in the file "Last_Period.py". I figured sharing the link is likely easier, but if it is easier to just paste all the separate .py files I'm happy to do so!

How to use a "while loop" to iterate in a non-sequential way and update limits of a range?

I'm simplifying an engineering problem as much as possible for this question:
I have a working code like this:
import numpy as np
# FUNCTION DEFINITION
def Calculations(a, b): # a function defined to work based on 2 arguments, a and b
A = a * b - a
B = a * b - b
d = A - B
return(A, B, d, a, b)
# STORE LIST CREATION
A_list = []
B_list = []
d_list = []
a_list = []
b_list = [] # I will need this list later
# 1st sequential iteration in a for loop
length = np.arange(60, 62.5, 0.5)
for l in length:
lower = 50 # this is what I want the program to update based on d
upper = 70.5 # this is what I want the program to update based on d
step = 0.5
width = np.arange(lower, upper, step)
# 2nd for loop, but here I wouldn't like a sequential iteration
for w in width:
A_list.append(Calculations(l, w)[0])
B_list.append(Calculations(l, w)[1])
d_list.append(Calculations(l, w)[2])
a_list.append(Calculations(l, w)[3])
b_list.append(Calculations(l, w)[4])
print(A_list, " \n")
print(B_list, " \n")
print(d_list, " \n")
print(a_list, " \n")
print(b_list, " \n")
This is the way I have it now, but not how I want it to work.
Here, the program iterates each time through the values of length(l) in a sequential manner, meaning it evaluates everything for l=60, then for l=60.5 and so on... this is ok, but then, for l=60 it evaluates first for w=50, then for w=50.5 and so on...
What I want is that, for l=60 he evaluates for any random value (let's call this n) between the 50 (lower) and 70.5 (upper) with a step of 0.5 (step), he will then find a particular d as one of the "returned" results, if the d is negative then the n he used is the new upper, if d is positive that n is the new lower, and he will continue to do this until d is zero.
I will keep trying to figure it out by myself, but any help would be appreciated.
PD:
As I said this example is a simplification of my real problem, as side questions I would like to ask:
The real condition of while loop to break is not when d is zero, but the closest possible to zero, or phrased in other way, the min() of the abs() values composing the d_list. I tried something like:
for value in d_list:
if value = min(abs(d_list)):
print(A_list, " \n")
print(B_list, " \n")
print(d_list, " \n")
print(a_list, " \n")
print(b_list, " \n")
but that's not correct.
I don't want to use a conditions such as if d < 0.2 because sometimes I will get d's like 0.6 and that may be ok, neither do I want a condition like if d < 1 because then if for example d = 0.005 I would get a lot of d's before that, satisfying the condition of being < 1, but I only want one for each l.
I also need to find the associated values in the returned lists, for that specific d
EDIT
I made a mistake earlier in the conditions for new upper and lower based on the obtained value of d, I fixed that.
Also, I tried solving the problem like this:
length = np.arange(60, 62.5, 0.5)
for l in length:
lower_w = 59.5 # this is what I want the program to update based on d
upper_w = 63 # this is what I want the program to update based on d
step = 0.5
width = np.arange(lower_w, upper_w, step)
np.random.shuffle(width)
for w in width:
while lower_w < w < upper_w:
A_list.append(Calculations(l,w)[0])
B_list.append(Calculations(l,w)[1])
d_list.append(Calculations(l,w)[2])
a_list.append(Calculations(l,w)[3])
b_list.append(Calculations(l,w)[4])
for element in d_list:
if element < 0:
upper = w
else:
lower = w
if abs(element) < 1 :
break
But the while loop does not get to break...

Use np.random.shuffle to pick the elements of width in a random order:
width = np.arange(lower, upper, step)
np.random.shuffle(width)
But here you don't really want the second loop, just pick one element from it at random, so use np.range.choice(width):
length = np.arange(60, 62.5, 0.5)
lower = 50 # this is what I want the program to update based on d
upper = 70.5 # this is what I want the program to update based on d
step = 0.5
for l in length:
width = np.arange(lower, upper, step)
if len(width) == 0:
width = [lower]
w = np.random.choice(width)
(A, B, d, a, b) = Calculations(l, w)
A_list.append(A)
B_list.append(B)
d_list.append(d)
a_list.append(l)
b_list.append(w)
if d < 0:
lower = w
elif d:
upper = w
No need to pass a and b in the return of the Calculations function, you can just append the original parameters to a_list and b_list.
Note that you will run into an error if your lower and upper bound are identical, because the list will just be empty, so you need to fill in the list with a bound if it returns [].

Could I get a clarification to this Python code below?

I'm a beginner to Python and I'm trying to calculate the angles (-26.6 &18.4) for this figure below and so on for the rest of the squares by using Python code.
I have found the code below and I'm trying to understand very well. How could it work here? Any clarification, please?
Python Code:
def computeDegree(a,b,c):
babc = (a[0]-b[0])*(c[0]-b[0])+(a[1]-b[1])*(c[1]-b[1])
norm_ba = math.sqrt((a[0]-b[0])**2 + (a[1]-b[1])**2)
norm_bc = math.sqrt((c[0]-b[0])**2 + (c[1]-b[1])**2)
norm_babc = norm_ba * norm_bc
radian = math.acos(babc/norm_babc)
degree = math.degrees(radian)
return round(degree, 1)
def funcAngle(p, s, sn):
a = (s[0]-p[0], s[1]-p[1])
b = (sn[0]-p[0], sn[1]-p[1])
c = a[0] * b[1] - a[1] * b[0]
if p != sn:
d = computeDegree(s, p, sn)
else:
d = 0
if c > 0:
result = d
elif c < 0:
result = -d
elif c == 0:
result = 0
return result
p = (1,4)
s = (2,2)
listSn= ((1,2),(2,3),(3,2),(2,1))
for sn in listSn:
func(p,s,sn)
The results
I expected to get the angles in the picture such as -26.6, 18.4 ...

Essentially, this uses the definition of dot products to solve for the angle. You can read more it at this link (also where I found these images).
To solve for the angle you first need to convert your 3 input points into two vectors.
# Vector from b to a
# BA = (a[0] - b[0], a[1] - b[1])
BA = a - b
# Vector from b to c
# BC = (a[0] - c[0], a[1] - c[1])
BC = c - b
Using the two vectors you can then find the angle between them by first finding the value of the dot product with the second formula.
# babc = (a[0]-b[0])*(c[0]-b[0])+(a[1]-b[1])*(c[1]-b[1])
dot_product = BA[0] * BC[0] + BA[1] * BC[1]
Then by going back to the first definition, you can divide off the lengths of the two input vectors and the resulting value should be the cosine of the angle between the vectors. It may be hard to read with the array notation but its just using the Pythagoras theorem.
# Length/magnitude of vector BA
# norm_ba = math.sqrt((a[0]-b[0])**2 + (a[1]-b[1])**2)
length_ba = math.sqrt(BA[0]**2 + BA[1]**2)
# Length/magnitude of vector BC
# norm_bc = math.sqrt((c[0]-b[0])**2 + (c[1]-b[1])**2)
length_bc = math.sqrt(BC[0]**2 + BC[1]**2)
# Then using acos (essentially inverse of cosine), you can get the angle
# radian = math.acos(babc/norm_babc)
angle = Math.acos(dot_product / (length_ba * length_bc))
Most of the other stuff is just there to catch cases where the program might accidentally try to divide by zero. Hopefully this helps to explain why it looks the way it does.
Edit: I answered this question because I was bored and didn't see harm in explaining the math behind that code, however in the future try to avoid asking questions like 'how does this code work' in the future.

Let's start with funcAngle since it calls computeDegree later.
The first thing it does is define a as a two item tuple. A lot of this code seems to use two item tuples, with the two parts referenced by v[0] and v[1] or similar. These are almost certainly two dimensional vectors of some sort.
I'm going to write these as 𝐯 for the vector and vₓ and vᵧ since they're probably the two components.
[don't look too closely at that second subscript, it's totally a y and not a gamma...]
a is the vector difference between s and p: i.e.
a = (s[0]-p[0], s[1]-p[1])
is aₓ=sₓ-pₓ and aᵧ=sᵧ-pᵧ; or just 𝐚=𝐬-𝐩 in vector.
b = (sn[0]-p[0], sn[1]-p[1])
again; 𝐛=𝐬𝐧-𝐩
c = a[0] * b[1] - a[1] * b[0]
c=aₓbᵧ-aᵧbₓ; c is the cross product of 𝐚 and 𝐛 (and is just a number)
if p != sn:
d = computeDegree(s, p, sn)
else:
d = 0
I'd take the above in reverse: if 𝐩 and 𝐬𝐧 are the same, then we already know the angle between them is zero (and it's possible the algorithm fails badly) so don't compute it. Otherwise, compute the angle (we'll look at that later).
if c > 0:
result = d
elif c < 0:
result = -d
elif c == 0:
result = 0
If c is pointing in the normal direction (via the left hand rule? right hand rule? can't remember) that's fine: if it isn't, we need to negate the angle, apparently.
return result
Pass the number we've just worked out to some other code.
You can probably invoke this code by adding something like:
print (funcangle((1,0),(0,1),(2,2))
at the end and running it. (Haven't actually tested these numbers)
So this function works out a and b to get c; all just to negate the angle if it's pointing the wrong way. None of these variables are actually passed to computeDegree.
so, computeDegree():
def computeDegree(a,b,c):
First thing to note is that the variables from before have been renamed. funcAngle passed s, p and sn, but now they're called a, b and c. And the note the order they're passed in isn't the same as they're passed to funcAngle, which is nasty and confusing.
babc = (a[0]-b[0])*(c[0]-b[0])+(a[1]-b[1])*(c[1]-b[1])
babc = (aₓ-bₓ)(cₓ-bₓ)+(aᵧ-bᵧ)(cᵧ-bᵧ)
If 𝐚' and 𝐜' are 𝐚-𝐛 and 𝐜-𝐛 respectively, this is just
a'ₓc'ₓ+a'ᵧc'ᵧ, or the dot product of 𝐚' and 𝐜'.
norm_ba = math.sqrt((a[0]-b[0])**2 + (a[1]-b[1])**2)
norm_bc = math.sqrt((c[0]-b[0])**2 + (c[1]-b[1])**2)
norm_ba = √[(aₓ-bₓ)² + (aᵧ-bᵧ)²] (and norm_bc likewise).
This looks like the length of the hypotenuse of 𝐚' (and 𝐜' respectively)
norm_babc = norm_ba * norm_bc
which we then multiply together
radian = math.acos(babc/norm_babc)
We use the arccosine (inverse cosine, cos^-1) function, with the length of those multiplied hypotenuses as the hypotenuse and that dot product as the adjacent length...
degree = math.degrees(radian)
return round(degree, 1)
but that's in radians, so we convert to degrees and round it for nice formatting.
Ok, so now it's in maths, rather than Python, but that's still not very easy to understand.
(sidenote: this is why descriptive variable names and documentation is everyone's friend!)

Metropolis-Hastings accept-reject implementation

I've been reading about the Metropolis-Hastings (MH) algorithm. Theoretically, I understood how the algorithm works. Now, I am trying to implement the MH algorithm using python.
I came across the following notebook. It suits exactly my problem since I want to fit my data by a straight line taking into consideration the measurement errors on my data. I am going to paste the code I am finding difficulties to understand:
# initial m, b
m,b = 2, 0
# step sizes
mstep, bstep = 0.1, 10.
# how many steps?
nsteps = 10000
chain = []
probs = []
naccept = 0
print 'Running MH for', nsteps, 'steps'
# First point:
L_old = straight_line_log_likelihood(x, y, sigmay, m, b)
p_old = straight_line_log_prior(m, b)
prob_old = np.exp(L_old + p_old)
for i in range(nsteps):
# step
mnew = m + np.random.normal() * mstep
bnew = b + np.random.normal() * bstep
# evaluate probabilities
# prob_new = straight_line_posterior(x, y, sigmay, mnew, bnew)
L_new = straight_line_log_likelihood(x, y, sigmay, mnew, bnew)
p_new = straight_line_log_prior(mnew, bnew)
prob_new = np.exp(L_new + p_new)
if (prob_new / prob_old > np.random.uniform()):
# accept
m = mnew
b = bnew
L_old = L_new
p_old = p_new
prob_old = prob_new
naccept += 1
else:
# Stay where we are; m,b stay the same, and we append them
# to the chain below.
pass
chain.append((b,m))
probs.append((L_old,p_old))
print 'Acceptance fraction:', naccept/float(nsteps)
The code is simple and easy, but I have difficulties in understanding how the MH is being implemented.
My question is in the chain.append (the third line from the bottom). The author is appending m and b whether they were accepted or rejected. Why? Shouldn't he append only the accepted points?

The following R code demonstrates why it is important to capture the rejected case:
# 20 samples from 0 or 1. 1 has an 80% probability of being chosen.
the.population <- sample(c(0,1), 20, replace = TRUE, prob=c(0.2, 0.8))
# Create a new sample that only catches changes
the.sample <- c(the.population[1])
# Loop though the.population,
# but only copy the.population to the.sample if the value changes
for( i in 2:length(the.population))
{
if(the.population[i] != the.population[i-1])
the.sample <- append(the.sample, the.population[i])
}
When this code runs, the.population gets 20 values, for example:
0 1 1 1 1 1 1 1 1 0 1 1 1 1 0 0 1 1 1 1
The probability of a 1 in this population is 16/20 or 0.8. Exactly the probability we expected...
The sample, on the other hand, which only records changes, looks like this:
0 1 0 1 0 1
The probability of a 1 in the sample is 3/6 or 0.5.
We are trying to build a distribution, rejecting the new values means that the old values are more likely than the new values. That needs to be captured so our distribution is correct.

From a quick reading of the algorithm description: When a candidate is rejected, it still counts as a step, but the value is the same as the old step. I.e. b, m are appended either way, but they only get updated (to bnew, mnew) in the case where the candidate is accepted.

Translating a function integrating another function from BASIC into Python

The book Calculus and Pizza by Clifford Pickover has a few code examples here and there, all written in some dialect of BASIC.
I wrote a Python version of the code example covering integration. His BASIC example goes like:
10 REM Integration
20 DEF FNY(X) = X*X*X
30 A = 0
40 B = 1
50 N = 10
55 R = 0
60 H = (B-A)/N
70 FOR X = A TO B - H/2 STEP H
80 R = R + FNY(X)
90 NEXT X
100 R = R * H
110 PRINT *INTEGRATION ESTIMATE*: R
I changed a few things here and there, allowing the user to specify the interval over which to take the integral, specify the function to be integrated as a lambda, and so forth. I knew right off the bat that the for loop wouldn't work as I have written it below. I'm just wondering if there's some direct or idiomatic translation of the BASIC for to a Python for.
def simpleintegration():
f = eval(input("specify the function as a lambda\n:%"))
a = int(input("take the integral from x = a = ...\n:%"))
b = int(input("to x = b = ...\n:%"))
n = 10
r = 0
h = (b-a)/n
for x in range(a,b-h/2,h):
r = r + f(x)
r = r * h
print(r)

Your translation isn't far off. The only difference between the for loop in other languages and Python's "loop-over-a-range" pattern is that the "stop" value is usually inclusive in other languages, but is exclusive in Python.
Thus, in most other languages, a loop including a and b looks like
for i = a to b step c
' Do stuff
next i
In Python, it would be
for i in range(a, b + 1, c):
# Do stuff

The formula is computing the Riemann sums using the values at the left end of the subdivision intervals. Thus the last used value for X should be B-H.
Due to floating point errors, stepping from A by H can give a last value that is off by some small amount, thus B-H is not a good bound (in the BASIC code) and B-H/2 is used to stop before X reaches B.
The Python code should work in the presented form for the same reasons, since the bound B-H/2 is unreachable, thus the range should stop with B-H or a value close by.
Using a slight modification you can actually compute the trapezoidal approximation, where you initialize with R=f(A)/2, step X from A+H to including B-H adding f(X) to R and then finish by adding f(B)/2 (which could already be done in the initialization). As before, the approximation of the integral is then R*H.

You can do as below, just changing iteration of 'i' in for loop.
def simpleintegration():
f = eval(input("specify the function as a lambda\n:%"))
a = int(input("take the integral from x = a = ...\n:%"))
b = int(input("to x = b = ...\n:%"))
n = 10
r = 0
h = (b-a)/n
for x = a to b-h/2 step h:
r = r + f(x)
r = r * h
print(r)