We Keep Coding

Combine scipy.root and Jax Jacobian

I am having trouble using the Jacobian from JAX with scipy.root. In the below example, the root works without the Jacobian, while it fails with the Jacobian. Any ideas on what I need to rewrite in order to get the code below working with the Jacobian?
from jax import jacfwd
from scipy.optimize import root
import numpy as np
def objectFunction(valuesEndo, varNamesEndo, valuesExo, varNamesExo, equations):
for i in range(len(varNamesExo)):
exec("%s = %.10f" %(varNamesExo[i], valuesExo[i]))
for i in range(len(varNamesEndo)):
exec("%s = %.10f" %(varNamesEndo[i], valuesEndo[i]))
equationVector = np.zeros(len(equations))
for i in range(len(equations)):
exec('equationVector[%d] = eval(equations[%d])' %(i, i))
return equationVector
varNamesEndo = ['x', 'y']
valuesEndoInitialGuess = [1., 1.]
varNamesExo = ['a', 'b']
valuesExo = [1., 1.]
equations = ['a*x+b*y**2-4',
'np.exp(x) + x*y - 3']
equations = ['a*x**2 + b*y**2',
'a*x**2 - b*y**2']
# Without Jacobian
sol1 = root(fun=objectFunction,
x0=valuesEndoInitialGuess,
args=(varNamesEndo, valuesExo, varNamesExo, equations))
#----> Works
# With Jacobian
jac = jacfwd(objectFunction)
sol2 = root(fun=objectFunction,
x0=valuesEndoInitialGuess,
args=(varNamesEndo, valuesExo, varNamesExo, equations),
jac=jac)
#----> Not woring
At least there seems to be problems with the line
for i in range(len(varNamesEndo)):
exec("%s = %.10f" %(varNamesEndo[i], valuesEndo[i]))

There are two issues:
to perform automatic differentiation, JAX relies on replacing values with tracers. This means your approach of printing and evaluating the string representation of the value will not work.
additionally, you are attempting to assign traced values to a standard numpy array. You should use a JAX array instead, as it knows how to handle traced values.
With this in mind, you can rewrite your function this way and it should work, so long as your equations only use Python arithmetic operations and jax functions (not things like np.exp):
import jax.numpy as jnp
def objectFunction(valuesEndo, varNamesEndo, valuesExo, varNamesExo, equations):
for i in range(len(varNamesExo)):
exec("%s = valuesExo[%d]" %(varNamesExo[i], i))
for i in range(len(varNamesEndo)):
exec("%s = valuesEndo[%d]" %(varNamesEndo[i], i))
equationVector = jnp.zeros(len(equations))
for i in range(len(equations)):
equationVector = equationVector.at[i].set(eval(equations[i]))
return equationVector
Side-note: this kind of approach based on setting variable names using exec is really brittle and error-prone; I'd suggest an approach based on building explicit namespaces for evaluating your equations. For example something like this:
def objectFunction(valuesEndo, varNamesEndo, valuesExo, varNamesExo, equations):
namespace = {
**dict(zip(varNamesEndo, valuesEndo)),
**dict(zip(varNamesExo, valuesExo))
}
return jnp.array([eval(eqn, namespace) for eqn in equations])

Cannot sample values of a trigonometric function

I have a problem with my code.
So i try to represent the sampled values of a function 'sin(t^3)/2^tan(t)' for
t between 0 and 1.5 and frequency fs=50Hz.
I have created a function 'sampleFunction' which takes as parameters the string which represents the trigonometric function,beginning of the interval,end of interval and the frequency.
I create tVector(0,0.02,0.04,..,1.48)
Then I take the elements of tVector and use them to evaluate the string and put the result in another vector y
I return both y and tVector
But I encounter a problem when i run it saying 'y' is not defined
This is the code:
import numpy as np
import matplotlib.pyplot as plt
import math
def sampleFunction(functionString,t0,t1,fs):
tVector=np.arange(start=t0, stop=t1, step=1/fs, dtype='float')
t=t0
for i in range(0,len(tVector)):
t=tVector[i]
y[i]=eval(functionString)
return y,tVector
t0=0
t1 =1.5
fs=50
thold=.1
functionString='math.sin(t**3)/2**math.tan(t)'
y,t=sampleFunction(functionString,t0,t1,fs)
plt.plot(t,y)
plt.xlabel('time')
plt.ylabel('Amplitude')

You can change your code in the following way:
def sampleFunction(functionString,t0,t1,fs):
tVector=np.arange(start=t0, stop=t1, step=1/fs, dtype='float')
t=t0
y = np.zeros( tVector.shape )
for i in range(0,len(tVector)):
t=tVector[i]
y[i]=eval(functionString)
return y,tVector
However, this is not good python. There are a couple of issues:
You should use vectorized operations.
You should avoid eval like the plague. This has security implications.
For vectorized operations, simply do:
def sampleFunction(functionString,t0,t1,fs):
t = np.arange(start=t0, stop=t1, step=1/fs, dtype='float')
y = eval(functionString)
return y, t
and call it as:
sampleFunction('np.sin(t**3)/2**np.tan(t)', 0, 10, 100)
This is much faster (especially for large arrays)
Finally, the vectorized form is only a single line long. You probably don't need the extra function.

You have a problem with the allocation of the 'y' variable as Harold is saying.
However, there are multiple ways of achieving what you are doing and the eval function is, unless you have a very good reason, the absolute worst. Maybe consider one of the possible examples below:
import numpy as np
import matplotlib.pyplot as plt
import math
def sampleFunction(functionString,t0,t1,fs):
tVector=np.arange(start=t0, stop=t1, step=1/fs, dtype='float')
t=t0
y = [float]*len(tVector) # <------------------- Allocate 'y' variable
for i in range(0,len(tVector)):
t = tVector[i]
y[i]=eval(functionString)
return y,tVector
t0=0
t1 =1.5
fs=50
thold=.1
# Your code
functionString = 'math.sin(t**3)/2**math.tan(t)'
y, t = sampleFunction(functionString,t0,t1,fs)
plt.plot(t, y, color='cyan')
# Using the 'map' built-in function
t = np.arange(start=t0, stop=t1, step=1./fs, dtype='float')
y = map(lambda ti: 0.9*math.sin(ti**3)/2**math.tan(ti), t)
plt.plot(t, y, color='magenta')
# Using Numpy's 'sin' and 'tan'
t = np.arange(start=t0, stop=t1, step=1./fs, dtype='float')
y = 0.8*np.sin(t**3)/2**np.tan(t)
plt.plot(t, y, color='darkorange')
# Using 'list comprehensions'
t = np.arange(start=t0, stop=t1, step=1./fs, dtype='float')
y = [ 0.7*math.sin(ti**3)/2**math.tan(ti) for ti in t]
plt.plot(t, y, color='darkgreen')
plt.xlabel('time')
plt.ylabel('Amplitude')
plt.show()
The result is:

When running the above code, you should have gotten an error message saying, in the end, "name 'y' is not defined". If you look at your function definition, you will see that it really isn't. You cannot passing a value to y[i] without defining y first! The following line before the "for" loop fixes that particular problem:
y = [None] * len(tVector)
The code will run fine after that correction.
But: why do you want to pass a function string when you can pass a function? Functions, in Python, are first-class-objects!

Python - How to plot argument in integral that is not the value being integrated

I want to integrate a function that has no closed form solution with an unknown variable and then plot vs the unknown variable. To try a simpler test, I tried to use the integral of f(x,c) = (x^2+c), integrated with respect to x and plot with different values of c. However, the code below gets the error
only size-1 arrays can be converted to Python scalars
even though the integral of a number, e.g. integral(5), seems to return the correct scalar value.
import numpy as np
import matplotlib.pyplot as plt
from scipy import integrate
def f(x,c):
return x**2+c
def integral(c):
return integrate.quad(f,0,10, args = (c,))[0]
y = np.linspace(0,20,200)
plt.plot(y, integral(y))

You pass a numpy array as the argument c while you wanted to integrate over x for all the items of c. Therefore you can use this:
def f(x,c):
return x**2+c
def integrate_f(c):
result = np.zeros(len(c))
counter = 0
for item in c:
result[counter] = integrate.quad(f,0,10, args = (item))[0]
counter +=1
return result
c_array = np.linspace(0,1,200)
plt.plot(c_array, integrate_f(c_array))

onno was a bit faster. But here is my similar solution. You need to loop over all the different c:
import numpy as np
import matplotlib.pyplot as plt
from scipy import integrate
def f(x,c):
return x**2+c
def getIntegral(c_list):
result = []
for c in c_list:
integral = integrate.quad(f,0,10,args = c)[0]
result.append(integral)
return result
if __name__ == "__main__":
c_list = np.linspace(0,20,200)
plt.plot(c_list, getIntegral(c_list))
plt.show()

NLopt minimize eigenvalue, Python

I have matrices where elements can be defined as arithmetic expressions and have written Python code to optimise parameters in these expressions in order to minimize particular eigenvalues of the matrix. I have used scipy to do this, but was wondering if it is possible with NLopt as I would like to try a few more algorithms which it has (derivative free variants).
In scipy I would do something like this:
import numpy as np
from scipy.linalg import eig
from scipy.optimize import minimize
def my_func(x):
y, w = x
arr = np.array([[y+w,-2],[-2,w-2*(w+y)]])
ev, ew=eig(arr)
return ev[0]
x0 = np.array([10, 3.45]) # Initial guess
minimize(my_func, x0)
In NLopt I have tried this:
import numpy as np
from scipy.linalg import eig
import nlopt
def my_func(x,grad):
arr = np.array([[x[0]+x[1],-2],[-2,x[1]-2*(x[1]+x[0])]])
ev, ew=eig(arr)
return ev[0]
opt = nlopt.opt(nlopt.LN_BOBYQA, 2)
opt.set_lower_bounds([1.0,1.0])
opt.set_min_objective(my_func)
opt.set_xtol_rel(1e-7)
x = opt.optimize([10.0, 3.5])
minf = opt.last_optimum_value()
print "optimum at ", x[0],x[1]
print "minimum value = ", minf
print "result code = ", opt.last_optimize_result()
This returns:
ValueError: nlopt invalid argument
Is NLopt able to process this problem?

my_func should return double, posted sample return complex
print(type(ev[0]))
None
<class 'numpy.complex128'>
ev[0]
(13.607794065928395+0j)
correct version of my_func:
def my_func(x, grad):
arr = np.array([[x[0]+x[1],-2],[-2,x[1]-2*(x[1]+x[0])]])
ev, ew=eig(arr)
return ev[0].real
updated sample returns:
optimum at [ 1. 1.]
minimum value = 2.7015621187164243
result code = 4

Multi-objective optimisation using PyGMO

I am using the PyGMO package for Python, for multi-objective optimisation. I am unable to fix the dimension of the fitness function in the constructor, and the documentation is not very descriptive either. I am wondering if anyone here has had experience with PyGMO in the past: this could be fairly simple.
I try to construct a minimum example below:
from PyGMO.problem import base
from PyGMO import algorithm, population
import numpy as np
import matplotlib.pyplot as plt
class my_problem(base):
def __init__(self, fdim=2):
NUM_PARAMS = 4
super(my_problem, self).__init__(NUM_PARAMS)
self.set_bounds(0.01, 100)
def _objfun_impl(self, K):
E1 = K[0] + K[2]
E2 = K[1] + K[3]
return (E1, E2, )
if __name__ == '__main__':
prob = my_problem() # Create the problem
print (prob)
algo = algorithm.sms_emoa(gen=100)
pop = population(prob, 50)
pop = algo.evolve(pop)
F = np.array([ind.cur_f for ind in pop]).T
plt.scatter(F[0], F[1])
plt.xlabel("$E_1$")
plt.ylabel("$E_2$")
plt.show()
fdim=2 above is a failed attempt to set the fitness dimension. The code fails with the following error:
ValueError: ..\..\src\problem\base.cpp,584: fitness dimension was changed inside objfun_impl().
I'd be grateful if someone can help figure this out. Thanks!

Are you looking at the correct documentation?
There is no fdim (which anyway does nothing in your example since it is only a local variable and is not used). But there is n_obj:
n_obj: number of objectives. Defaults to 1
So, I think you want something like (corrected thanks to #Distopia):
#(...)
def __init__(self, fdim=2):
NUM_PARAMS = 4
super(my_problem, self).__init__(NUM_PARAMS, 0, fdim)
self.set_bounds(0.01, 100)
#(...)

I modified their example and this seemed to work for me.
#(...)
def __init__(self, fdim=2):
NUM_PARAMS = 4
# We call the base constructor as 'dim' dimensional problem, with 0 integer parts and 2 objectives.
super(my_problem, self).__init__(NUM_PARAMS,0,fdim)
self.set_bounds(0.01, 100)
#(...)