I am trying to differentiate z(x) w.r.t. x using the ad library, where I know y(x) and z(y). If I cannot analytically find z(x), how can I perform the differentiation? In other words, I am trying to avoid the chain rule calculation as shown below:
from ad import gh
def y(x):
return 2*x
def z(y):
return 3*y
dzdy,hy = gh(z)
dydx,hz = gh(y)
x0 = 0 # does not matter for this example
dydx_x0 = dydx(x0)
y0 = y(x0)
dzdy_y0 = dzdy(y0)
dzdx_x0 = dzdy_y0[0] * dydx_x0[0]
print(dzdx_x0) # dz/dx = dz/dy*dy/dx = 3*2 = 6
def z_of_x(x):
return z(y(x))
gradient, hessian = gh(z_of_x)
Just define a function to compute z in terms of x, and apply automatic differentiation as usual.
Related
I'm trying to understand the behaviour of argnums in JAX's gradient function.
Suppose I have the following function:
def make_mse(x, t):
def mse(w,b):
return np.sum(jnp.power(x.dot(w) + b - t, 2))/2
return mse
And I'm taking the gradient in the following way:
w_gradient, b_gradient = grad(make_mse(train_data, y), (0,1))(w,b)
argnums= (0,1) in this case, but what does it mean? With respect to which variables the gradient is calculated? What will be the difference if I will use argnums=0 instead?
Also, can I use the same function to get the Hessian matrix?
I looked at JAX help section about it, but couldn't figure it out
When you pass multiple argnums to grad, the result is a function that returns a tuple of gradients, equivalent to if you had computed each separately:
def f(x, y):
return x ** 2 + x * y + y ** 2
df_dxy = grad(f, argnums=(0, 1))
df_dx = grad(f, argnums=0)
df_dy = grad(f, argnums=1)
x = 3.0
y = 4.25
assert df_dxy(x, y) == (df_dx(x, y), df_dy(x, y))
If you want to compute a mixed second derivatives, you can do this by repeatedly applying the gradient:
d2f_dxdy = grad(grad(f, argnums=0), argnums=1)
assert d2f_dxdy(x, y) == 1
I have learned how to automatically find the partial derivative of a function with sympy. My problem is, I need to define a new function that returns the partial derivative of the other function.
from sympy import Symbol, Derivative
y= Symbol('y')
function = y ** 2
deriv = Derivative(function, y).doit()
def func(y):
return deriv
Something like that. Hope you all understood. Thanks!
So y is another function as in predefined like y = Symbol('x') ** 2? I believe your function need another input.
x = Symbol('x')
y = x ** 2
def func(function, symbol):
deriv = Derivative(function, symbol).doit()
return deriv
derivative = func(y, x)
You can't do it without specifying symbol - especially since this is partial derivative you need to tell which symbol it's trying to derive against.
I am trying to solve this differential equation as part of my assignment. I am not able to understand on how can i put the condition for u in the code. In the code shown below, i arbitrarily provided
u = 5.
2dx(t)dt=−x(t)+u(t)
5dy(t)dt=−y(t)+x(t)
u=2S(t−5)
x(0)=0
y(0)=0
where S(t−5) is a step function that changes from zero to one at t=5. When it is multiplied by two, it changes from zero to two at that same time, t=5.
def model(x,t,u):
dxdt = (-x+u)/2
return dxdt
def model2(y,x,t):
dydt = -(y+x)/5
return dydt
x0 = 0
y0 = 0
u = 5
t = np.linspace(0,40)
x = odeint(model,x0,t,args=(u,))
y = odeint(model2,y0,t,args=(u,))
plt.plot(t,x,'r-')
plt.plot(t,y,'b*')
plt.show()
I do not know the SciPy Library very well, but regarding the example in the documentation I would try something like this:
def model(x, t, K, PT)
"""
The model consists of the state x in R^2, the time in R and the two
parameters K and PT regarding the input u as step function, where K
is the infimum of u and PT is the delay of the step.
"""
x1, x2 = x # Split the state into two variables
u = K if t>=PT else 0 # This is the system input
# Here comes the differential equation in vectorized form
dx = [(-x1 + u)/2,
(-x2 + x1)/5]
return dx
x0 = [0, 0]
K = 2
PT = 5
t = np.linspace(0,40)
x = odeint(model, x0, t, args=(K, PT))
plt.plot(t, x[:, 0], 'r-')
plt.plot(t, x[:, 1], 'b*')
plt.show()
You have a couple of issues here, and the step function is only a small part of it. You can define a step function with a simple lambda and then simply capture it from the outer scope without even passing it to your function. Because sometimes that won't be the case, we'll be explicit and pass it.
Your next problem is the order of arguments in the function to integrate. As per the docs (y,t,...). Ie, First the function, then the time vector, then the other args arguments. So for the first part we get:
u = lambda t : 2 if t>5 else 0
def model(x,t,u):
dxdt = (-x+u(t))/2
return dxdt
x0 = 0
y0 = 0
t = np.linspace(0,40)
x = odeint(model,x0,t,args=(u,))
Moving to the next part, the trouble is, you can't feed x as an arg to y because it's a vector of values for x(t) for particular times and so y+x doesn't make sense in the function as you wrote it. You can follow your intuition from math class if you pass an x function instead of the x values. Doing so requires that you interpolate the x values using the specific time values you are interested in (which scipy can handle, no problem):
from scipy.interpolate import interp1d
xfunc = interp1d(t.flatten(),x.flatten(),fill_value="extrapolate")
#flatten cuz the shape is off , extrapolate because odeint will go out of bounds
def model2(y,t,x):
dydt = -(y+x(t))/5
return dydt
y = odeint(model2,y0,t,args=(xfunc,))
Then you get:
#Sven's answer is more idiomatic for vector programming like scipy/numpy. But I hope my answer provides a clearer path from what you know already to a working solution.
I woul like to solve an n-dimensional optimisation problem using iminuit.
So my approach is the following.
I am trying to figure out how to extend this:
def f(x,y,z):
return (x-1.)**2 + (y-2*x)**2 + (z-3.*x)**2 -1.
to a variable "x" that is a numpy.array.
I would like to do something like this:
x = [1,2,3,4,5]
y = [2,4,6,8,10]# y=2x
class StraightLineChi2:
def __init__(self,x,y):
self.x = x
self.y = y
def __call__(self,m,c): #lets try to find slope and intercept
chi2 = sum((y - m*x+c)**2 for x,y in zip(self.x,self.y))
return chi2
but in my case x is my unknown, and it is an array. Like in many optimization/minimization problems, the function is a f=f(x1,...,xn) where n can be big. x1,...,xn are the unknowns of the problem.
(These examples are taken from here)
Something similar is achieved "hacking" pyminuit2, like described here
For your example I recommend you using iminuit and probfit. Having an argument as a list of parameter is not exactly what you want to do since you will get confused which parameter is what very soon.
Here is an example taken straight from probfit tutorial. Also see the documentation
import iminuit
import probfit
x = np.linspace(0, 10, 20)
y = 3 * x + 15 + np.random.randn(len(x))
err = np.ones(len(x))
def line(x, m, c): # define it to be parabolic or whatever you like
return m * x + c
chi2 = probfit.Chi2Regression(line, x, y, err)
minuit = iminuit.Minuit(chi2)
minuit.migrad();
print(minuit.values) #{'c': 16.137947520534624, 'm': 2.8862774144823855}
I have a python function with variable number of arguments:
F(x1, x2, ... , xN)
I want to automatically generate N functions representing the derivatives of F with respect to each argument.
F'_1 = dF/dx1
F'_2 = dF/dx2
...
F'_N = dF/dxN
For example, I be able to give both
F(x1) = sin(x1)
and
F(x1, x2) = sin(x1) * cos(x2)
and get all the derivatives automatically.
Edit2:
If function F was 2 variable (fixed number of arguments), I could use
def f(x,y):
return sin(x)*cos(y)
from sympy import *
x, y = symbols('x y')
f_1 = lambdify((x,y), f(x,y).diff(x))
The trick is to use inspect.getargspec to get the names of all the arguments to the function. After that, it's a simple list comprehension:
import inspect
from sympy import *
def get_derivatives(func):
arg_symbols = symbols(inspect.getargspec(func).args)
sym_func = func(*arg_symbols)
return [lambdify(arg_symbols, sym_func.diff(a)) for a in arg_symbols]
For example:
def f(x, y):
return sin(x)*cos(y)
all_derivatives = get_derivatives(f)