I am reading Data Science From Scratch by Joel Grus, and am stuck on understanding how a simple difference quotient function works using python.
Here is the code:
f is a function of a one variable function, dependent on x, as h approaches 0.
def difference_quotient(f, x, h):
return (f(x + h) - f(x)) / h
I understand that this is just the equation for finding the limit, and in this case the derivative, but I don't see how the script works. What arguments would you put into the function that would allow for a return statement of f(something) and not f*(something). So far I keep getting errors such as 'float'/'dict' object is not callable.
f is the function you are trying to find the derivative/difference quotient of at x.
Try this code: (python 3)
def difference_quotient(f, x, h):
return (f(x + h) - f(x)) / h
def func(x):
# this is f of x, the function you are taking the derivative of
return x * x
print(difference_quotient(func, 2, 0.001))
"Someone" has answered your question but you (and future readers) may find this supplementary information useful.
To get an accurate approximation of the derivative you need to make h fairly small. However, if you make it too small then you actually lose accuracy due to the limited precision of Python floats. You can get more precision by using the decimal module. However, that module only supports simple arithmetic operators and the square root function. If you need fancier functions like trigonometric or exponential functions, then you can use a 3rd-party package arbitrary-precision mathemtaics package such as the excellent mpmath, although if you do use mpmath then you'd probably use its numerical derivative functions.
FWIW, you can make the derivative approximation more accurate (for a given h) by making x the middle of the interval. Here's a short demo:
def dfa(f, x, h):
return (f(x + h) - f(x)) / h
def dfb(f, x, h):
hh = 0.5 * h
return (f(x + hh) - f(x - hh)) / h
# The function
def func(x): return x**3 + x*x + x + 1
# Its exact derivative
def dfunc(x): return 3*x*x + 2*x + 1
h = 0.001
for i in range(10):
x = 1 + 0.1 * i
print(x, dfunc(x), dfb(func, x, h), dfa(func, x, h))
output
1.0 6.0 6.00000024999936 6.004000999999093
1.1 6.830000000000001 6.830000249999024 6.8343009999995985
1.2 7.719999999999999 7.72000024999997 7.724600999999609
1.3 8.67 8.670000249998644 8.674900999999124
1.4 9.68 9.680000249999487 9.685200999998145
1.5 10.75 10.750000249998948 10.755500999998446
1.6 11.880000000000003 11.880000249998801 11.885800999996476
1.7000000000000002 13.070000000000002 13.07000024999816 13.07610099999934
1.8 14.32 14.320000249997022 14.326400999998157
1.9 15.629999999999999 15.630000249997167 15.636700999996478
Here are the results for the exponential function.
from math import exp
def dfa(f, x, h):
return (f(x + h) - f(x)) / h
def dfb(f, x, h):
hh = 0.5 * h
return (f(x + hh) - f(x - hh)) / h
func = dfunc = exp
h = 0.001
for i in range(10):
x = 1 + 0.1 * i
print(x, dfunc(x), dfb(func, x, h), dfa(func, x, h))
output
1.0 2.718281828459045 2.718281941720413 2.7196414225332255
1.1 3.0041660239464334 3.0041661491195804 3.005668607777512
1.2 3.3201169227365472 3.320117061074157 3.3217775346887635
1.3 3.6692966676192444 3.669296820505874 3.6711319276547805
1.4 4.0551999668446745 4.0552001358102885 4.057228242863253
1.5 4.4816890703380645 4.481689257074706 4.483930662008362
1.6 4.953032424395115 4.953032630771403 4.955509766318755
1.7000000000000002 5.473947391727201 5.473947619807795 5.476685277975513
1.8 6.0496474644129465 6.049647716480422 6.0526732966712515
1.9 6.6858944422792685 6.685894720857455 6.689238504094419
Note that dfb is not foolproof: if you're trying to find the derivative for an x value that's near a pole or discontinuity, then shifting the interval to include the pole or discontinuity can give erroneous results. Of course, you can get the same problem with dfa, eg if f(x) = 1/x and x is a negative number but x+h is positive.
Related
I'm tring to approximate an empirical cumulative distribution function (ECDF I want to approximate) with a smooth function (with less than 5 parameter) such as the generalized logistic function.
However, using scipy.optimize.curve_fit, the fitting operation gives really bad approximations or it doesn't work at all (depending on the initial values). The variable series represents my data stored as pandas.Series.
from scipy.optimize import curve_fit
def fit_ecdf(x):
x = np.sort(x)
def result(v):
return np.searchsorted(x, v, side='right') / x.size
return result
ecdf = fit_ecdf(series)
def genlogistic(x, B, M, Q, v):
return 1 / (1 + Q * np.exp(-B * (x - M))) ** (1 / v)
params = curve_fit(genlogistic, xdata = series, ydata = ecdf(series), p0 = (0.1, 10.0, 0.1, 0.1))[0]
Should I use another type of function for the fit?
Are there any code mistakes?
UPDATE - 1
As asked, I link to a csv containing the data.
UPDATE - 2
After a lot of search and trial and error I find out this function
f(x; a, b, c) = 1 - 1 / (1 + (x / b) ** a) ** c
with a = 4.61320000, b = 2.94570952, c = 0.5886922
which fits a lot better than the other one. The only problem is the little step that the ECDF shows near x=1. How can I modify f to improve the quality of the fit? I was thinking of adding some sort of function that is "relevant" only in those kind of points. Here are the graphical results of the fit where the solid blue line is the ECDF and the dotted line represents the (x, f(x)) points.
I find out how to deal with that little step near x=1. As expressed in the question, adding some sort of function that is significant only in that interval was the game changer.
The "step" ends at about (1.7, 0.04) so I needed a sort of function that flattens for x > 1.7 and has y = 0.04 as asymptote. The natural choice (just to stay on point) was to take a function like f(x) = 1/exp(x).
Thanks to JamesPhillips, I also picked up the proper data for the regression (no double values = no overweighted points).
Python Code
from scipy.optimize import curve_fit
def fit_ecdf(x):
x = np.sort(x)
def result(v):
return np.searchsorted(x, v, side = 'right') / x.size
return result
ecdf = fit_ecdf(series)
unique_series = series.unique().tolist()
def cdf_interpolation(x, a, b, c, d):
f_1 = 0.95 + (0 - 0.95) / (1 + (x / b) ** a) ** c + 0.05
f_2 = (0 - 0.05)/(np.exp(d * x))
return f_1 + f_2
params = curve_fit(cdf_interpolation,
xdata = unique_series ,
ydata = ecdf(unique_series),
p0 = (6.0, 3.0, 0.4, 1.0))[0]
Parameters
a = 6.03256462
b = 2.89418871
c = 0.42997956
d = 1.06864006
Graphical results
I got an OK fit for a 5-parameter logistic equation (see image and code) using unique values, not sure if the low end curve is sufficient for your needs, please check.
import numpy as np
def Sigmoidal_FiveParameterLogistic_model(x_in): # from zunzun.com
# coefficients
a = 9.9220221252324947E-01
b = -3.1572339989462903E+00
c = 2.2303376075685142E+00
d = 2.6271495036080207E-02
f = 3.4399008905318986E+00
return d + (a - d) / np.power(1.0 + np.power(x_in / c, b), f)
I have attempted to solve the following problem. I tried to solve it first with a set step size h using 0.1. However I need to change this in my code and use a for loop to loop through the values 0,1,..,20. I am a little confused how to do this problem but I was hoping to get some help with fixing the code I produced so far. Thanks!
import numpy as np
from math import sin
def derivative(func , x, h ):
for h in range(20):
return (func(x+h)-func(x))/h
def f(x):
return sin(x)
print(derivative(f, pi/4))
Gives the output
0.6706029729039897
MY EDIT:
def derivative(func , x, h ):
for h in range(20):
return (func(x+h)-func(x))/h
The exercise is asking you to compute the derivative using varying precision (represented using the variable h), and compare that to the exact/real derivative of the function.
Let h = 10 ^ -j, with j varying from 0 to 20. This means h will go (discretely) from 10⁻⁰ to 10⁻²⁰. You can use a for-loop and the range(...) function for that. Then pass that to the derivative function (to which you can a third parameter for the value of h)
def derivative(func, x, h):
return (func(x + h) - func(x)) / h
Next, you need to compare that to the exact derivative. The function f(x) = sin(x) has a known (exact) derivative which is cos(x). In math notation, d(sin x)/dx = cos x. This means that for any x, cos(x) will give you the exact derivative of sin at that x.
So you need to compare the result of the derivative(...) function to the value of cos(x). This will give you the difference. You can then use the basic Python function abs(x) to get the absolute value of that difference, which will give you the absolute difference, which is the desired result. Do that for each j from 0 to 20 and store the results somewhere, in an array or a dict.
from math import sin, cos, pi
x = pi / 4
diffs = {}
for j in range(21): # range is exclusive so range(21) will stop at 20
h = 10 ** -j
deriv = derivative(sin, x, h)
exact = cos(x)
diff = abs(deriv - exact)
diffs[h] = diff
Then, you can use pyplot's loglog function to plot those results on a graph, passing as X the range(...) result and as Y the array containing the results.
import matplotlib.pyplot as plt
ordered = sorted(diffs.items())
x, y = zip(*ordered)
plt.loglog(x, y)
Let's suppose that I have two transcendental functions f(x, y) = 0 and g(a, b) = 0.
a and b depend on y, so if I could solve the first equation analytically for y, y = f(x), I could have the second function depending only on x and thus solving it numerically.
I prefer to use python, but if matlab is able to handle this is ok for me.
Is there a way to solve analytically trascendent functions for a variable with python/matlab? Taylor is fine too, as long as I can choose the order of approximation.
I tried running this through Sympy like so:
import sympy
j, k, m, x, y = sympy.symbols("j k m x y")
eq = sympy.Eq(k * sympy.tan(y) + j * sympy.tan(sympy.asin(sympy.sin(y) / x)), m)
eq.simplify()
which turned your equation into
Eq(m, j*sin(y)/(x*sqrt(1 - sin(y)**2/x**2)) + k*tan(y))
which, after a bit more poking, gives us
k * tan(y) + j * sin(y) / sqrt(x**2 - sin(y)**2) == m
We can find an expression for x(y) like
sympy.solve(eq, x)
which returns
[-sqrt(j**2*sin(y)**2/(k*tan(y) - m)**2 + sin(y)**2),
sqrt(j**2*sin(y)**2/(k*tan(y) - m)**2 + sin(y)**2)]
but an analytic solution for y(x) fails.
Finite difference is a numerical method to approximate a derivative.
There are mainly three types of finite difference method: central,
forward and backward.
In this question, I will introduce central difference method. we can
approximate f '(x) by: f'(x)=(f(x+h)-f(x-h))/2h. Write a code to
calculate the first derivative of sin(x) at x=0 with h=1, 0.1,
0.01 and 0.001. Also compare the result with cos(0) (to use function “cos()”, you should import “math” first).
Draw a graph to compare the result of different h value, x axis is the
value of h and y1 axis is the calculated first derivative of sin(x)
and y2 is the result of cos(0).
#!/usr/bin/env python
import math
def f(x):
return sin(x)
def derivative(0,h=1)
deriv = (f(x + h) - f(x - h))/ 2 * h
return round(deriv, 0)
print "f'(x)=", derivative(0)
Can anyone give me some suggestions?
Your main problem is you are dividing by 2 * h, this means divide by 2, then multiply by h. You need to divide by (2 * h). You might have learned the BODMAS rules in school.
>>> 12 / 2 * 3
18
That divides 12 by 2 to get 6, then multiplies 6 by 3. If you group (2 * 3) using parentheses, the whole multiplication operation will be evaluated first:
>>> 12 / (2 * 3)
2
Fixed code:
def f_dash(f, x, h):
return (f(x + h) - f(x - h)) / (2 * h)
>>> from math import sin, cos
>>> f_dash(f=sin, x=0, h=0.1)
0.9983341664682815
>>> f_dash(sin, 0, 0.001)
0.9999998333333416
I try to make this simple program to compute the derivative of a function with normal precision:
# second derivative of a function
def diff2(f, x, h=1E-6):
r = (f(x-h) - 2*f(x) + f(x+h))/float(h*h)
return r
# define the function to derivate
def g(t):
return t**(-6)
# decresing h increasing the precision of the derivative
# ROUND-OFF problems are present
for k in range(1,15):
h = 10**(-k) # 15 different value of h
d2g = diff2(g, 1, h) # compute d'' of g 15-th times in point t=1
print 'h=%.0e: %.5f' % (h, d2g)
As one can see from the printing operation I have problem when k is larger than 8 due to round off. I know I can use:
from decimal import *
But I do not know how to implement these command in my functions.
Could someone help me please?
It is worth looking into the python module mpmath, which can handle arbitrary precision. For example:
>>> from mpmath import mp
>>> mp.dps = 50
>>> print(mp.quad(lambda x: mp.exp(-x**2), [-mp.inf, mp.inf]) ** 2)
3.1415926535897932384626433832795028841971693993751
You could simply change types and let your functions work with greater precision. It is worth noting the comments and the answer by #halex however.
If you want the derivative at position x_0 and perform your calculation with floats the optimal value for h which minimizes the numerical error is sqrt(sys.float_info.epsilon)*x_0 which is approximately 1E-8 for your case where x_0=1.
For further information and a derivation of this value see the chapter How to Choose hstarting at page 4 until the end in this short script on Numerical Differentiation.
You can use the decimal module:
from decimal import Decimal
# second derivative of a function
def diff2(f, x, h=1E-6):
x, h = Decimal(x), Decimal(h)
r = (f(x - h) - 2 * f(x) + f(x + h)) / Decimal(h * h)
return r