I am applying Euler's method to solve a differential equation. This is my code:
def f(x, y):
return ((x**(2))*y)/((x**(4)) + (y**(4)))
di = 0.01
I = 100
x = np.linspace(-I, I, int(I/di) + 1)
w = np.zeros(len(x))
x[0], w[0]
for i in range(1, len(w)):
w[i] = w[i - 1] + f(x[i - 1], w[i - 1])*di
plt.plot(x, w, label='approximation' )
plt.xlabel("x")
plt.ylabel("y")
plt.show()
When I run the code I have this Warning:
"C:\Users\USER\Anaconda3\lib\site-packages\ipykernel__main__.py:3: RuntimeWarning: invalid value encountered in double_scalars app.launch_new_instance()"
I would like to now how to solve it and make it work.
Your code is running into Divide by Zero Error. Try this to convince yourself:
>>> def f(x, y):
... return ((x**(2))*y)/((x**(4))+(y**(4)))
...
>>> I, di = 100, 0.01
>>> x = np.linspace(-I, I, int(I/di) + 1)
>>> w = np.zeros(len(x))
>>> i = len(x)//2 + 1
>>> i
5001
>>> f(x[i-1], w[i-1])
nan
It clearly emerges from the interactive session above that when i takes the value 5001 in the for loop, f(x[i-1], w[i-1]) yields nan. There are several solutions to this issue. For instance, in order to avoid NaN values you could check whether the denominator of the fraction returned by f() is zero prior to perform the division. If it is, you should return a conventional value of your choice (for example 0) instead of the result of the division. The following snippet implements such approach though a conditional expression:
def f(x, y):
return (0 if x==0 and y==0 else float(x**2*y)/(x**4 + y**4))
Alternatively, you could disable run time warnings (but if you do so you need to be aware of the potential risks) by including this code in your script:
import warnings
def f(x, y):
with warnings.catch_warnings():
warnings.simplefilter('ignore')
return ((x**(2))*y)/((x**(4)) + (y**(4)))
The proposed workarounds avoid the RuntimeWarning but don't get your code working as you expect. Indeed, the calculated solution w is a vector in which all the elements are zero. I guess the reason for your code not being working properly is that you missed to assign w[0] an initial value different to 0.
For example, if you simply add this line just before the for loop:
w[0] = 0.5
you get this (apparently meaningful) curve rather than a flat plot.
Hope this helps!
Related
I'm trying to get a solution by running the code below.
Python just "hangs" and won't find a numeric solution. I can use an app on my phone (Desmos) to graph the functions and find a numeric solution easily, 0.024. Does python have limitations when solving for 2 decimal places?
import sympy
x = sympy.symbols('x')
e_1 = x**-0.5
e_2 = -2*sympy.log(0.0001*3.7**-1*0.05**-1+2.51*350000**-1*x**-0.5, 10)
sol = sympy.solve(e_2 - e_1, x, 0.024)
num = float(sol[0])
print(num)
Usually, nsolve is the SymPy tool used to numerically solve an equation (or a system of equations). However, I wasn't able to use: it kept raising errors. The problem is that your function is defined on a very small region, and the zero is very close to the boundary:
So, in this case we can try numerical root finding techniques. Again, some tools might fail, but after a few tries I've found that bisect works fine:
from sympy import *
from scipy.optimize import root, brentq, bisect
x = symbols('x')
# you didn't provide the diameter, so I've computed it based on your expected solution
d = 1.56843878182221
e_1 = x**-0.5
e_2 = -2 * log(0.00013 * 7-1*d-1+2.51350000**-1*x**-0.5, 10)
# convert the symbolic expression to a numerical function
ff = lambdify(x, e_1 - e_2)
root, output = bisect(ff, 0.023, 0.025, full_output=True)
print(root)
# 0.024000000001862646
print(output)
# converged: True
# flag: 'converged'
# function_calls: 32
# iterations: 30
# root: 0.024000000001862646
The fixed point method is a great one to use for situations like this. Or at least the principles of transforming the equation into a compatible form can benefit standard solvers by providing a less ill-behaved form of the function.
You have an ill-defined equation in the form y - g(y) where y = 1/sqrt(x). So let's get the inverse of g (call it G) so we can solve G(y) - G(g(y)) = G(y) - y instead.
>>> g = e_2.subs(1/x**.5, y)
>>> d = Dummy()
>>> G = solve(g - d, y)[0].subs(d, y)
>>> nsolve(G - y, 6)
6.45497224367903
>>> solve(1/x**.5 - _, dict=True)
{x: 0.024}
The process of rearranging an equation f(x) into form x - g(x) could probably use a built-in method in SymPy, but it's not too hard to do this by replacing each x with a dummy variable, solving for it, and then replacing the dummy symbols with x again. Different g will be more favorable for finding different roots as can be seen in the example below where the purple dashed line is good for finding the root near 1 while the solid blue is better near the smaller root.
Here is a possibility for a "fixed point form" function:
def fixedpoint_Eqs(eq, x=None):
"""rearrange to give eq in form x = g(x)"""
f = eq.free_symbols
fp = []
if x is None:
assert len(f) == 1, 'must specify x in this case'
x = list(f)[0]
Xeq = eq.replace(lambda _:_ == x, lambda _:Dummy())
X = Xeq.free_symbols - f
reps = {xi: x for xi in X}
for xi in X:
try:
g = solve(Xeq, xi, dict=True)
if len(g) != 1:
raise NotImplementedError
fp.append(Eq(x, g[0][xi].xreplace(reps)))
except NotImplementedError:
pass
return fp
>>> fixedpoint_Eqs(x+exp(x)+1/x-5)
Eq(x, -1/(x + exp(x) - 5))
Eq(x, -exp(x) + 5 - 1/x)
Eq(x, log(-x + 5 - 1/x))
Hi I am trying to do linear regression and this what happens to me when n I try to run the code
the code is :
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
data = pd.read_csv('train.csv')
df = data.dropna()
X = np.array(df.x)
Y = np.array(df.y)
def compute_error(x,y,m,b):
error = 0
for i in range(len(x)):
error+= (y[i]-(m*x[i]+b))**2
return error / float(len(x))
def step_graident_descent(x,y,m,b , alpha):
N = float(len(x))
b_graident = 0
m_graident = 0
for i in range(0 , len(x)):
x = X[i]
y = Y[i]
b_graident +=(-2/N) * (y-(m*x+b))
m_graident += (-2/N) * x*(y-(m*x+b))
new_m = m - alpha*m_graident
new_b = b - alpha*b_graident
return new_m , new_b
def graident_decsent(x,y,m,b,num_itters,alpha):
for i in range(num_itters):
m,b = step_graident_descent(x,y,m,b,alpha)
return m,b
def run():
b=0
m=0
numberOfIttertions = 1000
m,b = graident_decsent(X , Y ,m,b,numberOfIttertions , 0.001)
print(m,b)
if __name__ == '__main__':
run()
and the error that i get is :
linearRegression.py:22: RuntimeWarning: overflow encountered in double_scalars
m_graident += (-2/N) * x*(y-(m*x+b))
linearRegression.py:21: RuntimeWarning: invalid value encountered in double_scalars
b_graident +=(-2/N) * (y-(m*x+b))
linearRegression.py:22: RuntimeWarning: invalid value encountered in double_scalars
m_graident += (-2/N) * x*(y-(m*x+b))
if any one can help me i would be so greatfull since i am stuck on this for about two months and thankyou
Edit: tl;dr Solution
Ok so here is the minimal reproducible example that I was talking of. I replace your X,Y by the following.
n = 10**2
X = np.linspace(0,10**6,n)
Y = 1.5*X+0.2*10**6*np.random.normal(size=n)
If I then run
b=0
m=0
numberOfIttertions = 1000
m,b = graident_decsent(X , Y ,m,b,numberOfIttertions , 0.001)
I get exactly the problem you describe. The only surprising thing is the ease of the solution. I just replace your alpha by 10**-14 and everything works fine.
Why and how to give a Minimal, Reproducible Example
Your example is not reproducible since we don't have train.csv. Generally both for understanding your problem yourself and to get concrete answers it is very helpful to have a very small example that people can run and tinker with it. E.g. maybe you can think of a much shorter input to your regression that also results in this error.
The first RuntimeWarning
But now to your question. Your first RuntimeWarning i.e.
linearRegression.py:22: RuntimeWarning: overflow encountered in double_scalars
m_graident += (-2/N) * x*(y-(m*x+b))
means x and hence m_graident are of type numpy.double=numpy.float64. This datatype can store numbers in the range (-1.79769313486e+308, 1.79769313486e+308). If you go bigger or smaller that's called an overflow. E.g.
np.double(1.79769313486e+308)
is still ok but if you multiply it by say 1.1 you get your favorite runtime warning. Notice that this is 'just' a warning and still runs. But it can't give you a number back since it would be too big. instead it gives you inf.
The other RuntimeWarnings
Ok but what does
linearRegression.py:21: RuntimeWarning: invalid value encountered in double_scalars
b_graident +=(-2/N) * (y-(m*x+b))
mean?
It comes from calculating with the infinity that I just mentioned. Some calculations with infinity are valid.
np.inf-10**6 -> inf
np.inf+10**6 -> inf
np.inf/10**6 -> inf
np.inf*10**6 -> inf
np.inf*(-10**6) -> -inf
1/np.inf -> 0
np.inf *np.inf -> inf
but some are not and give nan i.e. not a number.
np.inf/np.inf
np.inf-np.inf
These are called indeterminate forms in math since it depends on how you got to the infinity what you would get out. E.g.
(np.double(1e+309)+np.double(1e+309))-np.double(1e+309)
np.double(1e+309)-(np.double(1e+309)+np.double(1e+309))
are both inf-inf but you would expect different results.
Getting a nan is unfortunate since calculations with nan yield always nan. And you can't use your gradients anymore once you add a nan.
Other resources
An other option is to use an existing implementation of linear regression. E.g. from scikit-learn. See
scikit-learn linear regression reference
scikit-learn user guid on linear models
The issue I am having is that when I use the code below to find the norm-1 of my error. Firstly, when I plot the error against step-size h, the error values are quite small, in the range of 10^-14 to 10^-16. Secondly, underneath, you can see my attempt to apply the np.polyfit to my graph, which when run, won't fit a characteristic but will output values. The value of p[0] is not perfect, so I believe something is wrong, but it is "close" to the desired output of 3. Is this a matter of just the wrong input or bad data?
def rk3(A,bvector,y0,interval,N):
x0=interval[0]
x_end=interval[1]
x=np.linspace(x0,x_end,N+1)
h=(x_end-x0)/N
y=np.zeros((N+1,len(y0)))
y[0, :] = y0
for n in range(N):
y_1=y[n,:]+h*(np.dot(A,y[n,:])+bvector(x[n]))
y_2=(3/4)*y[n,:]+(1/4)*y_1+(1/4)*h*(np.dot(A,y_1)+bvector(x[n]+h))
y[n+1,:]=(1/3)*y[n,:]+(2/3)*y_2+(2/3)*h*(np.dot(A,y_2)+bvector(x[n]+(1/2)*h))
return x,y
err_vals = []
h_vals = []
for k in range(2,11): #for the range of N=40k, where k=1,...,10
N=40*k
x, y = rk3(A,bvector,y0,[0,0.1],N)
yc = y[-1,:]
h = (x[-1]-x[0])/N
h_vals.append(h)
yvals.append(yc)
yn = y[:,1]
abs_err = np.zeros(N)
print("The value of y at k=",k," is ",yc)
for j in range(1,N):
y_exact=np.array([np.exp(-1000*x[j]), (1000/999)*(np.exp(-x[j])-np.exp(-1000*x[j]))])
y_exact_2 = y_exact[1]
abs_err[j] = np.abs((y[j, 1] - y_exact_2)/y_exact_2)
Error = h*np.sum(abs_err[j])
err_vals.append(Error)
p = np.polyfit(np.log(h_vals), np.log(err_vals), 1)
pyplot.loglog(h_vals,err_vals,"kx")
pyplot.xlabel("h")
pyplot.ylabel("Error")
pyplot.loglog(h,np.exp(p[1])*h**(p[0]), 'r--')
print("Best fit line slope ",format(p[0]))
My evolution of your code below gives a completely straight line with slope close to 3 for the integration over the interval [0,0.01].
For the given interval [0,0.1] the slope value is about 1/3 larger. The error profiles, that is, the absolute error divided by the expected global error power of the step size, gives a converging pattern, confirming the convergence of order 3 of the method.
The error bound 2e7*h^3 is rather large, showing why the combination of problem and method can become very problematic for larger step sizes.
The error is computed via the L1 norms of the function difference and exact solution,
Error = sum(abs((y-y_exact(x))[:,1]))/sum(abs(y[:,1]))
giving a mathematically sound quantity. The summation of the local relative errors can lead to distortions of the total error where the exact solution has a root or small values. But still, even using your computation method of integrating the local relative error leaving out the first data point which is zero,
Error = sum(abs((y[1:,1]/y_exact(x)[1:,1]-1)))*h
gives a similar linear plot, with the range shifted down to 1e-7..1e-9, the slope staying at 3.0293
Note that if you want to use the list h_vals in a computation like the one to plot the fitted line, you have to convert in into a numpy array first.
h=np.asarray(h_vals)
complete code
def rk3(A,bvector,y0,interval,N):
"""Solves an IVP y'=f(x, y(x)) on x \in [0, x_end] with y(0) = y0 using N points, using Runge-Kutta method."""
x=np.linspace(*interval,N+1)
h=x[1]-x[0]
y=np.zeros((N+1,len(y0)))
y[0, :] = y0
for n in range(N):
y_1=y[n]+h*(np.dot(A,y[n])+bvector(x[n]))
y_2=(3/4)*y[n,:]+(1/4)*y_1+(1/4)*h*(np.dot(A,y_1)+bvector(x[n]+h))
y[n+1]=(1/3)*y[n]+(2/3)*y_2+(2/3)*h*(np.dot(A,y_2)+bvector(x[n]+0.5*h))
return x,y
A = np.array([[-1000.0,0.0],[1000.0,-1.0]]);
bvector = lambda x: 0
y_exact = lambda x: np.array([np.exp(-1000*x), (1000/999)*(np.exp(-x)-np.exp(-1000*x))]).T
y0 = y_exact(0)
plt.figure(figsize=(6,3));
h_vals, y_vals, err_vals = [],[],[]
for k in range(2,11): #for the range of N=40k, where k=1,...,10
N=40*k
x, y = rk3(A,bvector,y0,[0,0.01],N)
yc = y[-1,:]
h = x[1]-x[0];
plt.plot(x,(y-y_exact(x))[:,1]/h**3)
h_vals.append(h)
y_vals.append(yc)
yn = y[:,1]
print("The value of y at k=",k," is ",yc)
Error = sum(abs((y-y_exact(x))[:,1]))/sum(abs(y[:,1]))
err_vals.append(Error)
plt.grid(); plt.show()
p = np.polyfit(np.log(h_vals), np.log(err_vals), 1)
plt.figure(figsize=(6,4))
plt.loglog(h_vals,err_vals,"kx")
h=np.asarray(h_vals)
plt.plot(h,np.exp(p[1])*h**(p[0]), '--r', lw=0.5)
plt.xlabel("h")
plt.ylabel("Error")
plt.grid(); plt.show()
print("Best fit line slope ",format(p[0]))
I am trying to solve this differential equation as part of my assignment. I am not able to understand on how can i put the condition for u in the code. In the code shown below, i arbitrarily provided
u = 5.
2dx(t)dt=−x(t)+u(t)
5dy(t)dt=−y(t)+x(t)
u=2S(t−5)
x(0)=0
y(0)=0
where S(t−5) is a step function that changes from zero to one at t=5. When it is multiplied by two, it changes from zero to two at that same time, t=5.
def model(x,t,u):
dxdt = (-x+u)/2
return dxdt
def model2(y,x,t):
dydt = -(y+x)/5
return dydt
x0 = 0
y0 = 0
u = 5
t = np.linspace(0,40)
x = odeint(model,x0,t,args=(u,))
y = odeint(model2,y0,t,args=(u,))
plt.plot(t,x,'r-')
plt.plot(t,y,'b*')
plt.show()
I do not know the SciPy Library very well, but regarding the example in the documentation I would try something like this:
def model(x, t, K, PT)
"""
The model consists of the state x in R^2, the time in R and the two
parameters K and PT regarding the input u as step function, where K
is the infimum of u and PT is the delay of the step.
"""
x1, x2 = x # Split the state into two variables
u = K if t>=PT else 0 # This is the system input
# Here comes the differential equation in vectorized form
dx = [(-x1 + u)/2,
(-x2 + x1)/5]
return dx
x0 = [0, 0]
K = 2
PT = 5
t = np.linspace(0,40)
x = odeint(model, x0, t, args=(K, PT))
plt.plot(t, x[:, 0], 'r-')
plt.plot(t, x[:, 1], 'b*')
plt.show()
You have a couple of issues here, and the step function is only a small part of it. You can define a step function with a simple lambda and then simply capture it from the outer scope without even passing it to your function. Because sometimes that won't be the case, we'll be explicit and pass it.
Your next problem is the order of arguments in the function to integrate. As per the docs (y,t,...). Ie, First the function, then the time vector, then the other args arguments. So for the first part we get:
u = lambda t : 2 if t>5 else 0
def model(x,t,u):
dxdt = (-x+u(t))/2
return dxdt
x0 = 0
y0 = 0
t = np.linspace(0,40)
x = odeint(model,x0,t,args=(u,))
Moving to the next part, the trouble is, you can't feed x as an arg to y because it's a vector of values for x(t) for particular times and so y+x doesn't make sense in the function as you wrote it. You can follow your intuition from math class if you pass an x function instead of the x values. Doing so requires that you interpolate the x values using the specific time values you are interested in (which scipy can handle, no problem):
from scipy.interpolate import interp1d
xfunc = interp1d(t.flatten(),x.flatten(),fill_value="extrapolate")
#flatten cuz the shape is off , extrapolate because odeint will go out of bounds
def model2(y,t,x):
dydt = -(y+x(t))/5
return dydt
y = odeint(model2,y0,t,args=(xfunc,))
Then you get:
#Sven's answer is more idiomatic for vector programming like scipy/numpy. But I hope my answer provides a clearer path from what you know already to a working solution.
I'm trying to solve a system of ordinary differential equations with Euler's method, but when I try to print velocity I get
RuntimeWarning: overflow encountered in double_scalars
and instead of printing numbers I get nan (not a number). I think the problem might be when defining the acceleration, but I don't know for sure, I would really appreciate if someone could help me.
from numpy import *
from math import pi,exp
d=0.0005*10**-6
a=[]
while d<1.0*10**-6 :
d=d*2
a.append(d)
D=array(a)
def a_particula (D,x, v, t):
cc=((1.00+((2.00*lam)/D))*(1.257+(0.400*exp((-1.10*D)/(2.00*lam)))))
return (g-((densaire*g)/densparticula)-((mu*18.0*v)/(cc*densparticula* (D**2.00))))
def euler (acel,D, x, v, tv, n=15):
nv, xv, vv = tv.size, zeros_like(tv), zeros_like(tv)
xv[0], vv[0] = x, v
for k in range(1, nv):
t, Dt = tv[k-1], (tv[k]-tv[k-1])/float(n)
for i in range(n):
a = acel(D,x, v, t)
t, v = t+Dt, v+a*Dt
x = x+v*Dt
xv[k], vv[k] = x, v
return (xv, vv)
g=(9.80)
densaire= 1.225
lam=0.70*10**-6
densparticula=1000.00
mu=(1.785*10**-5)
tv = linspace(0, 5, 50)
x, v = 0, 0 #initial conditions
for j in range(len(D)):
xx, vv = euler(a_particula, D[j], x, v, tv)
print(D[j],xx,vv)
In future it would be helpful if you included the full warning message in your question - it will contain the line where the problem occurs:
tmp/untitled.py:15: RuntimeWarning: overflow encountered in double_scalars
return (g-((densaire*g)/densparticula)-((mu*18.0*v)/(cc*densparticula* (D**2.00))))
Overflow occurs when the magnitude of a variable exceeds the largest value that can be represented. In this case, double_scalars refers to a 64 bit float, which has a maximum value of:
print(np.finfo(float).max)
# 1.79769313486e+308
So there is a scalar value in the expression:
(g-((densaire*g)/densparticula)-((mu*18.0*v)/(cc*densparticula* (D**2.00))))
that is exceeding ~1.79e308. To find out which one, you can use np.errstate to raise a FloatingPointError when this occurs, then catch it and start the Python debugger:
...
with errstate(over='raise'):
try:
ret = (g-((densaire*g)/densparticula)-((mu*18.0*v)/(cc*densparticula* (D**2.00))))
except FloatingPointError:
import pdb
pdb.set_trace()
return ret
...
From within the debugger you can then check the values of the various parts of this expression. The overflow seems to occur in:
(mu*18.0*v)/(cc*densparticula* (D**2.00))
The first time the warning occurs, (cc*densparticula* (D**2.00) evaluates as 2.3210168586496022e-12, whereas (mu*18.0*v) evaluates as -9.9984582297025182e+299.
Basically you are dividing a very large number by a very small number, and the magnitude of the result is exceeding the maximum value that can be represented. This might be a problem with your math, or it might be that your inputs to the function are not reasonably scaled.
Your system reduces to
dv/dt = a = K - L*v
with K about 10 and L ranging between, at first glance 1e+5 to 1e+10. The actual coefficients used confirm that:
D=1.0000e-09 K= 9.787995 L=1.3843070e+08
D=3.2000e-08 K= 9.787995 L=4.2570244e+06
D=1.0240e-06 K= 9.787995 L=9.0146813e+04
The Euler step for the velocity is
v[j+1]=v[j]+(K-L*v[j])*dt =(1-L*dt)*v[j] + K*dt
For anything resembling the intended friction effect, i.e., the velocity falling to K/L one needs that abs(1-L*dt)<1, and if possible 0<1-L*dt<1, that is, dt < 1/L. Which means here that dt < 1e-10.
To be able to use larger time steps you need to use methods for stiff differential equations, which means implicit methods. The most simple ones are the implicit Euler method, the midpoint method and the trapezoidal method.
Because of the linearity, the midpoint and trapezoidal method amount to the same formula
v[j+1] = v[j] + dt * ( K - L*(v[j]+v[j+1])/2 )
or
v[j+1] = ( (1-L*dt/2)*v[j] + K*dt ) / (1+L*dt/2)
Of course, the most simple method is to just exactly integrate the ODE
(-L*v')/(K-L*v)=-L => K-L*v(t)=C*exp(-L*t), C=K-L*v(0)
v(t)=K/L + exp(-L*t)*(v(0)-K/L)
which integrates to
x(t)=x(0)+K/L*t+(1-exp(-L*t))/L*(v(0)-K/L).
Alternatively it is possible that you made an error in transcribing the physical laws into formulas so that the magnitudes of the constants are all wrong.