We Keep Coding

Convert string of a named expression in sympy to the expression itself

Description of the practical problem:
I have defined many expression using sympy, as in
import sympy as sp
a, b = sp.symbols('a,b', real=True, positive=True)
Xcharles_YclassA_Zregion1 = 1.01 * a**1.01 * b**0.99
Xbob_YclassA_Zregion1 = 1.009999 * a**1.01 * b**0.99
Xbob_YclassA_Zregion2 = 1.009999 * a**1.01 * b**0.99000000001
...
So I have used the names of the expressions to describe options (e.g., charles, bob) within categories (e.g., X).
Now I want a function that takes two strings (e.g., 'Xcharles_YclassA_Zregion1' and 'Xbob_YclassA_Zregion1') and returns its simplified ratio (in this example, 1.00000099009999), so I can quickly check "how different" they are, in terms of result, not in terms of how they are written.
E.g., 2*a and a*2 are the same for my objective.
How can I achieve this?
Notes:
The expressions in the example are hardcoded for the sake of simplicity. But in my actual case they come from a sequence of many other expressions and operations.
Not all combinations of options for all categories would exist. E.g., Xcharles_YclassA_Zregion2 may not exist. Actually, if I were to write a table for existing expression names, it would be sparsely filled.
I guess rewriting my code using dict to store the table might solve my problem. But I would have to modify a lot of code for that.
Besides the practical aspects of my objective, I don't know if there is any formal difference between Symbol (which is a specific class) and expression. From the sources I read (e.g., this) I did not arrive to a conclusion. This understanding may help in solving the question.
TL;DR - What I tried
I aimed at something like
def verify_ratio(vstr1, vstr2):
"""Compare the result of two different computations of the same quantity"""
ratio = sp.N(sp.parsing.sympy_parser.parse_expr(vstr1)) / sp.parsing.sympy_parser.parse_expr(vstr2)
print(vstr1 + ' / ' + vstr2, '=', sp.N(ratio))
return
This did not work.
Code below shows why
import sympy as sp
a, b = sp.symbols('a,b', real=True, positive=True)
expr2 = 1.01 * a**1.01 * b**0.99
print(type(expr2), '->', expr2)
expr2b = sp.parsing.sympy_parser.parse_expr('expr2')
print(type(expr2b), '->', expr2b)
expr2c = sp.N(sp.parsing.sympy_parser.parse_expr('expr2'))
print(type(expr2c), '->', expr2c)
#print(sp.N(sp.parsing.sympy_parser.parse_expr('expr2')))
expr2d = sp.sympify('expr2')
print(type(expr2d), '->', expr2d)
with output
<class 'sympy.core.mul.Mul'> -> 1.01*a**1.01*b**0.99
<class 'sympy.core.symbol.Symbol'> -> expr2
<class 'sympy.core.symbol.Symbol'> -> expr2
<class 'sympy.core.symbol.Symbol'> -> expr2
I need something that takes the string 'expr2' and returns the expression 1.01 * a**1.01 * b**0.99.
None of my attempts achieved the objective.
Questions or links which did not help (at least for me):
From string to sympy expression
https://docs.sympy.org/latest/tutorials/intro-tutorial/basic_operations.html
https://docs.sympy.org/latest/modules/parsing.html
https://docs.sympy.org/latest/modules/core.html#sympy.core.sympify.sympify
https://docs.sympy.org/latest/tutorials/intro-tutorial/manipulation.html

If, when parsing, you want to use the expression that has been mapped to a variable you have to pass the dictionary that python uses to keep track of those mappings, i.e. locals()
>>> from sympy.abc import x
>>> from sympy import sympify, parse_expr
>>> y = x + 2
>>> sympify('y')
y
>>> sympify('y', locals=locals())
x + 2
>>> parse_expr('y', local_dict=locals())
x + 2

As suggested by Oscar Benjamin from the Sympy Google group, eval does the job
def verify_ratio(vstr1, vstr2):
"""Compare the result of two different computations of the same quantity"""
print(vstr1 + ' / ' + vstr2, '=', eval(vstr1 + ' / ' + vstr2))
return
What shows this would work is
>>> import sys
>>> import sympy as sp
>>> a, b = sp.symbols('a,b', real=True, positive=True)
>>> a is eval('a')
True
In my case, all expressions and symbols I am using in vstr1 and vstr2 are global.
If nesting within other functions, I might need to pass further parameters to verify_ratio, as in the solution by smichr.

How to specify derivative for SymPy custom function?

I am using sympy to help automate the process of finding equations of motion for some systems using the Euler Lagrange method. What would really make this easy is if I could define a function q and specify its time derivative qd --> d/dt(q) = qd. Likewise I'd like to specify d/dt(qd) = qdd. This is helpful because as part of the process of finding the equations of motion, I need to take derivatives (with respect to time, q, and qd) of expressions that are functions of q and qd. In the end I'll end up with an equation in terms of q, qd, and qdd and I'd like to be able to either print this neatly or use lambdify to convert this to a neat numpy function for use in a simulation.
Currently I've accomplished this is a very roundabout and annoying way by defining q as a function and qd as the derivative of that function:
q = sympy.Function('q', real=True)(t)
q_diff = diff(q,t)
This is fine for most of the process but then I end up with a messy expression filled with "Derivative (q, t)" and "Derivative (Derivative(q, t), t)" which is hard to wrangle into a neat printing format and difficult to turn into a numpy function using lambdify. My current solution is thus to use the subs function and replace q_diff and diff(q_diff, t) with sympy symbols qd and qdd respectively, which cleans things up and makes manipulating the expression much easier. This seems like a bad hack though and takes tons of time to do for more complicated equations with lots of state variables.
What I'd like is to define a function, q, with a specific value for the time derivative. I'd pass in that value when creating the function, and sympy could treat it like a generic Function object but would use whatever I'd given it for the time derivative instead of just saying "Derivative(q, t)". I'd like something like this:
qdd = sympy.symbols('qdd')
qd = my_func(name='qd', time_deriv=qdd)
q = my_func(name='q', time_deriv=qd)
diff(q, t)
>>> qd
diff(q**2, t)
>>> 2*q*qd
diff(diff(q**2, t))
>>> 2*q*qdd + 2*qd**2
expr = q*qd**2
expr.subs(q, 5)
>>> 5*qd**2
Something like that, where I could still use the subs command and lambdify command to substitute numeric values for q and qd, would be extremely helpful. I've been trying to do this but I don't understand enough of how the base sympy.Function class works to get this going. This is what I have right now:
class func(sp.Function):
def __init__(self, name, deriv):
self.deriv = deriv
self.name = name
def diff(self, *args, **kwargs):
return self.deriv
def fdiff(self, argindex=1):
assert argindex == 1
return self.deriv
This code so far does not really work, I don't know how to specify that specifically the time derivative of q is qd. Right now all derivatives of q are returning q?
I don't know if this is just a really bad solution, if I should be avoiding this issue entirely, or if there's already a clean way to solve this. Any advice would be very appreciated.

python sympy - how to process expressions in order to protect them from specific evaluations

sympy seems to evaluate expressions by default which is problematic in scenarios where automatic evaluation negatively impacts numerical stability. I need a way to control what gets evaluated to preserve that stability.
The only official mechanism I'm aware of is the UnevaluatedExpr class, but this solution is problematic for my purpose. Users of my code are not supposed to be burdened by any numerical stability considerations. They simply want to enter an expression and the code needs to do all the rest. Making them analyze the numerical stability of their own expressions is not an option. It needs to be done automatically.
First I tried to gain control of sympify() by monkeypatching it as it seems the main culprit behind most calls that lead to unwanted evaluation, but I only came as far as catchig all the calls, without being able to really control them the way I wanted. I bumped against so many walls there that I wouldn't even know where to start.
Modifying sympy itself, as you can probably imagine, is not an option either as I can't possibly require users to make some exotic patches of their local sympy installations.
Next I discovered that it's possible to say
with evaluate(False): doSomeStuffToExpression(expr)
This seems to violently shove evaluate=False down the throat of sympy no matter what.
However that means it radically deactivates all evaluation and does not allow any fine control.
Specifically I want to deactivate evaluation when there is an Add inside an sympy.exp
So the third attempt was to modify the expression tree. Basically developing a method that takes the expression, traverses it and automatically wraps args with UnevaluatedExpr where needed (remember: I can't bother the user with doing that manually)
So I wrote the following code to test the new apporach:
from sympy.core.expr import UnevaluatedExpr
from sympy.core.symbol import Symbol
import sympy as sp
from sympy.core.numbers import Float
x, z = sp.symbols('x z')
#expr = (x + 2.*x)/4. + sp.exp((x+sp.UnevaluatedExpr(32.))/6.)
expr = sp.sympify('(x + 2.*x)/4. + exp((x+32.)/6.)', evaluate=False)
expr_ = expr.subs(x, z)
print(expr)
print(expr_)
print('///////////\n')
def prep(expr, exp_depth = 0):
# once we are inside UnevaluatedExpr, we need to continue to traverse
# down to the Symbol and also wrap it with UnevaluatedExpr
if isinstance(expr, UnevaluatedExpr):
for arg in expr.args:
newargs = []
for arg_inside in arg.args:
if isinstance(arg_inside, Symbol) or isinstance(arg_inside, Float):
newargs.append(UnevaluatedExpr(arg_inside))
else:
newargs.append(arg_inside)
arg._args = tuple(newargs)
for arg_inside in arg.args:
prep(arg_inside, exp_depth = exp_depth + 1)
return
original_args = expr.args
# if args empty
if not original_args: return
# check if we just entered exp
is_exp = (expr.func == sp.exp)
print('\n-----')
print('expression\t\t-->', expr)
print('func || args\t\t-->', expr.func, ' || ', original_args)
print('is it exp right now?\t-->', is_exp)
print('inside exp?\t-->', exp_depth > 0)
# if we just received exp or if we are inside exp
if is_exp or exp_depth > 0:
newargs = []
for arg in original_args:
if isinstance(arg, sp.Add):
newargs.append(UnevaluatedExpr(arg))
else:
newargs.append(arg)
expr._args= tuple(newargs)
for arg in expr.args:
prep(arg, exp_depth = exp_depth + 1)
else:
for arg in original_args: prep(arg, exp_depth)
prep(expr)
print('///////////\n')
print(expr)
substituted = expr.subs(x, z)
print("substitution after prep still does not work:\n", substituted)
wewantthis = expr.subs(x, UnevaluatedExpr(z))
print("we want:\n", wewantthis)
print('///////////\n')
However the output was dissapointing as subs() triggers the dreaded evaluation again, despite wrapping args in UnevaluatedExpr where needed. Or let's say where I understood wrapping would be needed.
For some reason subs() completely ignores my changes.
So the question is: is there even any hope in this last approach (maybe I still missed something when traversing the tree) - and if there is no hope in my approach, how else should I achieve the goal of disabling evaluation of a specific Symbol when encountering an Add inside an sympy.exp (which is the exponential function)
PS:
I should probably also mention that for reasons that seem puzzling, the following works (but as I mentioned it's a manual solution that I don't desire)
expr = (x + 2.*x)/4. + sp.exp((x+sp.UnevaluatedExpr(32.))/6.)
expr_ = expr.subs(x, z)
print(expr)
print(expr_)
Here we successfully prevented the evaluation of the Add inside sp.exp
Output:
0.75*x + exp(0.166666666666667*(x + 32.0))
0.75*z + exp(0.166666666666667*(z + 32.0))
Edit 0:
The solution should permit the usage of floats. For example some of the values may describe physical properties, measured beyond the accuracy of an integer. I need to be able to allow those.
Substituting Floats with Symbols is also problematic as it substantially complicates handling of the expressions or the usage of those expressions at a later time

I'm not sure but I think that the problem you are having is to do with automatic distribution of a Number over an Add which is controlled by the distribute context manager:
In [326]: e1 = 2*(x + 1)
In [327]: e1
Out[327]: 2⋅x + 2
In [328]: from sympy.core.parameters import distribute
In [329]: with distribute(False):
...: e2 = 2*(x + 1)
...:
In [330]: e2
Out[330]: 2⋅(x + 1)
The automatic distribution behaviour is something that would be good to change in sympy. It's just not easy to change because it is such a low-level operation and it has been this way for a long time (it would break a lot of people's code).
Other parts of the evaluation that you see are specific to the fact that you are using floats and would not happen for Rational or for a symbol e.g.:
In [337]: exp(2*(x + 1))
Out[337]:
2⋅x + 2
ℯ
In [338]: exp(2.0*(x + 1))
Out[338]:
2.0⋅x
7.38905609893065⋅ℯ
In [339]: exp(y*(x + 1))
Out[339]:
y⋅(x + 1)
ℯ
You could convert rationals to float with nsimplify to avoid that e.g.:
In [340]: parse_expr('exp(2.0*(x + 1))', evaluate=False)
Out[340]:
2.0⋅(x + 1)
ℯ
In [341]: parse_expr('exp(2.0*(x + 1))', evaluate=False).subs(x, z)
Out[341]:
2.0⋅z
7.38905609893065⋅ℯ
In [342]: nsimplify(parse_expr('exp(2.0*(x + 1))', evaluate=False))
Out[342]:
2⋅x + 2
ℯ
In [343]: nsimplify(parse_expr('exp(2.0*(x + 1))', evaluate=False)).subs(x, z)
Out[343]:
2⋅z + 2
ℯ
Another possibility is to use symbols and delay substitution of any values until numerical evaluation. This is the way to get the most accurate result from evalf:
In [344]: exp(y*(z + 1)).evalf(subs={y:1, z:2})
Out[344]: 20.0855369231877

Approximating Numerical 2nd Derivative with Python

To preface this question, I understand that it could be done better. But this is a question in a class of mine and I must approach it this way. We cannot use any built in functions or packages.
I need to write a function to approximate the numerical value of the second derivative of a given function using finite difference. The function is below we are using.
2nd Derivative Formula (I lost the login info to my old account so pardon my lack of points and not being able to include images).
My question is this:
I don't understand how to make the python function accept the input function it is to be deriving. If someone puts in the input 2nd_deriv(2x**2 + 4, 6) I dont understand how to evaluate 2x^2 at 6.
If this is unclear, let me know and I can try again to describe. Python is new to me so I am just getting my feet wet.
Thanks

you can pass the function as any other "variable":
def f(x):
return 2*x*x + 4
def d2(fn, x0, h):
return (fn(x0+h) - 2*fn(x0) + fn(x0-h))/(h*h)
print(d2(f, 6, 0.1))

you can't pass a literal expression, you need a function (or a lambda).
def d2(f, x0, h = 1e-9):
func = f
if isinstance(f, str):
# quite insecure, use only with controlled input
func = eval ("lambda x:%s" % (f,))
return (func(x0+h) - 2*func(x0) + func(x0-h))/(2*h)
Then to use it
def g(x):
return 2*x**2 + 4
# using explicit function, forcing h value
print d2(g, 6, 1e-10)
Or directly:
# using lambda and default value for h
print d2(lambda x:2x**2+4, 6)
EDIT
updated to take into account that f can be a string or a function

lambdify a sympy expression that contains a Derivative of UndefinedFunction

I have several expressions of an undefined function some of which contain the corresponding (undefined) derivatives of that function. Both the function and its derivatives exist only as numerical data. I want to make functions out of my expressions and then call that function with the corresponding numerical data to numerically compute the expression. Unfortunately I have run into a problem with lambdify.
Consider the following simplified example:
import sympy
import numpy
# define a parameter and an unknown function on said parameter
t = sympy.Symbol('t')
s = sympy.Function('s')(t)
# a "normal" expression
a = t*s**2
print(a)
#OUT: t*s(t)**2
# an expression which contains a derivative
b = a.diff(t)
print(b)
#OUT: 2*t*s(t)*Derivative(s(t), t) + s(t)**2
# generate an arbitrary numerical input
# for demo purposes lets assume that s(t):=sin(t)
t0 = 0
s0 = numpy.sin(t0)
sd0 = numpy.cos(t0)
# labdify a
fa = sympy.lambdify([t, s], a)
va = fa(t0, s0)
print (va)
#OUT: 0
# try to lambdify b
fb = sympy.lambdify([t, s, s.diff(t)], b) # this fails with syntax error
vb = fb(t0, s0, sd0)
print (vb)
Error message:
File "<string>", line 1
lambda _Dummy_142,_Dummy_143,Derivative(s(t), t): (2*_Dummy_142*_Dummy_143*Derivative(_Dummy_143, _Dummy_142) + _Dummy_143**2)
^
SyntaxError: invalid syntax
Apparently the Derivative object is not resolved correctly, how can I work around that?
As an alternative to lambdify I'm also open to using theano or cython based solutions, but I have encountered similar problems with the corresponding printers.
Any help is appreciated.

As far as I can tell, the problem originates from an incorrect/unfortunate dummification process within the lambdify function. I have written my own dummification function that I apply to the parameters as well as the expression before passing them to lambdifying.
def dummify_undefined_functions(expr):
mapping = {}
# replace all Derivative terms
for der in expr.atoms(sympy.Derivative):
f_name = der.expr.func.__name__
var_names = [var.name for var in der.variables]
name = "d%s_d%s" % (f_name, 'd'.join(var_names))
mapping[der] = sympy.Symbol(name)
# replace undefined functions
from sympy.core.function import AppliedUndef
for f in expr.atoms(AppliedUndef):
f_name = f.func.__name__
mapping[f] = sympy.Symbol(f_name)
return expr.subs(mapping)
Use like this:
params = [dummify_undefined_functions(x) for x in [t, s, s.diff(t)]]
expr = dummify_undefined_functions(b)
fb = sympy.lambdify(params, expr)
Obviously this is somewhat brittle:
no guard against name-collisions
perhaps not the best possible name-scheme: df_dxdy for Derivative(f(x,y), x, y)
it is assumed that all derivatives are of the form:
Derivative(s(t), t, ...) with s(t) being an UndefinedFunction and t a Symbol. I have no idea what will happen if any argument to Derivative is a more complex expression. I kind of think/hope that the (automatic) simplification process will reduce any more complex derivative into an expression consisting of 'basic' derivatives. But I certainly do not guard against it.
largely untested (except for my specific use-cases)
Other than that it works quite well.

First off, rather than an UndefinedFunction, you could go ahead and use the implemented_function function to tie your numerical implementation of s(t) to a symbolic function.
Then, if you are constrained to discrete numerical data defining the function whose derivative occurs in the troublesome expression, much of the time, the numerical evaluation of the derivative may come from finite differences. As an alternative, sympy can automatically replace derivative terms with finite differences, and let the resulting expression be converted to a lambda. For example:
import sympy
import numpy
from sympy.utilities.lambdify import lambdify, implemented_function
from sympy import Function
# define a parameter and an unknown function on said parameter
t = sympy.Symbol('t')
s = implemented_function(Function('s'), numpy.cos)
print('A plain ol\' expression')
a = t*s(t)**2
print(a)
print('Derivative of above:')
b = a.diff(t)
print(b)
# try to lambdify b by first replacing with finite differences
dx = 0.1
bapprox = b.replace(lambda arg: arg.is_Derivative,
lambda arg: arg.as_finite_difference(points=dx))
print('Approximation of derivatives:')
print(bapprox)
fb = sympy.lambdify([t], bapprox)
t0 = 0.0
vb = fb(t0)
print(vb)

The similar question was discussed at here
You just need to define your own function and define its derivative as another function:
def f_impl(x):
return x**2
def df_impl(x):
return 2*x
class df(sy.Function):
nargs = 1
is_real = True
_imp_ = staticmethod(df_impl)
class f(sy.Function):
nargs = 1
is_real = True
_imp_ = staticmethod(f_impl)
def fdiff(self, argindex=1):
return df(self.args[0])
t = sy.Symbol('t')
print f(t).diff().subs({t:0.1})
expr = f(t) + f(t).diff()
expr_real = sy.lambdify(t, expr)
print expr_real(0.1)