We Keep Coding

Optimize sympy expression evaluation by combining as many free symbols as possible

Imagine I have an unknown, quite complex expression that I need to repeatedly evaluate numerically, e.g.:
my_expr = (a*b*c**2 - 2*sqrt(d*(a*b-c-e+x)))/(b - 1)
Each time I reevaluate the expression, the only symbol that changes is 'x', so it makes sense for me to precompute all the others (I will be using c code generation eventually).
So what I want is to automatically pull out and combine as many free symbols as possible in advance, except for x. This would work a bit like cse, but making the final expression contain as few calculations as possible.
e.g. for the above I might end up with a system equivalent to this:
var1 = a*b*c**2
var2 = a*b-c-e
var3 = b - 1
my_new_expr = (var1-2*sqrt(d*(var2+x)))/var3
This means I can precalculate var1,var2 & var3, and the repeated calculation (my_new_expr) is as simple as possible computationally.
Is there anyway I can do this in sympy? I've looked through all the simplification methods etc, including collect etc, and none quite do what I need. Failing that, is there any traversal of the expression I could do to achieve this?

Although my model branch at sympy/smichr has a more comprehensive solution, the following will do pretty well at condensing those sub-expressions that are constant:
def condense(eq, *x):
"""collapse additive/multiplicative constants into single
variables, returning condensed expression and replacement
values.
Examples
========
Simple constants are left unchanged
>>> condense(2*x + 2, x)
(2*x + 2, {})
More complex constants are replaced by a single variable
>>> first = condense(eq, x); first
(c6*(c5 - 2*sqrt(d*(c4 + x))), {c4: a*b - c - e, c6: 1/(b - 1), c5: a*b*c**2})
If a condensed expression is expanded, there may be more simplification possible:
>>> second = condense(first[0].expand(), x); second
(c0 + c2*sqrt(c1 + d*x), {c1: c4*d, c2: -2*c6, c0: c5*c6})
>>> full_reps = {k: v.xreplace(first[1]) for k, v in second[1].items()}; full_reps
{c1: d*(a*b - c - e), c2: -2/(b - 1), c0: a*b*c**2/(b - 1)}
More than 1 variable can be designated:
>>> condense(eq, c, e)
(c4*(c**2*c1 - 2*sqrt(d*(-c + c2 - e))), {c4: 1/(b - 1), c1: a*b, c2: a*b + x})
"""
reps = {}
con = numbered_symbols('c')
free = eq.free_symbols
def c():
while True:
rv = next(con)
if rv not in free:
return rv
def do(e):
i, d = e.as_independent(*x)
if not i.args: return e
return e.func(reps.get(i, reps.setdefault(i, c())), d)
rv = eq.replace(lambda x: x.is_Add or x.is_Mul, lambda x: do(x))
reps = {v: k for k, v in reps.items()}
keep = rv.free_symbols & set(reps)
reps = {k: reps[k].xreplace(reps) for k in keep}
return rv, reps

Sympy Subs, Replace, and Xreplace not working with expressions

I am trying to substitute an expression that is the output of a solved system of equations and am getting errors.
q,a,b,c=symbols('q,a,b,c')
def p(q):
return a-b*q
some_equation=eqn1-eqn2
new_equation=sol(some_equation,q)
New equation returns
[(a-c)/(2*b)]
when I use p(q).subs(q,new_equation) I get p(q) and return gives me a type error no matter what I do. Xreplace throws a Sympify error. Any suggestions to get what should be a simple sub to work???

Sympy's solve always returns a Python list of solutions. There can be zero, one or multiple solutions to a general equation. subs needs one concrete value from the list, it can not work with the list as a whole. Therefore, you need to iterate through the list to do the substitutions. If you are certain that there is only one solution, you can just use solutions[0]:
from sympy import symbols, solve, Eq
q, a, b, c = symbols('q a b c')
def p(q):
return a - b * q
some_equation = p(q) - c
solutions = solve(some_equation, q)
print("solutions:", solutions)
for sol in solutions:
print("solution for q =", sol, " --> p(q) =", p(q).subs(q, sol))
Output:
solutions: [(a - c)/b]
solution for q = (a - c)/b --> p(q) = c
Note that instead of writing p as a function of q, you can also write it directly as a sympy expression: p = a - b * q. Also note that although you can write the equation as some_equation = p(q) - c, sympy's canonical way to write such equations is some_equation = Eq(p(q), c).
Here is another example, which has two solutions. It also uses simplify() because more complicated expressions by default are only simplified just a little bit.
p = a * q ** 2 + b * q
some_equation = Eq(p, c)
solutions = solve(some_equation, q)
print("solutions:", solutions)
for sol in solutions:
print("solution for q =", sol, " --> p(q) =", p.subs(q, sol).simplify())
Output:
solutions: [(-b + sqrt(4*a*c + b**2))/(2*a), -(b + sqrt(4*a*c + b**2))/(2*a)]
solution for q = (-b + sqrt(4*a*c + b**2))/(2*a) --> p(q) = c
solution for q = -(b + sqrt(4*a*c + b**2))/(2*a) --> p(q) = c

Could I get a clarification to this Python code below?

I'm a beginner to Python and I'm trying to calculate the angles (-26.6 &18.4) for this figure below and so on for the rest of the squares by using Python code.
I have found the code below and I'm trying to understand very well. How could it work here? Any clarification, please?
Python Code:
def computeDegree(a,b,c):
babc = (a[0]-b[0])*(c[0]-b[0])+(a[1]-b[1])*(c[1]-b[1])
norm_ba = math.sqrt((a[0]-b[0])**2 + (a[1]-b[1])**2)
norm_bc = math.sqrt((c[0]-b[0])**2 + (c[1]-b[1])**2)
norm_babc = norm_ba * norm_bc
radian = math.acos(babc/norm_babc)
degree = math.degrees(radian)
return round(degree, 1)
def funcAngle(p, s, sn):
a = (s[0]-p[0], s[1]-p[1])
b = (sn[0]-p[0], sn[1]-p[1])
c = a[0] * b[1] - a[1] * b[0]
if p != sn:
d = computeDegree(s, p, sn)
else:
d = 0
if c > 0:
result = d
elif c < 0:
result = -d
elif c == 0:
result = 0
return result
p = (1,4)
s = (2,2)
listSn= ((1,2),(2,3),(3,2),(2,1))
for sn in listSn:
func(p,s,sn)
The results
I expected to get the angles in the picture such as -26.6, 18.4 ...

Essentially, this uses the definition of dot products to solve for the angle. You can read more it at this link (also where I found these images).
To solve for the angle you first need to convert your 3 input points into two vectors.
# Vector from b to a
# BA = (a[0] - b[0], a[1] - b[1])
BA = a - b
# Vector from b to c
# BC = (a[0] - c[0], a[1] - c[1])
BC = c - b
Using the two vectors you can then find the angle between them by first finding the value of the dot product with the second formula.
# babc = (a[0]-b[0])*(c[0]-b[0])+(a[1]-b[1])*(c[1]-b[1])
dot_product = BA[0] * BC[0] + BA[1] * BC[1]
Then by going back to the first definition, you can divide off the lengths of the two input vectors and the resulting value should be the cosine of the angle between the vectors. It may be hard to read with the array notation but its just using the Pythagoras theorem.
# Length/magnitude of vector BA
# norm_ba = math.sqrt((a[0]-b[0])**2 + (a[1]-b[1])**2)
length_ba = math.sqrt(BA[0]**2 + BA[1]**2)
# Length/magnitude of vector BC
# norm_bc = math.sqrt((c[0]-b[0])**2 + (c[1]-b[1])**2)
length_bc = math.sqrt(BC[0]**2 + BC[1]**2)
# Then using acos (essentially inverse of cosine), you can get the angle
# radian = math.acos(babc/norm_babc)
angle = Math.acos(dot_product / (length_ba * length_bc))
Most of the other stuff is just there to catch cases where the program might accidentally try to divide by zero. Hopefully this helps to explain why it looks the way it does.
Edit: I answered this question because I was bored and didn't see harm in explaining the math behind that code, however in the future try to avoid asking questions like 'how does this code work' in the future.

Let's start with funcAngle since it calls computeDegree later.
The first thing it does is define a as a two item tuple. A lot of this code seems to use two item tuples, with the two parts referenced by v[0] and v[1] or similar. These are almost certainly two dimensional vectors of some sort.
I'm going to write these as 𝐯 for the vector and vₓ and vᵧ since they're probably the two components.
[don't look too closely at that second subscript, it's totally a y and not a gamma...]
a is the vector difference between s and p: i.e.
a = (s[0]-p[0], s[1]-p[1])
is aₓ=sₓ-pₓ and aᵧ=sᵧ-pᵧ; or just 𝐚=𝐬-𝐩 in vector.
b = (sn[0]-p[0], sn[1]-p[1])
again; 𝐛=𝐬𝐧-𝐩
c = a[0] * b[1] - a[1] * b[0]
c=aₓbᵧ-aᵧbₓ; c is the cross product of 𝐚 and 𝐛 (and is just a number)
if p != sn:
d = computeDegree(s, p, sn)
else:
d = 0
I'd take the above in reverse: if 𝐩 and 𝐬𝐧 are the same, then we already know the angle between them is zero (and it's possible the algorithm fails badly) so don't compute it. Otherwise, compute the angle (we'll look at that later).
if c > 0:
result = d
elif c < 0:
result = -d
elif c == 0:
result = 0
If c is pointing in the normal direction (via the left hand rule? right hand rule? can't remember) that's fine: if it isn't, we need to negate the angle, apparently.
return result
Pass the number we've just worked out to some other code.
You can probably invoke this code by adding something like:
print (funcangle((1,0),(0,1),(2,2))
at the end and running it. (Haven't actually tested these numbers)
So this function works out a and b to get c; all just to negate the angle if it's pointing the wrong way. None of these variables are actually passed to computeDegree.
so, computeDegree():
def computeDegree(a,b,c):
First thing to note is that the variables from before have been renamed. funcAngle passed s, p and sn, but now they're called a, b and c. And the note the order they're passed in isn't the same as they're passed to funcAngle, which is nasty and confusing.
babc = (a[0]-b[0])*(c[0]-b[0])+(a[1]-b[1])*(c[1]-b[1])
babc = (aₓ-bₓ)(cₓ-bₓ)+(aᵧ-bᵧ)(cᵧ-bᵧ)
If 𝐚' and 𝐜' are 𝐚-𝐛 and 𝐜-𝐛 respectively, this is just
a'ₓc'ₓ+a'ᵧc'ᵧ, or the dot product of 𝐚' and 𝐜'.
norm_ba = math.sqrt((a[0]-b[0])**2 + (a[1]-b[1])**2)
norm_bc = math.sqrt((c[0]-b[0])**2 + (c[1]-b[1])**2)
norm_ba = √[(aₓ-bₓ)² + (aᵧ-bᵧ)²] (and norm_bc likewise).
This looks like the length of the hypotenuse of 𝐚' (and 𝐜' respectively)
norm_babc = norm_ba * norm_bc
which we then multiply together
radian = math.acos(babc/norm_babc)
We use the arccosine (inverse cosine, cos^-1) function, with the length of those multiplied hypotenuses as the hypotenuse and that dot product as the adjacent length...
degree = math.degrees(radian)
return round(degree, 1)
but that's in radians, so we convert to degrees and round it for nice formatting.
Ok, so now it's in maths, rather than Python, but that's still not very easy to understand.
(sidenote: this is why descriptive variable names and documentation is everyone's friend!)

Is it possible to minimise with PyMinuit using a dictionary for parameter reference

Is it possible to carry out a PyMinuit function minimisation by passing a dictionary of parameters to the minimiser?
For example, the usual use of PyMinuit would be called using something like:
def f(x, a, b): return a + b*x
def chi2(a,b):
c2 = 0.
for x, y, yerr in data:
c2 += (f(x, a, b) - y)**2 / yerr**2
return c2
m = minuit.Minuit(chi2)
m.migrad()
From this question, I understand PyMinuit uses introspection to determine the parameters x and y (but I am not entirely sure what that means). Ideally, I would like to be able to do something like:
p = dict()
p['x'] = 0.
p['y'] = 0.
def f(x,a,b): return a + b*x
def chi2():
c2 = 0.
for x, y, yerr in data:
c2 += (f(x, a, b) - y)**2 / yerr**2
return c2
m = minuit.Minuit(chi2,**p)
m.migrad()
or even:
p = <dictionary of parameters + initial values>
model = <list containing strings representing functions e.g. 'a*b+a**2*x'>
data = x, y, yerr, model
def chi2():
c2 = 0.
for x, y, yerr, model in data:
c2 += (eval(model,{"__builtins__":None},p) - y)**2 / yerr**2
return c2
m = minuit.Minuit(chi2)
m.migrad()
I saw a work-around to a similar problem on the google groups issues page where they generated 'fake code' and 'fake functions' from an integer input (follow link to see). I tried something similar with my dictionary p:
class fake_code:
def __init__(self,p):
self.co_argcount = len(p)
self.co_varnames = tuple(p.keys())
print tuple(p.keys())
class fake_function:
def __init__(self,p):
self.func_code = fake_code(p)
def __call__(self,*args):
c2 = 0.
print args
for x, y, yerr in data:
c2 += (f(x, a, b) - y)**2 / yerr**2
return c2
but for some reason all the parameters are classed as 'fixed' and I can't seem to 'unfix' them.
I think it should be possible to do it this way, but I do not know enough about python to say if this is the best way, or even if it should be attempted. If anyone can shed some light onto this I'd be grateful to know. :)

OK, I don't like answering my own questions, but I think I've found a solution using exec. If one defines the chi2 function in a template and builds it at run-time with a function make_chi_squared then it is possible. The solution I've managed to come up with is shown below.
import minuit
import numpy
chi_squared_template = """
def chi_squared(%(params)s):
li = [%(params)s]
for i,para in enumerate(li):
p[l[i]] = para
return (((f(data_x, p) - data_y) / errors) ** 2).sum()
"""
l = ['a1','a2','a3','a4']
p = dict()
p['a1'] = 1.
p['a2'] = 1.
p['a3'] = 1.
p['a4'] = 1.
def make_chi_squared(f, data_x, data_y, errors):
params = ", ".join(l)
exec chi_squared_template % {"params": params}
return chi_squared
def f(x,p):
return eval('a1 + a2*x + a3*x**2 + a4*x**3',
{"__builtins__":locals()},
p)
data_x = numpy.arange(50)
errors = numpy.random.randn(50) * 0.3
data_y = data_x**3 + errors
chi_squared = make_chi_squared(f, data_x, data_y, errors)
m = minuit.Minuit(chi_squared)
m.printMode = 1
m.migrad()
print m.values
p = m.values
print p
It's a bit messy, and I'm not sure if its the best way of handling this type of problem, but it works!

This following is largely untested, which I usually try to avoid doing, but am making an exception to better explain to you the simplified way I referred to in my comments that might work for this. It's based on the first example shown here.
import minuit
def minuit_call(func, **kwargs):
CALL_TEMPLATE = "minuit.Minuit({0.__name__}, {1})"
arg_str = ', '.join('{}={}'.format(k, v) for k,v in kwargs.iteritems())
return eval(CALL_TEMPLATE.format(func, arg_str))
def f(x, y):
return ((x-2) / 3)**2 + y**2 + y**4
m = minuit_call(f, x=0, y=0)
m.migrad()
As you can see, the template used is fairly trivial and creating it didn't require manually translating any of the code in the body of the function to be minimization into a formatting string.

Might be late for answer. Try this out iminuit. I wrote it because of the lack of this specific feature among others.
http://iminuit.github.com/iminuit/
See example how you would write a generic cost function here:
http://nbviewer.ipython.org/urls/raw.github.com/iminuit/iminuit/master/tutorial/hard-core-tutorial.ipynb
However, although it's easy to write a chi^2/likelihood function, it's already written for you in probfit
http://iminuit.github.com/probfit/

efficiently computing parafac / CP product in numpy

This question focuses on numpy.
I have a set of matrices which all share the same number of columns and have different number of rows. Let's call them A, B, C, D, etc and let their dimensions be IaxK IbxK, IcxK, etc
What I want is to efficiently compute the IaxIbxIc... tensor P defined as follow:
P(ia,ib,ic,id,ie,...)=\sum_k A(ia,k)B(ib,k)C(ic,k)...
So if I have two factors, I end up with simple matrix product.
Of course I can compute this "by hand" through outer products, something like:
def parafac(factors,components=None):
ndims = len(factors)
ncomponents = factors[0].shape[1]
total_result=array([])
if components is None:
components=range(ncomponents)
for k in components:
#for each component (to save memory)
result = array([])
for dim in range(ndims-1,-1,-1):
#Augments model with next dimension
current_dim_slice=[slice(None,None,None)]
current_dim_slice.extend([None]*(ndims-dim-1))
current_dim_slice.append(k)
if result.size:
result = factors[dim].__getitem__(tuple(current_dim_slice))*result[None,...]
else:
result = factors[dim].__getitem__(tuple(current_dim_slice))
if total_result.size:
total_result+=result
else:
total_result=result
return total_result
Still, I would like something much more computationally efficient, like relying on builtin numpy functions, but I cannot find relevant functions, can someone help me ?
Cheers, thanks

Thank you all very much for your answers, I've spent the day on this and I eventually found the solution, so I post it here for the record
This solution requires numpy 1.6 and makes use of einsum, which is
powerful voodoo magic
basically, if you had factor=[A,B,C,D] with A,B,C and D matrices with
the same number of columns, then you would compute the parafac model using:
import numpy
P=numpy.einsum('az,bz,cz,dz->abcd',A,B,C,D)
so, one line!
In the general case, I end up with this:
def parafac(factors):
ndims = len(factors)
request=''
for temp_dim in range(ndims):
request+=string.lowercase[temp_dim]+'z,'
request=request[:-1]+'->'+string.lowercase[:ndims]
return einsum(request,*factors)

Having in mind that outer product is Kronecker product in disguise your problem should be solved by this simple functions:
def outer(vectors):
shape=[v.shape[0] for v in vectors]
return reduce(np.kron, vectors).reshape(shape)
def cp2Tensor(l,A):
terms=[]
for r in xrange(A[0].shape[1]):
term=l[r]*outer([A[n][:,r] for n in xrange(len(A))])
terms.append(term)
return sum(terms)
cp2Tensor takes list of real numbers and list of matrices.
Edited after comment by Jaime.

Ok, so the following works. First a worked out example of what's going on...
a = np.random.rand(5, 8)
b = np.random.rand(4, 8)
c = np.random.rand(3, 8)
ret = np.ones(5,4,3,8)
ret *= a.reshape(5,1,1,8)
ret *= b.reshape(1,4,1,8)
ret *= c.reshape(1,1,3,8)
ret = ret.sum(axis=-1)
And a full function
def tensor(elems) :
cols = elems[0].shape[-1]
n_elems = len(elems)
ret = np.ones(tuple([j.shape[0] for j in elems] + [cols]))
for j,el in enumerate(elems) :
ret *= el.reshape((1,) * j + (el.shape[0],) +
(1,) * (len(elems) - j - 1) + (cols,))
return ret.sum(axis=-1)