I have no idea how or where to start, I need to have some vocabulary or terms to get going and researching so let's ask the community.
Problem: I have a value X that is the final answer of the equation. I have a range of values, let's say 10 (A-J), that will be used in the equation. Using simple calculations (+, -, *, /, (), ^) to form an equation that uses some or all of the values A-J to solve the equation to the value X.
Example: A^2+B*C+(D+E+F+G)*J+30 = X
Input: final value X, the values used in the equation, max number of values to be used in the equation meaning I want to use at least Z number of given values in the equation (in the example Z = 8)
Output: the equation that solves it with the given values
Can this be turned into a python script for example? Is it possible at all to calculate this way? What are the terms that describe this kind of calculations?
If I understand the question correctly:
The algorithm you are looking for outputs a mathematical function, along with the specific values that, when applied to the function, give you the input value x, for an arbitrary value x.
In general I believe this is not possible. It may be possible from a non-deterministic point of view, where you try and guess values but that usually isn't feasible from an algorithmic or computational standpoint.
Let's first limit the function to one variable. Finding a function that gives you a value x, for some value a, i.e. f(a) = x, is the same as asking for f(a) - x = 0.
Limiting the operations to +,-,*, we see that f(a) is a polynomial.
Limiting the problem in this way relates the solution to algebraic numbers and constructible numbers. The more general mathematical theory that explain properties of these numbers and functions is called Galois Theory.
It is possible to find the polynomial of a given the input value x IF and only IF the input value x is algebraic. You can produce a simple algorithm that takes powers of x until that power of x is an integer (or a rational number).
You can produce a simple algorithm that takes powers of the irrational part of x until that power of x is an integer (or a rational number). Note that even in this algorithm you would need to take into account some sort of measurement error, because sqrt(2) = 1.41421356237... for an infinite number of decimal places, and the computer can only keep track of some finite amount of decimal places. For example:
def sqrt(x):
return x**(1/2)
if __name__ == "__main__":
num = sqrt(2)
print(num)
print(num**2)
Will output:
1.4142135623730951
2.0000000000000004
So the output of such a simple algorithm will never be 100% correct. It is possible for a human to do, but not a computer. You may want to look into the field of symbolic computation, but the algorithm for solving even part of your problem will not be easy to turn into a script.
If you are okay with not solving this problem 100% of the time, you may want to look into linear approximations and non-linear approximations.
The reason why even without measurement errror, I believe this is not possible in general is that adding the operations (^,/) may not result in a polynomial ring. Which is the basis of solving problems relating to algebraic numbers.
Introducing extra variables b,c,....,n,.. to f, such that f(a,b,c,...,n,...) = x would also restrict what functions would satisfy the properties of a ring.
Regardless, this problem is difficult, even when only considering polynomials of one variable. Googling the terms in bold may help provide additional insight into your problem. Sorry if some of this was already obvious for you and in any case I hope this helps!
Related
I am trying to solve an equation that can include truncations in Python with a numerical approach. I am wondering what the best library and approach would be? Following is more detail about the problem:
The equation changes every time. From a human perspective, the equations should be pretty simple; they include common operators such as +,-,*,/, and they also sometimes have truncation functions (truncate to integer) or limit functions (limit the value in parenthesis between two provided bounds) or (rarely) multiple variables. A couple of examples (with these being separate examples and not a system of equations) would be:
TRUNCATE(VAR_1 + 300) - 50.4 = 200
(VAR_2 + VAR_3)*3 = 35
LIMIT(3,5)(VAR_4) = 8
VAR_5 = 34
(This is not exactly what the equations look like, because I am writing them in postfix notation, but I have a calculator to determine their value with provided input values.)
All I need for these equations is some value for each variable that would solve each equation; I do not need to know every solution.
Some additional things to note about this is a) these variables all have maximum and minimum values, b) while perfection would be nice, occasional errors are acceptable, and c) some of the variables are integers, which I expect really complicates things. Right now, I'm handling this very sloppily but also mostly acceptably for my case by rounding the integer values to the nearest int.
In an attempt to solve this problem, I tried solving analytically with Sympy (which as you might expect didn't work on truncations and was difficult to implement), and I also tried using Scipy minimize as follows:
minimize(minimization, x0, method = 'SLSQP', constraints = cons, tol = 1e-3, options={'ftol': 1e-3, 'disp':True, 'maxiter': 100, "eps":.1}, args = (x_vals, postfix, const_values, value))
This one got stuck on truncations, presumably because it didn't know what direction to move, unless I set the step to 1, which decreased accuracy. For some reason, it also didn't seem to follow the ftol, because it would give acceptable answers within the tolerance but would just keep going to the iteration limit.
I am considering using something that does random walks like the "Markov Chain Monte Carlo" method, but I really don't know much about this and was eager to hear other thoughts.
I ended up solving the problem two slightly different ways. Both of them used the Powell solver as suggested by joni in the comments on the original question, and for both of them I had to multiply the output of the function that gets passed to the "fun" parameter (a function that I named minimize) by 100, because I could never get the tolerance adjusted in the solver function call.
When the equation had only one variable, I removed the truncation from the minimize function. This worked for my purposes because the reason the equations I was using was being truncated was so they would equal an integer value (generally). So, when the equation output is an integer and there is only one variable, I believe the correct solution will be obtained by just pretending the truncation function does not exist in the solver (though remember to be wary of floating point math). (And if any numbers outside of the truncation are integers, the equation may not have a solution anyways.)
In cases with multiple variables, my solution was to a) include the truncation function in the minimize function and b) round the x values suggested by the solver as I planned to round them in the end (ex. round them to an integer if they were an integer value).
Anyways, this solution worked for the problem defined above, but it otherwise has some limitations. It is not guaranteed to always find the correct output, especially the second part. Another approach people with this problem may wish to consider would be some sort of integer programming, if they have linear equations.
How can the characteristic polynomial of a binary matrix (one with only zeros and ones) be found programmatically, where the process operates in the finite field F2 (also known as GF(2)) and the coefficients are zeros and ones?
Here's what I have tried:
SymPy's charpoly() method doesn't give the answer I want, since it doesn't operate on the field F2 and gives a polynomial with coefficients well beyond 0 and 1. However, is it possible to adapt the output of charpoly() to return the characteristic polynomial over F2, or to have the charpoly() method operate on that field?
This repository is about the most convenient thing I could find that could solve this question. As of this writing I am trying it out now. However, it is very slow (is on track to take many hours) for the sizes of matrices I am interested in (128x128 to 256x256). Moreover, I had to modify the source code to fit my needs since the code, as is, doesn't take arbitrary matrices.
I am asking this question because finding the characteristic polynomial in F2 is part of the process of calculating the appropriate jump parameter for certain random number generators (see my note on this).
As it turns out, the coefficients of the characteristic polynomial returned by charpoly() can be adapted for the GF(2) finite field, and it's easy to do: odd coefficients become ones, and even coefficients become zeros. And this is enough for my purposes. Therefore, my issue is solved.
I have a lot of 10 by 10 (0,1)-matrices and I would like to determine which have determinant exactly 0 (that is which are singular). Using scipy.linalg.det I get a floating point number which I have to test to see if it is close to zero. Is it possible to do the calculation exactly so I can be sure I am not finding false positives?
On the other hand, maybe there is some guarantee about the smallest eigenvalue which can be used to make sure the floating point method never makes a false positive?
As long as you are using floats, you can not guarantee you will get exactly zero. I would use this:
scipy.allclose(det, 0)
You can control the tolerance with the kwargs.
In your case (10x10 matrices with 0,1 elements) you shouldn't have to worry about false positives.
I don't have a proof for this, but it's just geometrical intuition: the group of 10-vectors with 0/1 elements can not be "nearly" linearly dependent in the way that would be necessary for you to get a false positive using floats. As vectors their "directions"† are necessarily discrete/atomic if elements are in 0,1.
† Think of the 3D case and generalise your thoughts to 10-dimensional space ;)
You can use Gaussian elimination to bring the matrix to a triangular form.
Since your elements are all 0 or 1, the calculation even using floating point arithmetic will be exact (you are only multiplying/dividing/adding/subtracting by -1, 0 and 1, which is exact).
The determinant is then 0 if one element of the diagonal is zero and nonzero otherwise.
So for this specific algorithm (Gaussian elimination), calculation of the determinant will be exact even in floating point arithmetic.
This algorithm also should be pretty efficient. It can even be implemented using integers, which is faster and shows even in a more obvious way that the problem is exactly solvable.
EDIT: the point is, that an algorithm which operates on the 0,1 matrix can be exact. It depends on the algorithm. I would check how det() is implemented and maybe, there is no issue with numerical noise, and, in fact, you could just test for det(M) == 0.0 and get neither false negatives nor false positives.
I think you should consider examining the condition number rather than the determinant. In python you'd want
numpy.linalg.cond(x, p=None)
Reference http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.cond.html.
That was the advice from the applied maths prof on the coursera course on scientific computing. Essentially the condition number will give you the best indication of numerical instability for operations like inverting a matrix etc, which is probably what you're interested in. See this answer on scicomp stackexchange for full details.
how about test batches while playing with the tolerance arg, then decide on the max acceptable tolerance, rinse and repeat:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.allclose.html
As the entries in the matrices are either 1 or 0 the smallest non-zero absolute value of a determinant is 1. So there is no need to fear a true non-zero value that is very close to 0.
Alternatively one can apparently use sympy to get an exact answer.
I have a graph between 2 functions f and g.
I know it follows a power law function with exponential cutoff.
f(x) = x**(-alpha)*e**(-lambda*x)
How do I find the value of exponent alpha?
If you have sufficiently close x points (for example one every 0.1), you can try the following:
ln(f(x)) = -alpha ln(x) - lambda x
ln(f(x))' = - alpha / x - lambda
So depending on where you have your points:
If you have a lot of points near 0, you can try:
h(x) = x ln(f(x))' = -alpha - lambda x
So the limit of the function h when x goes to 0 is -alpha
If you have large values of x, the function x -> ln(f(x))' tends toward lambda when x goes to infinity, so you can guess lambda and use pwdyson's expression.
If you don't have close x points, the numerical derivative will be very noisy, so I would try to guess lambda as the limit of -ln(f(x)/x for large x's...
If you don't have large values, but a large number of x's, you can try a minimization of
sum_x_i (ln(y_i) + alpha ln(x_i) + lambda x_i) ^2
on both alpha and lambda (I guess It would be more precise than the initial expression)...
It is a simple least square regression (numpy.linalg.lstsq will do the job).
So you have plenty of methods, the one to chose really depends on you inputs.
The usual and general way of doing what you want is to perform a non-linear regression (even though, as noted in another response, it is possible to linearize the problem). Python can do this quite easily with the help of the SciPy package, which is used by many scientists.
The routine you are looking for is its least-square optimization routine (scipy.optimize.leastsq). Once you wrap your head around the way this general optimization procedure works (see the example), you will probably find many other opportunities to use it. Basically, you calculate the list of differences between your measurements and their ideal value f(x), and you ask SciPy to find the parameters that make these differences as small as possible, so that your data fits the model as well as possible. This then gives you the parameter you are looking for.
It sounds like you might be trying to fit a power-law to a distribution with an exponential cutoff at the low end due to incompleteness - but I may be reading too far into your problem.
If that is the problem you're dealing with, this website (and accompanying publication) addresses the issue: http://tuvalu.santafe.edu/~aaronc/powerlaws/. I wrote the python implementation of the power-law fitter on that page; it is linked from there.
If you know that the points follow this law exactly, then invert the equation and put in an x and its corresponding f(x) value:
import math
alpha = -(lambda*x + math.log(f(x)))/math.log(x)
But the if the points do not exactly fit the equation you will need to do some sort of regression to determine alpha.
EDIT: Ok, so they don't fit exactly. This is getting beyond a Python question, but there may be something in numpy that can handle it. Here is a numpy linear regression recipe but your equation can't be rearranged into a linear form, so you'll have to look into non-linear regression.
I am trying to numerically integrate an arbitrary (known when I code) function in my program
using numerical integration methods. I am using Python 2.5.2 along with SciPy's numerical integration package. In order to get a feel for it, i decided to try integrating sin(x) and observed this behavior-
>>> from math import pi
>>> from scipy.integrate import quad
>>> from math import sin
>>> def integrand(x):
... return sin(x)
...
>>> quad(integrand, -pi, pi)
(0.0, 4.3998892617846002e-14)
>>> quad(integrand, 0, 2*pi)
(2.2579473462709165e-16, 4.3998892617846002e-14)
I find this behavior odd because -
1. In ordinary integration, integrating over the full cycle gives zero.
2. In numerical integration, this (1) isn't necessarily the case, because you may just be
approximating the total area under the curve.
In any case, either assuming 1 is True or assuming 2 is True, I find the behavior to be inconsistent. Either both integrations (-pi to pi and 0 to 2*pi) should return 0.0 (first value in the tuple is the result and the second is the error) or return 2.257...
Can someone please explain why this is happening? Is this really an inconsistency? Can someone also tell me if I am missing something really basic about numerical methods?
In any case, in my final application, I plan to use the above method to find the arc length of a function. If someone has experience in this area, please advise me on the best policy for doing this in Python.
Edit
Note
I already have the first differential values at all points in the range stored in an array.
Current error is tolerable.
End note
I have read Wikipaedia on this. As Dimitry has pointed out, I will be integrating sqrt(1+diff(f(x), x)^2) to get the Arc Length. What I wanted to ask was - is there a better approximation/ Best practice(?) / faster way to do this. If more context is needed, I'll post it separately/ post context here, as you wish.
The quad function is a function from an old Fortran library. It works by judging by the flatness and slope of the function it is integrating how to treat the step size it uses for numerical integration in order to maximize efficiency. What this means is that you may get slightly different answers from one region to the next even if they're analytically the same.
Without a doubt both integrations should return zero. Returning something that is 1/(10 trillion) is pretty close to zero! The slight differences are due to the way quad is rolling over sin and changing its step sizes. For your planned task, quad will be all you need.
EDIT:
For what you're doing I think quad is fine. It is fast and pretty accurate. My final statement is use it with confidence unless you find something that really has gone quite awry. If it doesn't return a nonsensical answer then it is probably working just fine. No worries.
I think it is probably machine precision since both answers are effectively zero.
If you want an answer from the horse's mouth I would post this question on the scipy discussion board
I would say that a number O(10^-14) is effectively zero. What's your tolerance?
It might be that the algorithm underlying quad isn't the best. You might try another method for integration and see if that improves things. A 5th order Runge-Kutta can be a very nice general purpose technique.
It could be just the nature of floating point numbers: "What Every Computer Scientist Should Know About Floating Point Arithmetic".
This output seems correct to me since you have absolute error estimate here. The integral value of sin(x) is indeed should have value of zero for full period (any interval of 2*pi length) in both ordinary and numeric integration and your results is close to that value.
To evaluate arc length you should calculate integral for sqrt(1+diff(f(x), x)^2) function, where diff(f(x), x) is derivative of f(x). See also Arc length
0.0 == 2.3e-16 (absolute error tolerance 4.4e-14)
Both answers are the same and correct i.e., zero within the given tolerance.
The difference comes from the fact that sin(x)=-sin(-x) exactly even in finite precision. Whereas finite precision only gives sin(x)~sin(x+2*pi) approximately. Sure it would be nice if quad were smart enough to figure this out, but it really has no way of knowing apriori that the integral over the two intervals you give are equivalent or that the the first is a better result.