I have the following equation:
result = (A * np.exp(b * (t - t0))) / (1 + np.exp(c * (t - t0)))
I feed in an array of t values to get results out. A, b, c, t0 are all constants (b and c are very large, t0 is small but not as small as b and c are large). The problem is, I run into an overflow error because the exponential value quickly gets much too large to fit into a float64 beyond a certain range of t. I'm trying to find a workaround to this while still maintaining a decent level of precision. The result value is well within the range of a float64 container, however the overly large intermediate values of the np.exp calculation prevent me from getting as far as the result.
Some thoughts I had:
Scale down the t input to be able to get the desired range of values, and then de-scale the output so the result is correct
Convert the exponential to a log function
However I'm not sure how to implement either of these ideas, or if they would actually work.
Essential this problem boils down to result = np.exp(a) / np.exp(b), where a and b are in the range of 100-1000. np.exp(709) results in 8.2e307, right at the limit of a float64, but I have larger values that need to feed into it. While the comparison of the two exponentials produces a reasonable value, the exponentials themselves are too large to be calculated.
keeping everything in the log scale is the common solution to this sort of thing. at least that's what we do in statistics where you're often down in the 1e-10000 range, especially at the start before you're any where near convergence. for example, all the scipy probability density functions have logpdf variants which work in the log scale.
I think your expression would be rewritten something like:
d = t - t0
log_result = (np.log(A) + (b * d)) - np.logaddexp(0, c * d)
(untested)
Related
when modelling a lp production problem, I use a high number of integer.
First the problem was being solved in a reasonable time but when I added a big number of binary values (y) due to big M method, it takes hours and hours and the program doesn't appear to be working at all. This is because Pulp is quite slow with binary variables.
The reason I did big M is because I wanted to seperate the variable bounds, in order to get
x(i,t) = 0 or x(i,t) >= 10 (which means either produce nothing or at least 10 pieces of product i in every time period t):
x(i,t) - 1 <= e + M * y(i,t)
x(i,t) - 9 >= -e - (1 - y(i,t) ) * M
(e is a very small constant number and M a very big constant number)
Is there any other way than using the method shown above?
I could define the bounds of x(i,t) like [10, inf] but I still need the value 0, since producing nothing is also a very valid important solution.
Thanks in advance!
I'm not sure why you are formulating like that with big M and the small constant for integer/binary variables.
If x is integer and y is corresponding binary variable, re-formulate to this:
x <= y * M
x >= y * 10
Where M is a logical upper bound on x
This is similar to what you had, but the small constant e is unnecessary and M should not be overly huge.
All solvers (including the one pre-packaged in pulp slow significantly with the introduction of large numbers of binary or integer variables, under most conditions.
Have you tried setting a mip-gap or such to convince yourself that it is working?
In some point of my project i have to multiply two small float like 8.696503446228892e-159 and 1.2425389522444519e-158 as i test in following code:
def a2(a,b):
a = tf.cast(a, tf.float64)
b = tf.cast(b, tf.float64)
d = a*b
return d
it will return 0 which cause lots of problem (because it is used in my loss function) any solution how can i multiply them?
Handling large discrepancies in computational magnitude is a field of study in itself.
The first-order way to do this is to write your evaluation code to detect the situation and re-order the operations so as to preserve significant bits of each result. For instance, let's simplify your names a bit:
tf.log(tf.linalg.det(temp_sigma) /
(tf.sqrt(tf.linalg.det(sigma1) * tf.linalg.det(sigma2))))
turns into
log(det(A) / (sqrt(det(B) * det(c))))
The case you have is that det(B) and det(C) are barely above zero, but relatively near each other: the result of sqrt(det(B) * det(C)) will be close to either determinant.
Change the order of operations. For this instance, distribute the square root and do the divisions individually:
log(
( det(A) / sqrt(det(B)) ) / sqrt(det(C)) )
Does that get you moving along?
I need to estimate the size of a population, by finding the value of n which maximises scipy.misc.comb(n, a)/n**b where a and b are constants. n, a and b are all integers.
Obviously, I could just have a loop in range(SOME_HUGE_NUMBER), calculate the value for each n and break out of the loop once I reach an inflexion in the curve. But I wondered if there was an elegant way of doing this with (say) numpy/scipy, or is there some other elegant way of doing this just in pure Python (e.g. like an integer equivalent of Newton's method?)
As long as your number n is reasonably small (smaller than approx. 1500), my guess for the fastest way to do this is to actually try all possible values. You can do this quickly by using numpy:
import numpy as np
import scipy.misc as misc
nMax = 1000
a = 77
b = 100
n = np.arange(1, nMax+1, dtype=np.float64)
val = misc.comb(n, a)/n**b
print("Maximized for n={:d}".format(int(n[val.argmax()]+0.5)))
# Maximized for n=181
This is not especially elegant but rather fast for that range of n. Problem is that for n>1484 the numerator can already get too large to be stored in a float. This method will then fail, as you will run into overflows. But this is not only a problem of numpy.ndarray not working with python integers. Even with them, you would not be able to compute:
misc.comb(10000, 1000, exact=True)/10000**1001
as you want to have a float result in your division of two numbers larger than the maximum a float in python can hold (max_10_exp = 1024 on my system. See sys.float_info().). You couldn't use your range in that case, as well. If you really want to do something like that, you will have to take more care numerically.
You essentially have a nicely smooth function of n that you want to maximise. n is required to be integral but we can consider the function instead to be a function of the reals. In this case, the maximising integral value of n must be close to (next to) the maximising real value.
We could convert comb to a real function by using the gamma function and use numerical optimisation techniques to find the maximum. Another approach is to replace the factorials with Stirling's approximation. This gives a moderately complicated but tractable algebraic expression. This expression is not hard to differentiate and set to zero to find the extrema.
I did this and obtained
n * (b + (n-a) * log((n-a)/n) ) = a * b - a/2
This is not straightforward to solve algebraically but easy enough numerically (e.g. using Newton's method, as you suggest).
I may have made a mistake in the algebra, but I typed the a = 77, b = 100 example into Wolfram Alpha and got 180.58 so the approach seems to work.
I have a convolution integral of the type:
To solve this integral numerically, I would like to use numpy.convolve(). Now, as you can see in the online help, the convolution is formally done from -infinity to +infinity meaning that the arrays are moved along each other completely for evaluation - which is not what I need. I obviously need to be sure to pick the correct part of the convolution - can you confirm that this is the right way to do it or alternatively tell me how to do it right and (maybe even more important) why?
res = np.convolve(J_t, dF, mode="full")[:len(dF)]
J_t is an analytical function and I can evaluate as many points as I need, dF are derivatives of measurement data. for this attempt I choose len(J_t) = len(dF) because from my understanding I do not need more.
Thank you for your thoughts, as always, I appreciate your help!
Background information (for those who might be interested)
These type of integrals can be used to evaluate viscoelastic behaviour of bodies (or the response of an electric circuit during change of voltage, if you feel more familiar on this topic). For viscoelasticity, J(t) is the creep compliance function and F(t) can be the deviatoric strains over time, then this integral would yield the deviatoric stresses.
If you now e.g. have a J(t) of the form:
J_t = lambda p, t: p[0] + p[1]*N.exp(-t/p[2])
with p = [J_elastic, J_viscous, tau] this would be the "famous" standard linear solid. The integral limits are the start of the measurement t_0 = 0 and the moment of interest, t.
To get it right, I have chosen the following two functions:
a(t) = t
b(t) = t**2
It is easy to do the math and find that their "convolution" as defined in your case, takes
on the values:
c(t) = t**4 / 12
So lets try them out:
>>> delta = 0.001
>>> t = np.arange(1000) * delta
>>> a = t
>>> b = t**2
>>> c = np.convolve(a, b) * delta
>>> d = t**4 / 12
>>> plt.plot(np.arange(len(c)) * delta, c)
[<matplotlib.lines.Line2D object at 0x00000000025C37B8>]
>>> plt.plot(t[::50], d[::50], 'o')
[<matplotlib.lines.Line2D object at 0x000000000637AB38>]
>>> plt.show()
So by doing the above, if both your a and b have n elements, you get the right convolution values in the first n elements of c.
Not sure if the following explanation will make any sense, but here it goes... If you think of convolution as mirroring one of the functions along the y-axis, then sliding it along the x axis and computing the integral of the product at each point, it is easy to see how, since outside of the area of definition numpy takes them as if padded with zeros, you are effectively setting an integration interval from 0 to t, since the first function is zero below zero, and the second is zero above t, since it originally was zero below zero, but has been mirrored and moved t to the right.
I was tackling this same problem and solved it using a highly inefficient but functionally correct algorithm:
def Jfunk(inz,t):
c0 = inz[0]
c1 = inz[1]
c2 = inz[2]
J = c0 - c1*np.exp(-t/c2)
return J
def SLS_funk(inz, t, dl_dt):
boltz_int = np.empty(shape=(0,))
for i,v in enumerate(t, start=1):
t_int = t[0:i]
Jarg = v - t[0:i]
J_int = Jfunk(inz,Jarg)
dl_dt_int = dl_dt[0:i]
inter_grand = np.multiply(J_int, dl_dt_int)
boltz_int = np.append(boltz_int, simps (inter_grand, x=t_int) )
return boltz_int
Thanks to this question and its answers, I was able to implement a much better solution based on the numpy convolution function suggested above. In case the OP was curious I did a time comparison of the two methods.
For an SLS (three parameter J function) with 20,000 time points:
Using Numpy convolution: ~0.1 seconds
Using Brute Force method: ~7.2 seconds
If if helps to get a feeling for the alignment, try convolving a pair of impulses. With matplotlib (using ipython --pylab):
In [1]: a = numpy.zeros(20)
In [2]: b = numpy.zeros(20)
In [3]: a[0] = 1
In [4]: b[0] = 1
In [5]: c = numpy.convolve(a, b, mode='full')
In [6]: plot(c)
You can see from the resultant plot that the first sample in c corresponds to the first position of overlap. In this case, only the first samples of a and b overlap. All the rest are floating in undefined space. numpy.convolve effectively replaces this undefined space with zeros, which you can see if you set a second non-zero value:
In [9]: b[1] = 1
In [10]: plot(numpy.convolve(a, b, mode='full'))
In this case, the first value of the plot is 1, as before (showing that the second value of b is not contributing at all).
I have been struggling with similar question for past 2 days.
The OP may have moved on, but I am still presenting my analysis here.
Following two sources helped me:
Discussion on stackoverflow
These notes
I will consider time-series data defined on the same time series starting from time .
Let the two series be A and B.
Their (continuous) convolution is
Substituting with in the above equation we get what np.convolve(A,B) returns:
What you want is
Again making the same substitution, we get
which is same as above because A for negative indices is extrapolated to zero and for i > (j + m) B[j - i + m] is zero.
If you look at the notes cited above, you can figure out that corresponds to time for our time series.
The next value in the list will correspond to and so on.
Therefore, the correct answer will be
is equal to np.convolve(A,B)[0:M], where M = len(A) = len(B).
Here keep in mind that M*dt = T, where T is the last element of time array.
Disclaimer: I am not a programmer, mathematician or an engineer. I had to use convolution somewhere and have derived these conclusions from my own struggle with the problem. I will be happy to cite any book which has this analysis if someone can point it out.
I have this line of MATLAB code:
a/b
I am using these inputs:
a = [1,2,3,4,5,6,7,8,9,1,2,3,4,5,6,7,8,9]
b = ones(25, 18)
This is the result (a 1x25 matrix):
[5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
What is MATLAB doing? I am trying to duplicate this behavior in Python, and the mrdivide documentation in MATLAB was unhelpful. Where does the 5 come from, and why are the rest of the values 0?
I have tried this with other inputs and receive similar results, usually just a different first element and zeros filling the remainder of the matrix. In Python when I use linalg.lstsq(b.T,a.T), all of the values in the first matrix returned (i.e. not the singular one) are 0.2. I have already tried right division in Python and it gives something completely off with the wrong dimensions.
I understand what a least square approximation is, I just need to know what mrdivide is doing.
Related:
Array division- translating from MATLAB to Python
MRDIVIDE or the / operator actually solves the xb = a linear system, as opposed to MLDIVIDE or the \ operator which will solve the system bx = a.
To solve a system xb = a with a non-symmetric, non-invertible matrix b, you can either rely on mridivide(), which is done via factorization of b with Gauss elimination, or pinv(), which is done via Singular Value Decomposition, and zero-ing of the singular values below a (default) tolerance level.
Here is the difference (for the case of mldivide): What is the difference between PINV and MLDIVIDE when I solve A*x=b?
When the system is overdetermined, both algorithms provide the
same answer. When the system is underdetermined, PINV will return the
solution x, that has the minimum norm (min NORM(x)). MLDIVIDE will
pick the solution with least number of non-zero elements.
In your example:
% solve xb = a
a = [1,2,3,4,5,6,7,8,9,1,2,3,4,5,6,7,8,9];
b = ones(25, 18);
the system is underdetermined, and the two different solutions will be:
x1 = a/b; % MRDIVIDE: sparsest solution (min L0 norm)
x2 = a*pinv(b); % PINV: minimum norm solution (min L2)
>> x1 = a/b
Warning: Rank deficient, rank = 1, tol = 2.3551e-014.
ans =
5.0000 0 0 ... 0
>> x2 = a*pinv(b)
ans =
0.2 0.2 0.2 ... 0.2
In both cases the approximation error of xb-a is non-negligible (non-exact solution) and the same, i.e. norm(x1*b-a) and norm(x2*b-a) will return the same result.
What is MATLAB doing?
A great break-down of the algorithms (and checks on properties) invoked by the '\' operator, depending upon the structure of matrix b is given in this post in scicomp.stackexchange.com. I am assuming similar options apply for the / operator.
For your example, MATLAB is most probably doing a Gaussian elimination, giving the sparsest solution amongst a infinitude (that's where the 5 comes from).
What is Python doing?
Python, in linalg.lstsq uses pseudo-inverse/SVD, as demonstrated above (that's why you get a vector of 0.2's). In effect, the following will both give you the same result as MATLAB's pinv():
from numpy import *
a = array([1,2,3,4,5,6,7,8,9,1,2,3,4,5,6,7,8,9])
b = ones((25, 18))
# xb = a: solve b.T x.T = a.T instead
x2 = linalg.lstsq(b.T, a.T)[0]
x2 = dot(a, linalg.pinv(b))
TL;DR: A/B = np.linalg.solve(B.conj().T, A.conj().T).conj().T
I did not find the earlier answers to create a satisfactory substitute, so I dug into Matlab's reference documents for mrdivide further and found the solution. I cannot explain the actual mathematics here or take credit for coming up with the answer. I'm just following Matlab's explanation. Additionally, I wanted to post the actual detail from Matlab to give credit. If it's a copyright issue, someone tell me and I'll remove the actual text.
%/ Slash or right matrix divide.
% A/B is the matrix division of B into A, which is roughly the
% same as A*INV(B) , except it is computed in a different way.
% More precisely, A/B = (B'\A')'. See MLDIVIDE for details.
%
% C = MRDIVIDE(A,B) is called for the syntax 'A / B' when A or B is an
% object.
%
% See also MLDIVIDE, RDIVIDE, LDIVIDE.
% Copyright 1984-2005 The MathWorks, Inc.
Note that the ' symbol indicates the complex conjugate transpose. In python using numpy, that requires .conj().T chained together.
Per this handy "cheat sheet" of numpy for matlab users, linalg.lstsq(b,a) -- linalg is numpy.linalg.linalg, a light-weight version of the full scipy.linalg.
a/b finds the least square solution to the system of linear equations bx = a
if b is invertible, this is a*inv(b), but if it isn't, the it is the x which minimises norm(bx-a)
You can read more about least squares on wikipedia.
according to matlab documentation, mrdivide will return at most k non-zero values, where k is the computed rank of b. my guess is that matlab in your case solves the least squares problem given by replacing b by b(:1) (which has the same rank). In this case the moore-penrose inverse b2 = b(1,:); inv(b2*b2')*b2*a' is defined and gives the same answer