I would like to write the following summation in python
The coefficients are given as the following list
cn=[1.2,0.4,0.6]
vn=[1e-6,5e-5,1e-6]
gn=[4.5e3,6.5e3,9e3]
t=np.linspace(0,10,100)
I tried the following
import numpy as np
cn=[1.2,0.4,0.6]
vn=[1e-6,5e-5,1e-6]
gn=[4.5e3,6.5e3,9e3]
t=np.linspace(0,10,100)
yt=np.sum(cn *np.exp(-vn*(t-gn)**2))
but am getting the error
TypeError: bad operand type for unary -: 'list'
I would like to know where am getting it wrong or how to do this task
This run:
import numpy as np
cn=np.array([1.2,0.4,0.6])
vn=np.array([1e-6,5e-5,1e-6])
gn=np.array([4.5e3,6.5e3,9e3])
t=np.linspace(0,10,3)
yt=np.sum(cn * np.exp(-vn * (t - gn)**2))
Transform lists into numpy arrays
Make sure the matrix / array sizes are compatible, (ie. You can't add arrays of different lengths)
Example:
Add int to python list:
cn=[1.2,0.4,0.6]
cn+1
# output: TypeError: can only concatenate list (not "int") to list
Add int to numpy array:
cn=np.array([1.2,0.4,0.6])
cn+1
# output: array([2.2, 1.4, 1.6])
Add numpy arrays with different dimensions:
cn = np.arange(1,3)
t = np.arange(1,100)
cn + t
# output: ValueError: operands could not be broadcast together with shapes (2,) (99,)
Add numpy arrays with the same dimensions:
cn = np.arange(1,3)
t = np.arange(3,5)
cn + t
# output: array([4, 6])
Here is a lazy way of fixing it:
yt=np.sum(cn *np.exp(0-vn*(np.c_[t]-gn)**2), 1)
^ ^------^ ^-^
I've highlighted the changes. The most important change is the np.c_ which does two things:
It converts t to array
It makes t a column vector
1) serves as a "germ" for converting all the other lists to arrays via overloaded arithmetic operators.
Exception: the unary - in front of vn hits vn before it gets the chance to become an array. We put a zero in front the - to make it binary, thereby reducing it's precedence and closing the array coercion chain. This is not the most obvious fix but the one involving the least typing.
2) separates the time dimension from the summation dimension which is likely the correct interpretation. We have to add an eplicit axis argument to the sum which is the 1 we inserted at the very end of the expression.
If found two issues which I fixed but I am not sure is what you intended.
you don't need to convert the list to numpy array because you can perform arithmetic array between ndarray and list which will result ndarray.
Two error found are
1. shape of t was not matching with other arrays
2. you were trying to negate python list which doesn't support it
Also as you haven't put tn in your mathematical expression of summation above so I doubt it you want the length of t to be 3
import numpy as np
cn=[1.2,0.4,0.6]
vn=[1e-6,5e-5,1e-6]
gn=[4.5e3,6.5e3,9e3]
t=np.linspace(0,10,3) # shape of t what 100 and not matching with other arrays
yt=np.sum(cn *np.exp(-(vn*(t-gn)**2))) # -(vn*(t-gn)**2)) wasn't wrapped in brackets
Related
So I'm trying to do the following:
self.cashflows["Balance"] * self.assumptions["Default Rates"][current_period - 1]
where cashflows is a list with python floats and assumptions is a list with numpy floats.
I used numpy to fill the assumption vector and I am getting the following error when I try to multiply the two:
can't multiply sequence by non-int of type 'numpy.float64
I get the error but what would be my best course of action here?
thx
Depends on what you want to do. Do you want to multiply every item in the list self.cashflow["Balance"] with self.assumptions["Default Rates"][current_period - 1]? Then you can use some list comprehension:
result = [q * self.assumptions["Default Rates"][current_period - 1] for q in self.cashflow["Balance"]]
or convert your second argument to np.float:
result = self.assumptions["Default Rates"][current_period - 1]* np.asarray(self.cashflow["Balance"])
Otherwise, multiplying a whole list by N repeats that list N times. If thats what you want, cast your np.float64 to int.
EDIT: Added missing multiplication sign
I'm converting MATLAB code to Python
This is my code in python:
import numpy as np
import math
n=150
L=1
inter=L/n
y=np.linspace(inter/2,L-inter/2,n).transpose()
E=(210000000000)*np.ones(n)
Rho=7800*np.ones(n)
PI=math.pi
A=np.exp( 5+2*y*(np.sin(2*PI*y/L)) )*0.000001
This works fine up until this point with no difference in values or issues until I have to execute this piece of MATLAB code.
Mass=sum(Rho*inter.*A)
I tried the same using np.sum(Rho*inter*A) and just Rho*inter*A
The first case I got a single answer 1.0626206716847877 but MATLAB returns a 150 element array.
In the scond case I got an ndarray like I wanted but the values were not the same as what I got in MATLAB.
Values I got in MATLAB : matlab values pastebin
Values I got in python : python values pastebin
What am I doing wrong?
(Rho[:,None]*inter*A).sum(axis=0)
matches your MATLAB pastebin.
Or using einsum to sort out the axes:
np.einsum('i,j->j', Rho,inter*A)
which just reduces to:
Rho.sum() * inter*A
Is that really what you are trying to do in MATLAB?
It might help if you showed the actual MATLAB code used to create Rho, A etc.
Mass=sum(Rho*inter.*A)
What's the size of Rho and A in MATLAB? One may be [1x150], but the other? Is Rho [1x150] also, or [150x150]. The * is matrix multiplication, like # in numpy, but .* is elementwise.
In the numpy code y, Rho and A all have shape (150,). The transpose on y does nothing. Rho*inter*A is elementwise multiplication producing a (150,) as well.
NumPy always sums all elements of a matrix. MATLAB's default is column-based, i.e. all of your 150 columns sum to a single total, hence the array. Use sum(matrix,'all'); in MATLAB to sum over all elements in a matrix. If you have a MATLAB older than 2018b, use sum(matrix(:)), i.e. store your matrix in a temporary variable, then flatten it to a column before summing.
To sum over columns in Python, specify the axis, being 0: np.sum(matrix,axis=0)
numpy.sum():
Axis or axes along which a sum is performed. The default, axis=None, will sum all of the elements of the input array.
sum() from MATLAB:
S = sum(A) returns the sum of the elements of A along the first array dimension whose size does not equal 1.
If A is a matrix, then sum(A) returns a row vector containing the sum of each column.
S = sum(A,'all') computes the sum of all elements of A. This syntax is valid for MATLAB® versions R2018b and later.
To prevent this kind of unclarities, I prefer to always specify which direction to sum over, i.e. sum(matrix,1) for MATLAB and np.sum(matrix,axis=0) for NumPy, regardless of the default.
I think that in MATLAB using sum on a matrix you will get the sum of its individual columns and you will end up with an array with its number of elements equal to that of the columns. Use one more sum command in MATLAB: sum(sum(M)), which is the equivalent of np.sum(M) in Python.
I am trying to multiply all the row values and column values of a 2 dimensional numpy array with an explicit for-loop:
product_0 = 1
product_1 = 1
for x in arr:
product_0 *= x[0]
product_1 *= x[1]
I realize the product will blow up to become an extremely large number but from my previous experience python has had no memory problem dealing very very extremely large numbers.
So from what I can tell this is a problem with numpy except I am not storing the gigantic product in a numpy array or any numpy data type for that matter its just a normal python variable.
Any idea how to fix this?
Using non inplace multiplication hasn't helped product_0 = x[0]*product_0
Python int are represented in arbitrary precision, so they cannot overflow. But numpy uses C++ under the hood, so the highest long signed integer is with fixed precision, 2^63 - 1. Your number is far beyond this value, having in average ((716-1)/2)^86507).
When you, in the for loop, extract the x[0] this is still a numpy object. To use the full power of python integers you need to clearly assign it as python int, like this:
product_0 = 1
product_1 = 1
for x in arr:
t = int(x[0])
product_0 = product_0 * t
and it will not overflow.
Following your comment, which makes your question more specific, your original problem is to calculate the geometric mean of the array for each row/column. Here the solution:
I generate first an array that has the same properties of your array:
arr = np.resize(np.random.randint(1,716,86507*2 ),(86507,2))
Then, calculate the geometric mean for each column/row:
from scipy import stats
gm_0 = stats.mstats.gmean(arr, axis = 0)
gm_1 = stats.mstats.gmean(arr, axis = 1)
gm_0 will be an array that contains the geometric mean of the xand y coordinates. gm_1 instead contains the geometric mean of the rows.
Hope this solves your problem!
You say
So from what I can tell this is a problem with numpy except I am not storing the gigantic product in a numpy array or any numpy data type for that matter its just a normal python variable.
Your product may not be a NumPy array, but it is using a NumPy data type. x[0] and x[1] are NumPy scalars, and multiplying a Python int by a NumPy scalar produces a NumPy scalar. NumPy integers have a finite range.
While you technically could call int on x[0] and x[1] to get a Python int, it'd probably be better to avoid needing such huge ints. You say you're trying to perform this multiplication to compute a geometric mean; in that case, it'd be better to compute the geometric mean by transforming to and from logarithms, or to use scipy.stats.mstats.gmean, which uses logarithms under the hood.
Numpy is compiled for 32 bit and not 64 bit , so while Python can handle this numpy will overflow for smaller values , if u want to use numpy then I recommend that you build it from source .
Edit
After some testing with
import numpy as np
x=np.abs(np.random.randn(1000,2)*1000)
np.max(x)
prod1=np.dtype('int32').type(1)
prod2=np.dtype('int32').type(1)
k=0
for i,j in x:
prod1*=i
prod2*=j
k+=1
print(k," ",prod1,prod2)
1.797693134e308 is the max value (to this many digits my numpy scalar was able to take)
if you run this you will see that numpy is able to handle quite a large value but when you said your max value is around 700 , even with a 1000 values my scalar overflowed.
As for how to fix this , rather than doing this manually the answer using scipy seems more viable now and is able to get the answer so i suggest that you go forward with that
from scipy.stats.mstats import gmean
x=np.abs(np.random.randn(1000,2)*1000)
print(gmean(x,axis=0))
You can achieve what you want with the following command in numpy:
import numpy as np
product_0 = np.prod(arr.astype(np.float64))
It can still reach np.inf if your numbers are large enough, but that can happen for any type.
I have an array of Cartesian coordinates
xy = np.array([[0,0], [2,3], [3,4], [2,5], [5,2]])
which I want to convert into an array of complex numbers representing the same:
c = np.array([0, 2+3j, 3+4j, 2+5j, 5+2j])
My current solution is this:
c = np.sum(xy * [1,1j], axis=1)
This works but seems crude to me, and probably there is a nicer version with some built-in magic using np.complex() or similar, but the only way I found to use this was
c = np.array(list(map(lambda c: np.complex(*c), xy)))
This doesn't look like an improvement.
Can anybody point me to a better solution, maybe using one of the many numpy functions I don't know by heart (is there a numpy.cartesian_to_complex() working on arrays I haven't found yet?), or maybe using some implicit conversion when applying a clever combination of operators?
Recognize that complex128 is just a pair of floats. You can then do this using a "view" which is free, after converting the dtype from int to float (which I'm guessing your real code might already do):
xy.astype(float).view(np.complex128)
The astype() converts the integers to floats, which requires construction of a new array, but once that's done the view() is "free" in terms of runtime.
The above gives you shape=(n,1); you can np.squeeze() it to remove the extra dimension. This is also just a view operation, so takes basically no time.
How about
c=xy[:,0]+1j*xy[:,1]
xy[:,0] will give an array of all elements in the 0th column of xy and xy[:,1] will give that of the 1st column.
Multiply xy[:,1] with 1j to make it imaginary and then add the result with xy[:,0].
In python 3.5, the # operator was introduced for matrix multiplication, following PEP465. This is implemented e.g. in numpy as the matmul operator.
However, as proposed by the PEP, the numpy operator throws an exception when called with a scalar operand:
>>> import numpy as np
>>> np.array([[1,2],[3,4]]) # np.array([[1,2],[3,4]]) # works
array([[ 7, 10],
[15, 22]])
>>> 1 # 2 # doesn't work
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: unsupported operand type(s) for #: 'int' and 'int'
This is a real turnoff for me, since I'm implementing numerical signal processing algorithms that should work for both scalars and matrices. The equations for both cases are mathematically exactly equivalent, which is no surprise, since "1-D x 1-D matrix multiplication" is equivalent to scalar multiplication. The current state however forces me to write duplicate code in order to handle both cases correctly.
So, given that the current state is not satisfactory, is there any reasonable way I can make the # operator work for scalars? I thought about adding a custom __matmul__(self, other) method to scalar data types, but this seems like a lot of hassle considering the number of involved internal data types. Could I change the implementation of the __matmul__ method for numpy array data types to not throw an exception for 1x1 array operands?
And, on a sidenote, which is the rationale behind this design decision? Off the top of my head, I cannot think of any compelling reasons not to implement that operator for scalars as well.
As ajcr suggested, you can work around this issue by forcing some minimal dimensionality on objects being multiplied. There are two reasonable options: atleast_1d and atleast_2d which have different results in regard to the type being returned by #: a scalar versus a 1-by-1 2D array.
x = 3
y = 5
z = np.atleast_1d(x) # np.atleast_1d(y) # returns 15
z = np.atleast_2d(x) # np.atleast_2d(y) # returns array([[15]])
However:
Using atleast_2d will lead to an error if x and y are 1D-arrays that would otherwise be multiplied normally
Using atleast_1d will result in the product that is either a scalar or a matrix, and you don't know which.
Both of these are more verbose than np.dot(x, y) which would handle all of those cases.
Also, the atleast_1d version suffers from the same flaw that would also be shared by having scalar # scalar = scalar: you don't know what can be done with the output. Will z.T or z.shape throw an error? These work for 1-by-1 matrices but not for scalars. In the setting of Python, one simply cannot ignore the distinction between scalars and 1-by-1 arrays without also giving up all the methods and properties that the latter have.