I have:
import numpy as np
from mpmath import *
mpf(np.array(range(0,600)))
But it won't let me do it:
TypeError: cannot create mpf from array
So what should I be doing?
Essentially I'm going to have use this array and multiply element-wise with an incredibly large or incredible small number depending on circumstance (eg 1.35626567e1084 or 6.2345252e-2732) hence the need for mpf.
More specifically I'll be using the besseli and besselk function which create the incredible large and incredible small values.
How do I get an mpf array to hold these numbers?
Multiplying an array by a mpf number just works:
import numpy as np
import mpmath as mp
small_number = mp.besseli(400, 2) # This is an mpf number
# Note that creating a list using `range` and then converting it
# to an array is not very efficient. Do this instead:
A = np.arange(600)
result = small_number * A # Array of dtype object, ie, it contains mpf numbeers
Multiplying element-wise two arrays containing mpf numbers also works:
result * result
So your real problem is how to evaluate an mpmath function in a numpy array. To do that, I'd use np.frompyfunc (some time ago this was the only option).
besseli_vec = np.frompyfunc(mp.besseli, 2, 1)
besseli_vec(0, A)
Check out mpmath.arange:
import numpy as np
import mpmath as mp
np.array(mp.arange(600))
Related
In code:
import pandas as pd
import numpy as np
from scipy.signal import savgol_filter
s = pd.Series(np.random.rand(20), index=np.arange(20))
def smooth(x):
return savgol_filter(x, 7, 3)
s.apply(smooth)
I got an error If mode is 'interp', window_length must be less than or equal to the size of x..
The reason I think is smooth takes the value row-wisely from Series, so it got only 1 value each time which is smaller than the window_length=7.
If I use pd.DataFrame.apply(), there is a parameter axis that can be set to get the correct result.
I also want to avoid using smooth(s) which returns a NumPy array instead of pandas.Series.
Is there any way to .apply() sav_gol filter on pandas.Series directly?
How would I efficiently initialise a fixed-size pyarray.ListArray
from a suitably prepared numpy array?
The documentation of pyarray.array indicates that a nested iterable input structure works, but in practice that does not work if the outer iterable is a numpy array:
import numpy as np
import pyarrow as pa
n = 1000
w = 3
data = np.arange(n*w,dtype="i2").reshape(-1,w)
# this works:
pa.array(list(data),pa.list_(pa.int16(),w))
# this fails:
pa.array(data,pa.list_(pa.int16(),w))
# -> ArrowInvalid: only handle 1-dimensional arrays
It seems ridiculus to split an input array directly matching the Arrow specification into n separate arrays and then re-assemble from there.
pyarray.ListArray.from_arrays seems to require an offsets argument, which only has a meaning for variable-size lists.
I believe you are looking for pyarrow.FixedSizeListArray.from_arrays which, regrettably, appears undocumented (I went ahead and filed a JIRA ticket)
You'll want to reshape your numpy array as a contiguous array first.
import numpy as np
import pyarrow as pa
len = 10
width = 3
# Or just skip the initial reshape but keeping it in to simulate real data
arr = np.arange(len*width,dtype="i2").reshape(-1,width)
arr.shape = -1
pa.FixedSizeListArray.from_arrays(arr, width)
In c/c++ we use to declare three-dimensional using the following syntax.
`long long dp[20][180][2]; `
memset(dp, -1, sizeof(dp));
My code:
import numpy as np
x = np.zeros((20,180,2))
How can we declare and initialize a three-dimensional array in python?
If you want all the values initialized to -1 like in your memset example, then you'd want np.full instead of np.zeros
import numpy as np
x = np.full((20,180,2), -1)
Let's say I have an existing array that we don't want to make any changes to, but like to be converted to a ctype array and be shared among all the multiprocessing later on.
The actual array I want to be shared is of shape 120,000 x 4, which is too large to type all out here, so let's pretend such an array is way smaller and looks like this:
import numpy as np
import multiprocessing as mp
import ctypes
array_from_data = np.array([[275,174,190],
[494, 2292, 9103],
[10389,284,28],
[193,746,293]])
I have read other posts that discuss the ctype array and multiprocessing, like this one. However, the answers are not quite the same as what I am looking for, because so far they are not exactly about converting an existing NumPy array.
My questions are the following:
1) How to do a simple conversion from an existing Numpy array to a ctype array?
2) How to make the array to be shared among all the multiprocessing in a simple fashion?
Thank you in advance.
EDIT: spellings and some clarifications on the actual array
EDIT2: Apparently the os itself affects how the multiprocessing will behave and I need to specify it: My os is Windows 10 64-bit.
The workaround I found months ago requires flattening the array into a 1-dimensional array first, even though I only understand half of what is under the hood.
The gist of the solution is to:
1) make a RawArray of the same size and same dtypes as the array we are trying to share
2) create a numpy array that uses the same memory location as the RawArray
3) fill in the elements to the newly created numpy array
Workaround:
import ctypes
import multiprocessing as mp
import numpy as np
array_from_data = np.array([[275,174,190],
[494, 2292, 9103],
[10389,284,28],
[193,746,293]])
flattened_array1 = array_from_data.flatten(order='C')
flattened_array2 = np.array([1,0,1,0,1]).astype(bool)
flattened_array3 = np.array([1,0,1,0,-10]).astype(np.float32)
array_shared_in_multiprocessing1 = mp.RawArray(ctypes.c_int32,len(flattened_array1))
temp1 = np.frombuffer(array_shared_in_multiprocessing1, dtype=np.int32)
temp1[:] = flattened_array1
array_shared_in_multiprocessing2 = mp.RawArray(ctypes.c_bool,len(flattened_array2))
temp2 = np.frombuffer(array_shared_in_multiprocessing2, dtype=bool)
temp2[:] = flattened_array2
array_shared_in_multiprocessing3 = mp.RawArray(ctypes.c_float,len(flattened_array3))
temp2 = np.frombuffer(array_shared_in_multiprocessing3, dtype=np.float32)
temp2[:] = flattened_array3
I have a numpy array A with mpf elements that have decimal precision 100. Is this precision cast away if I decide to take the numpy dot product of A with itself?
If this is the case, is there any way to convert a numpy array to an mpmath matrix, so I can keep the precision?
Numpy arrays can hold objects, in particular mpf objects, and their methods such as dot can use the addition/multiplication methods of these objects. Example:
import mpmath
import numpy
mpmath.mp.dps = 25 # higher precision for demonstration
a = [mpmath.sin(mpmath.pi*n/3) for n in range(99)]
b = numpy.array(a)
b.dot(b)
outputs mpf('49.50000000000000000000000165')
For comparison, this is what happens if the array elements are cast to double-precision floats when converting to numpy:
c = numpy.array(a, dtype=float)
c.dot(c)
outputs 49.499999999999993. So, the higher precision provided by mpmath is preserved when the dot method is invoked in the first version.
The previous answer is correct. However, sometimes there are things that are working in numpy and it is not working in mpmath (at least it is done in a different way). Hence, the original (general) question of
"...is there any way to convert a numpy array to an mpmath matrix, so I can keep the precision?.."
In my experience, this (more general) question still needs to have a general answer. One of the answers to this problem is to convert the numpy array first to list then List to mpmath matrix.
Here is a simple example that works for me (warning, may not be efficient):
import mpmath as mp, numpy as np
N = 5
L = np.ones(N)
M = np.diag(L, 2) # A numpy matrix 7x7
# Notes that MPMath "diag" function is limited to one parameter only
M = mp.matrix(M.tolist())
print(type(M),'\n', M)