I'm building a function to calculate the Reliability of a given component/subsystem. For this, I wrote the following in a script:
import math as m
import numpy as np
def Reliability (MTBF,time):
failure_param = pow(MTBF,-1)
R = m.exp(-failure_param*time)
return R
The function works just fine for any time values I call in the function. Now I wanna call the function to calculate the Reliability for a given array, let's say np.linspace(0,24,25). But then I get errors like "Type error: only length-1 arrays can be converted to Python scalars".
Anyone that could help me being able to pass arrays/vectors on a Python function like that?
Thank you very much in advance.
The math.exp() function you are using knows nothing about numpy. It expects either a scalar, or an iterable with only one element, which it can treat as a scalar. Use the numpy.exp() instead, which accepts numpy arrays.
To be able to work with numpy arrays you need to use numpy functions:
import numpy as np
def Reliability (MTBF,time):
return np.exp(-(MTBF ** -1) * time)
If possible you should always use numpy functions instead of math functions, when working with numpy objects.
They do not only work directly on numpy objects like arrays and matrices, but they are highly optimized, i.e using vectorization features of the CPU (like SSE). Most functions like exp/sin/cos/pow are available in the numpy module. Some more advanced functions can be found in scipy.
Rather than call Reliability on the vector, use list comprehension to call it on each element:
[Reliability(MTBF, test_time) for test_time in np.linspace(0,24,25)]
Or:
map(Reliability, zip([MTBF]*25, linspace(0,24,25))
The second one produces a generator object which may be better for performance if the size of your list starts getting huge.
Related
When I try call curve_data
Like so:
curve_fit(func, xdata=np.arange(50), ydata=some_array)
It calls func using all xdata at once ( the whole array) instead of e.g. the first element of the xadata (xdata[0])
What is happening.
Cheers.
For people who happen to come across this: Scipy uses broadcasting, meaning if you use a math function in the function your are trying to fit, it also has to work with broadcasting, which basically means it internally loops over input vectors using c which is more efficient than looping in python. https://numpy.org/doc/stable/user/basics.broadcasting.html
For me the problem was, that I used the math library which does not support this.
I tried giving numba a go, as I was told it works very well for numerical/scientific computing applications. However, it seems that I've already run into a problem in the following scenario:
I have a function that computes a 12x12 Jacobian matrix, represented by a numpy array, and then returns this Jacobian. However, when I attempt to decorate said function with #numba.njit, I get the following error:
This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.
As a basic example of my usage, the following code tries to declare a 12x12 numpy zero matrix, but it fails:
import numpy as np
import numba
#numba.njit
def numpy_matrix_test():
A = np.zeros([12,12])
return A
A_out = numpy_matrix_test()
print(A_out)
Since I assumed declaring numpy arrays in such a way was common enough that numba would be able to handle them, I'm quite surprised.
The assumption that the functions called in a numba jitted function are the same functions when not used in a numba function is actually wrong (but understandable). In reality numba (behind the scenes) delegates to its own functions instead of using the "real" NumPy functions.
So it's not really np.zeros that is called in the jitted function, it's their own function. So some differences between Numba and NumPy are unavoidable.
For example you cannot use a list for the shape, it has to be a tuple (lists and arrays produce the exception you've encountered). So the correct syntax would be:
#numba.njit
def numpy_matrix_test():
A = np.zeros((12, 12))
return A
Something similar applies to the dtype argument. It has to be a real NumPy/numba type, a Python type cannot be used:
#numba.njit
def numpy_matrix_test():
A = np.zeros((12, 12), dtype=int) # to make it work use numba.int64 instead of int here
return A
Even if "plain" NumPy allows it:
np.zeros((12, 12), dtype=int)
Do you perhaps mean numpy.zeros((12,12)), because you want a shape of 12 rows and 12 columns?
Numpy Zeros reference
This question already has answers here:
How do I use numpy.where()? What should I pass, and what does the result mean? [closed]
(2 answers)
Closed 4 years ago.
I have already found the source for the numpy.ma.where() function but it seems to be calling the numpy.where() function and to better understand it I would like to take a look if possible.
Most Python functions are written in the Python language, but some functions are written in something more native (often the C language).
Regular Python functions ("pure Python")
There are a few techniques you can use to ask Python itself where a function is defined. Probably the most portable uses the inspect module:
>>> import numpy
>>> import inspect
>>> inspect.isbuiltin(numpy.ma.where)
False
>>> inspect.getsourcefile(numpy.ma.where)
'.../numpy/core/multiarray.py'
But this won't work with a native ("built-in") function:
>>> import numpy
>>> import inspect
>>> inspect.isbuiltin(numpy.where)
True
>>> inspect.getsourcefile(numpy.where)
TypeError: <built-in function where> is not a module, class, method, function, traceback, frame, or code object
Native ("built-in") functions
Unfortunately, Python doesn't provide a record of source files for built-in functions. You can find out which module provides the function:
>>> import numpy as np
>>> np.where
<built-in function where>
>>> np.where.__module__
'numpy.core.multiarray'
Python won't help you find the native (C) source code for that module, but in this case it's reasonable to look in the numpy project for C source that has similar names. I found the following file:
numpy/core/src/multiarray/multiarraymodule.c
And in that file, I found a list of definitions (PyMethodDef) including:
{"where",
(PyCFunction)array_where,
METH_VARARGS, NULL},
This suggests that the C function array_where is the one that Python sees as "where".
The array_where function is defined in the same file, and it mostly delegates to the PyArray_Where function.
In short
NumPy's np.where function is written in C, not Python. A good place to look is PyArray_Where.
First there are 2 distinct versions of where, one that takes just the condition, the other that takes 3 arrays.
The simpler one is most commonly used, and is just another name for np.nonzero. This scans through the condition array twice. Once with np.count_nonzero to determine how many nonzero entries there are, which allows it to allocate the return arrays. The second step is to fill in the coordinates of all nonzero entries. The key is that it returns a tuple of arrays, one array for each dimension of condition.
The condition, x, y version takes three arrays, which it broadcasts against each other. The return array has the common broadcasted shape, with elements chosen from x and y as explained in the answers to your previous question, How exactly does numpy.where() select the elements in this example?
You do realize that most of this code is c or cython, with a significant about of preprocessing. It is hard to read, even for experienced users. It is easier to run a variety of test cases and get a feel for what is happening that way.
A couple things to watch out for. np.where is a python function, and python evaluates each input fully before passing them to it. This is conditional assignment, not conditional evaluation function.
And unless you pass 3 arrays that match in shape, or scalar x and y, you'll need a good understanding of broadcasting.
You can find the code in numpy.core.multiarray
C:\Users\<name>\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\numpy\core\multiarray.py is where I found it.
Say I have a function foo() that takes in a single float and returns a single float. What's the fastest/most pythonic way to apply this function to every element in a numpy matrix or array?
What I essentially need is a version of this code that doesn't use a loop:
import numpy as np
big_matrix = np.matrix(np.ones((1000, 1000)))
for i in xrange(np.shape(big_matrix)[0]):
for j in xrange(np.shape(big_matrix)[1]):
big_matrix[i, j] = foo(big_matrix[i, j])
I was trying to find something in the numpy documentation that will allow me to do this but I haven't found anything.
Edit: As I mentioned in the comments, specifically the function I need to work with is the sigmoid function, f(z) = 1 / (1 + exp(-z)).
If foo is really a black box that takes a scalar, and returns a scalar, then you must use some sort of iteration. People often try np.vectorize and realize that, as documented, it does not speed things up much. It is most valuable as a way of broadcasting several inputs. It uses np.frompyfunc, which is slightly faster, but with a less convenient interface.
The proper numpy way is to change your function so it works with arrays. That shouldn't be hard to do with the function in your comments
f(z) = 1 / (1 + exp(-z))
There's a np.exp function. The rest is simple math.
I have a python function that employs the numpy package. It uses numpy.sort and numpy.array functions as shown below:
def function(group):
pre_data = np.sort(np.array(
[c["data"] for c in group[1]],
dtype = np.float64
))
How can I re-write the sort and array functions using only Python in such a way that I no longer need the numpy package?
It really depends on the code after this. pre_data will be a numpy.ndarray which means that it has array methods which will be really hard to replicate without numpy. If those methods are being called later in the code, you're going to have a hard time and I'd advise you to just bite the bullet and install numpy. It's popularity is a testament to it's usefulness...
However, if you really just want to sort a list of floats and put it into a sequence-like container:
def function(group):
pre_data = sorted(float(c['data']) for c in group[1])
should do the trick.
Well, it's not strictly possible because the return type is an ndarray. If you don't mind to use a list instead, try this:
pre_data = sorted(float(c["data"]) for c in group[1])
That's not actually using any useful numpy functions anyway
def function(group):
pre_data = sorted(float(c["data"]) for c in group[1])