I have a python function that employs the numpy package. It uses numpy.sort and numpy.array functions as shown below:
def function(group):
pre_data = np.sort(np.array(
[c["data"] for c in group[1]],
dtype = np.float64
))
How can I re-write the sort and array functions using only Python in such a way that I no longer need the numpy package?
It really depends on the code after this. pre_data will be a numpy.ndarray which means that it has array methods which will be really hard to replicate without numpy. If those methods are being called later in the code, you're going to have a hard time and I'd advise you to just bite the bullet and install numpy. It's popularity is a testament to it's usefulness...
However, if you really just want to sort a list of floats and put it into a sequence-like container:
def function(group):
pre_data = sorted(float(c['data']) for c in group[1])
should do the trick.
Well, it's not strictly possible because the return type is an ndarray. If you don't mind to use a list instead, try this:
pre_data = sorted(float(c["data"]) for c in group[1])
That's not actually using any useful numpy functions anyway
def function(group):
pre_data = sorted(float(c["data"]) for c in group[1])
Related
When I try call curve_data
Like so:
curve_fit(func, xdata=np.arange(50), ydata=some_array)
It calls func using all xdata at once ( the whole array) instead of e.g. the first element of the xadata (xdata[0])
What is happening.
Cheers.
For people who happen to come across this: Scipy uses broadcasting, meaning if you use a math function in the function your are trying to fit, it also has to work with broadcasting, which basically means it internally loops over input vectors using c which is more efficient than looping in python. https://numpy.org/doc/stable/user/basics.broadcasting.html
For me the problem was, that I used the math library which does not support this.
I've tried searching quite a lot on this one, but being relatively new to python I feel I am missing the required terminology to find what I'm looking for.
I have a function:
def my_function(x,y):
# code...
return(a,b,c)
Where x and y are numpy arrays of length 2000 and the return values are integers. I'm looking for a shorthand (one-liner) to loop over this function as such:
Output = [my_function(X[i],Y[i]) for i in range(len(Y))]
Where X and Y are of the shape (135,2000). However, after running this I am currently having to do the following to separate out 'Output' into three numpy arrays.
Output = np.asarray(Output)
a = Output.T[0]
b = Output.T[1]
c = Output.T[2]
Which I feel isn't the best practice. I have tried:
(a,b,c) = [my_function(X[i],Y[i]) for i in range(len(Y))]
But this doesn't seem to work. Does anyone know a quick way around my problem?
my_function(X[i], Y[i]) for i in range(len(Y))
On the verge of crossing the "opinion-based" border, ...Y[i]... for i in range(len(Y)) is usually a big no-no in Python. It is even a bigger no-no when working with numpy arrays. One of the advantages of working with numpy is the 'vectorization' that it provides, and thus pushing the for loop down to the C level rather than the (slower) Python level.
So, if you rewrite my_function so it can handle the arrays in a vectorized fashion using the multiple tools and methods that numpy provides, you may not even need that "one-liner" you are looking for.
I'm building a function to calculate the Reliability of a given component/subsystem. For this, I wrote the following in a script:
import math as m
import numpy as np
def Reliability (MTBF,time):
failure_param = pow(MTBF,-1)
R = m.exp(-failure_param*time)
return R
The function works just fine for any time values I call in the function. Now I wanna call the function to calculate the Reliability for a given array, let's say np.linspace(0,24,25). But then I get errors like "Type error: only length-1 arrays can be converted to Python scalars".
Anyone that could help me being able to pass arrays/vectors on a Python function like that?
Thank you very much in advance.
The math.exp() function you are using knows nothing about numpy. It expects either a scalar, or an iterable with only one element, which it can treat as a scalar. Use the numpy.exp() instead, which accepts numpy arrays.
To be able to work with numpy arrays you need to use numpy functions:
import numpy as np
def Reliability (MTBF,time):
return np.exp(-(MTBF ** -1) * time)
If possible you should always use numpy functions instead of math functions, when working with numpy objects.
They do not only work directly on numpy objects like arrays and matrices, but they are highly optimized, i.e using vectorization features of the CPU (like SSE). Most functions like exp/sin/cos/pow are available in the numpy module. Some more advanced functions can be found in scipy.
Rather than call Reliability on the vector, use list comprehension to call it on each element:
[Reliability(MTBF, test_time) for test_time in np.linspace(0,24,25)]
Or:
map(Reliability, zip([MTBF]*25, linspace(0,24,25))
The second one produces a generator object which may be better for performance if the size of your list starts getting huge.
Is the following was possible in python?
(I am pretty new to python, not sure what the appropriate search term would be)
I have a class that stores and manipulates a large numpy array.
Now I would like to access parts of this array via an alias 'reference function'
Here is a dummy example for illustration
import numpy as np
class Trajectory(object):
def __init__(self,M=np.random.random((4,4))):
self.M=M
def get_second_row(self):
return self.M[1,:]
def set_second_row(self,newData):
self.M[1,:]=newData
t=Trajectory()
print t.M
initialData=t.get_second_row()
t.set_second_row(np.random.random(4))
print t.M
What I don't like about this is that I have to write separate set and get functions. is there a simpler way to use just one function to refer to the parts of the array M that would work for both getting and setting values?
so speaking in dummy code, something that would allow me to do this:
values=t.nth_row
t.nth_row=values+1
I would like to use t.nth_row as a reference for both getting and setting the value if that makes sense
is there a simpler way to use just one function to refer to the parts of the array M that would work for both getting and setting values?
Yes, and you've written it. It is your get function:
initialData=t.get_second_row()
t.get_second_row()[:] = np.random.random(4)
t.get_second_row()[0] = 1997
I'm trying to subclass numpy.complex64 in order to make use of the way numpy stores the data, (contiguous, alternating real and imaginary part) but use my own __add__, __sub__, ... routines.
My problem is that when I make a numpy.ndarray, setting dtype=mysubclass, I get a numpy.ndarray with dtype='numpy.complex64' in stead, which results in numpy not using my own functions for additions, subtractions and so on.
Example:
import numpy as np
class mysubclass(np.complex64):
pass
a = mysubclass(1+1j)
A = np.empty(2, dtype=mysubclass)
print type(a)
print repr(A)
Output:
<class '__main__.mysubclass'>
array([ -2.07782988e-20 +4.58546896e-41j, -2.07782988e-20 +4.58546896e-41j], dtype=complex64)'
Does anyone know how to do this?
Thanks in advance - Soren
The NumPy type system is only designed to be extended from C, via the PyArray_RegisterDataType function. It may be possible to access this functionality from Python using ctypes but I wouldn't recommend it; better to write an extension in C or Cython, or subclass ndarray as #seberg describes.
There's a simple example dtype in the NumPy source tree: newdtype_example/floatint.c. If you're into Pyrex, reference.pyx in the pytables source may be worth a look.
Note that scalars and arrays are quite different in numpy. np.complex64 (this is 32-bit float, just to note, not double precision). You will not be able to change the array like that, you will need to subclass the array instead and then override its __add__ and __sub__.
If that is all you want to do, it should just work otherwise look at http://docs.scipy.org/doc/numpy/user/basics.subclassing.html since subclassing an array is not that simple.
However if you want to use this type also as a scalar. For example you want to index scalars out, it gets more difficult at least currently. You can get a little further by defining __array_wrap__ to convert to scalars to your own scalar type for some reduce functions, for indexing to work in all cases it appears to me that you may have define your own __getitem__ currently.
In all cases with this approach, you still use the complex datatype, and all functions that are not explicitly overridden will still behave the same. #ecatmur mentioned that you can create new datatypes from the C side, if that is really what you want.