my first question, surfed a lot!
I have an array in Numpy, like
myarray=np.zeros((raws,cols))
then I have raws*cols one dimensional numpy array, all same lenght, let's say deep
then I would insert each one of this one dimensional array into myarray.
expected result:
newarray.shape
(raws,cols,deep)
I use this in a bigger function and the fact I operate this way is due to a parallelization paradigma.
Thank you in advance.
Related
Let's say I have a 2d python (rectangular i.e. like matrix) list which I want to sort in descending order based on its 2nd column, and I want the list to change and not interested in a copy. What is the best alternative to this approach (using numpy, ...)
arr.sort(key= lambda i:i[1],revere=True)
I've searched a lot but couldn't find an intuitive way that seemed better that the code above.
Any help would be highly appreciated.
You have to give some more informations. A 2d list can be something more than a rectangular 2d numpy array.
In this answer i asume that your data can be represented as a 2d- array.
import numpy as np
np_arr=np.array(arr) #create a numpy array from your list
idx=np.argsort(-np_arr[:,1])
np_arr=np_arr[idx,:]
I generate feature vectors for examples from large amount of data, and I would like to store them incrementally while i am reading the data. The feature vectors are numpy arrays. I do not know the number of numpy arrays in advance, and I would like to store/retrieve them incrementally.
Looking at pytables, I found two options:
Arrays: They require predetermined size and I am not quite sure how
much appending is computationally efficient.
Tables: The column types do not support list or arrays.
If it is a plain numpy array, you should probably use Extendable Arrays (EArray) http://pytables.github.io/usersguide/libref/homogenous_storage.html#the-earray-class
If you have a numpy structured array, you should use a Table.
Can't you just store them into an array? You have your code and it should be a loop that will grab things from the data to generate your examples and then it generates the example. create an array outside the loop and append your vector into the array for storage!
array = []
for row in file:
#here is your code that creates the vector
array.append(vector)
then after you have gone through the whole file, you have an array with all of your generated vectors! Hopefully that is what you need, you were a bit unclear...next time please provide some code.
Oh, and you did say you wanted pytables, but I don't think it's necessary, especially because of the limitations you mentioned
I have a large 4D array (time,height,latitude,longitude) of float values. I want to efficiently force any values in the array that are greater than 100.0 to be 100.0. I think the map function (+lambda?) can do this, but I'm stuck. Currently I have a crude for loop that goes through each index, but this is taking much too long!
Thanks for your help in advance!
Solution: numpy.clip(array,0,100.0)
In order to be efficient, you should probably be using NumPy.
With NumPy you can have space efficient multidimiensional arrays and ready to use solution to your problem.
I have a large 4D array (time,height,latitude,longitude) of float values. I want to efficiently
stop.
Use numpy.
I need to diagonalise a very large number of matrices.
These matrices are by themselves quite small (say a x a where a<=10) but due to
their sheer number, it takes a lot of time to diagonalise them all using a for loop
and the numpy.linalg.eig function. So I wanted to make an array of matrices, i.e.,
an array of 2D arrays, but unfortunately, Python seems to consider this to be a 3-dimensional array, gets confused and refuses to do the job. So, is there any way to prevent Python from looking at this array of 2D arrays as a 3D array?
Thanks,
A Python novice
EDIT: To be more clear, I'm not interested in this 3D array per se. Since in general, feeding an array to a function seems to be much faster than using a for loop to feed all elements one by one, I just tried to put all matrices which I need to diagonalise in an array.
If you have an 3D array like:
a = np.random.normal(size=(20,10,10))
you can then just loop through all 20 of the 10x10 arrays using:
for k in xrange(a.shape[0]):
b = np.linalg.eig(a[k,:,:])
where you would save b in a more sophisticated way. This may be what you are already doing, but you can't apply np.linalg.eig to a 3D array and have it calculate along a single axis, so you are stuck with the loop unless there is a formalism for combining all of your arrays into a single 2D array. I doubt however that that would be faster than just looping over the individual 2D arrays.
I want to create a MATLAB-like cell array in Numpy. How can I accomplish this?
Matlab cell arrays are most similar to Python lists, since they can hold any object - but scipy.io.loadmat imports them as numpy object arrays - which is an array with dtype=object.
To be honest though you are just as well off using Python lists - if you are holding general objects you will loose almost all of the advantages of numpy arrays (which are designed to hold a sequence of values which each take the same amount of memory).