I would like to append a value at the end of my numpy.array.
I saw numpy.append function but this performs an exact copy of the original array adding at last my new value. I would like to avoid copies since my arrays are big.
I am using resize method and then set the last index available to the new value.
Can you confirm that resize is the best way to append a value at the end?
Is it not moving memory around someway?
oldSize = myArray,shape(0)
myArray.resize( oldSize + 1 )
myArray[oldSize] = newValue
My simple timing experiment of append vs. resizing showed that resizing is about 3x faster and its the fastest way that I can think of to do this. Also, the answer to this question seems to imply that resizing the array is the way to go because it is in-place.
Verdict:
Use resize
P.S. You also might want to check out this discussion from a numpy mailing list.
Related
Here is my function to set value of 'cropped_inputs_final' array from 'cropped_inputs_index' array and 'inputs_data_list' array.
for i in range(cropped_inputs_final.shape[0]):
a1 = int(cropped_inputs_index[i])
cropped_inputs_final[i] = inputs_data_list[a1][0]
However, this for loop spend too much time.
Is there any way to replace for loop to numpy (or any else) to reduce the running time?
Thank you very much.
What you're trying to do is called vectorization, and is in general a very good thing, though it can be pretty counter intuitive sometimes.
I would need more context to give you an exact answer (what are the arrays shapes?), but array indexing seems to be what you need.
cropped_inputs_final = input_data_list[cropped_input_index][0]
I'm writing a Python script intended to split a big array of numbers into equal sub-arrays. For that purpose, I use Numpy's split method as follows:
test=numpy.array_split(raw,nslices)
where raw is the complete array containing all the values, which are float64-type by the way.
nslices is the number of sub-arrays I want to create from the raw array.
In the script, nslices may vary depending of the size of the raw array, so I would like to "automatically" save each created sub-arrays in a particular array as : resultsarray(i)in a similar way that it can be made in MATLAB/Octave.
I tried to use afor in range loop in Python but I am only able to save the last sub-array in a variable.
What is the correct way to save the sub-array for each each incrementation from 1 to nslices?
Here, the complete code as is it now (I am a Python beginner, please bother the low-level of the script).
import numpy as np
file = open("results.txt", "r")
raw = np.loadtxt(fname=file, delimiter="/n", dtype='float64')
nslices = 3
rawslice = np.array_split(raw,nslices)
for i in range(0,len(rawslice)):
resultsarray=(rawslice[i])
print(rawslice[i])
Thank you very much for your help solving this problem!
First - you screwed up delimiter :)
It should be backslash+n \n instead of /n.
Second - as Serge already mentioned in comment you can just access to split parts by index (resultarray[0] to [2]). But if you really wanted to assign each part to a separate variable you can do this in fommowing way:
result_1_of_3, result_2_of_3, result_3_of_3 = rawslice
print(result_1_of_3, result_2_of_3, result_3_of_3)
But probably it isn't the way you should go.
I want to initialise an array that will hold some data. I have created a random matrix (using np.empty) and then multiplied it by np.nan. Is there anything wrong with that? Or is there a better practice that I should stick to?
To further explain my situation: I have data I need to store in an array. Say I have 8 rows of data. The number of elements in each row is not equal, so my matrix row length needs to be as long as the longest row. In other rows, some elements will not be filled. I don't want to use zeros since some of my data might actually be zeros.
I realise I can use some value I know my data will never, but nans is definitely clearer. Just wondering if that can cause any issues later with processing. I realise I need to use nanmax instead of max and so on.
I have created a random matrix (using np.empty) and then multiplied it by np.nan. Is there anything wrong with that? Or is there a better practice that I should stick to?
You can use np.full, for example:
np.full((100, 100), np.nan)
However depending on your needs you could have a look at numpy.ma for masked arrays or scipy.sparse for sparse matrices. It may or may not be suitable, though. Either way you may need to use different functions from the corresponding module instead of the normal numpy ufuncs.
A way I like to do it which probably isn't the best but it's easy to remember is adding a 'nans' method to the numpy object this way:
import numpy as np
def nans(n):
return np.array([np.nan for i in range(n)])
setattr(np,'nans',nans)
and now you can simply use np.nans as if it was the np.zeros:
np.nans(10)
I'm looking for the most efficient (=fastest) way of changing an item's value in a numpy array. Assumming I have an array a = np.zeros((N,N)), and I want to set a value at i,i to x, I have found following option: np.itemset((i,i), x) - this one seems fast, but it is not inplace. There is also numpy.put, but it seems to be slow... Are there any other options?
I have some data represented in a 1300x1341 matrix. I would like to split this matrix in several pieces (e.g. 9) so that I can loop over and process them. The data needs to stay ordered in the sense that x[0,1] stays below (or above if you like) x[0,0] and besides x[1,1].
Just like if you had imaged the data, you could draw 2 vertical and 2 horizontal lines over the image to illustrate the 9 parts.
If I use numpys reshape (eg. matrix.reshape(9,260,745) or any other combination of 9,260,745) it doesn't yield the required structure since the above mentioned ordering is lost...
Did I misunderstand the reshape method or can it be done this way?
What other pythonic/numpy way is there to do this?
Sounds like you need to use numpy.split() which has its documentation here ... or perhaps its sibling numpy.array_split() here. They are for splitting an array into equal subsections without re-arranging the numbers like reshape does,
I haven't tested this but something like:
numpy.array_split(numpy.zeros((1300,1341)), 9)
should do the trick.
reshape, to quote its docs,
Gives a new shape to an array without
changing its data.
In other words, it does not move the array's data around at all -- it just affects the array's dimension. You, on the other hand, seem to require slicing; again quoting:
It is possible to slice and stride
arrays to extract arrays of the same
number of dimensions, but of different
sizes than the original. The slicing
and striding works exactly the same
way it does for lists and tuples
except that they can be applied to
multiple dimensions as well.
So for example thearray[0:260, 0:745] is the "upper leftmost part, thearray[260:520, 0:745] the upper left-of-center part, and so forth. You could have references to the various parts in a list (or dict with appropriate keys) to process them separately.