Related
I'd like to be able to reference an element of a Numpy n-dimensional array when the shape of the array itself is variable.
For a 4-D array, I know I'd be able to access some specific element index (let's say the very first one for argument) by direct referencing, for example
array[1, 1, 1, 1]
But what if the next iteration I only had a 3-dimension array? I'd need;
array[1, 1, 1]
And then a 6-dimension array? Then I'd need
array[1, 1, 1, 1, 1, 1]
Assuming that I know the appropriate index for each case, how do I go about programatically referencing array indices when the shape can change?
Thanks
you can always reshape the array and expand the index:
import numpy as np
a = np.arange(10*100*77*10*10).reshape(10,100,77,10,10)
print(a[1,2,5,9,9])
#output: 785999
def new_index(index, array):
shape = array.shape
terms = [v*np.prod(shape[i+1:]) for i,v in enumerate(index)]
return int(sum(terms))
# make a 1-D array
oneD = a.reshape(np.prod(a.shape))
print(oneD[new_index([1,2,5,9,9], a)])
#output: 785999
I'm assuming there's a numpy function that does this but I don't know it off the top of my head
hpaulj had the answer I was after. I'd tried referencing via a list (created by appending values with increasing dimension), but not via a tuple.
With the variable dimension index to be accessed as a list, first convert to a tuple, and then reference;
list_index = [1,1,1,1]
tuple_index = tuple(list_index)
result = array[tuple_index]
I have a numpy array arr of numpy arrays each with varying length. I can get the shape of arr:
arr.shape
>>> (9,)
I can get the shape of one of the elements of arr:
arr[0].shape
>>> (6, 1, 2)
And I know that all such elements have shape (n, 1, 2).
I want to slice arr to get a 1 dimensional result as follows:
arr[:,:,:,0]
But I get the following error:
IndexError: too many indices for array
EDIT: My original question was misleading. I want to do this slice so that I can assign values to the slice. So getting the slice in a new variable is not useful for my case. Essentially I want to do something like this in a simple one liner:
arr[:,:,:,0] = arr[:,:,:,0] - np.min(arr[:,:,:,0])
You can fix your first (in fact all varying ones) dimension, and apply your transformation per static-shaped elements of arr
import numpy as np
from random import randint
arr=np.array([np.random.randint(3,15, size=(randint(3,9),randint(3,7),randint(6,19))) for el in range(9)])
print(arr.shape)
print(arr[0].shape)
for i in range(arr.shape[0]):
arr[i][:,:,0]-=arr[i][:,:,0].min()
print(arr[i][:,:,0])
You could use list comprehension version of your solution.
desired_result = np.array([el[:,:,0] for el in arr])
Why do the following code samples:
np.array([[1, 2], [2, 3, 4]])
np.array([1.2, "abc"], dtype=float)
...all give the following error?
ValueError: setting an array element with a sequence.
Possible reason 1: trying to create a jagged array
You may be creating an array from a list that isn't shaped like a multi-dimensional array:
numpy.array([[1, 2], [2, 3, 4]]) # wrong!
numpy.array([[1, 2], [2, [3, 4]]]) # wrong!
In these examples, the argument to numpy.array contains sequences of different lengths. Those will yield this error message because the input list is not shaped like a "box" that can be turned into a multidimensional array.
Possible reason 2: providing elements of incompatible types
For example, providing a string as an element in an array of type float:
numpy.array([1.2, "abc"], dtype=float) # wrong!
If you really want to have a NumPy array containing both strings and floats, you could use the dtype object, which allows the array to hold arbitrary Python objects:
numpy.array([1.2, "abc"], dtype=object)
The Python ValueError:
ValueError: setting an array element with a sequence.
Means exactly what it says, you're trying to cram a sequence of numbers into a single number slot. It can be thrown under various circumstances.
1. When you pass a python tuple or list to be interpreted as a numpy array element:
import numpy
numpy.array([1,2,3]) #good
numpy.array([1, (2,3)]) #Fail, can't convert a tuple into a numpy
#array element
numpy.mean([5,(6+7)]) #good
numpy.mean([5,tuple(range(2))]) #Fail, can't convert a tuple into a numpy
#array element
def foo():
return 3
numpy.array([2, foo()]) #good
def foo():
return [3,4]
numpy.array([2, foo()]) #Fail, can't convert a list into a numpy
#array element
2. By trying to cram a numpy array length > 1 into a numpy array element:
x = np.array([1,2,3])
x[0] = np.array([4]) #good
x = np.array([1,2,3])
x[0] = np.array([4,5]) #Fail, can't convert the numpy array to fit
#into a numpy array element
A numpy array is being created, and numpy doesn't know how to cram multivalued tuples or arrays into single element slots. It expects whatever you give it to evaluate to a single number, if it doesn't, Numpy responds that it doesn't know how to set an array element with a sequence.
In my case , I got this Error in Tensorflow , Reason was i was trying to feed a array with different length or sequences :
example :
import tensorflow as tf
input_x = tf.placeholder(tf.int32,[None,None])
word_embedding = tf.get_variable('embeddin',shape=[len(vocab_),110],dtype=tf.float32,initializer=tf.random_uniform_initializer(-0.01,0.01))
embedding_look=tf.nn.embedding_lookup(word_embedding,input_x)
with tf.Session() as tt:
tt.run(tf.global_variables_initializer())
a,b=tt.run([word_embedding,embedding_look],feed_dict={input_x:example_array})
print(b)
And if my array is :
example_array = [[1,2,3],[1,2]]
Then i will get error :
ValueError: setting an array element with a sequence.
but if i do padding then :
example_array = [[1,2,3],[1,2,0]]
Now it's working.
for those who are having trouble with similar problems in Numpy, a very simple solution would be:
defining dtype=object when defining an array for assigning values to it. for instance:
out = np.empty_like(lil_img, dtype=object)
In my case, the problem was another. I was trying convert lists of lists of int to array. The problem was that there was one list with a different length than others. If you want to prove it, you must do:
print([i for i,x in enumerate(list) if len(x) != 560])
In my case, the length reference was 560.
In my case, the problem was with a scatterplot of a dataframe X[]:
ax.scatter(X[:,0],X[:,1],c=colors,
cmap=CMAP, edgecolor='k', s=40) #c=y[:,0],
#ValueError: setting an array element with a sequence.
#Fix with .toarray():
colors = 'br'
y = label_binarize(y, classes=['Irrelevant','Relevant'])
ax.scatter(X[:,0].toarray(),X[:,1].toarray(),c=colors,
cmap=CMAP, edgecolor='k', s=40)
When the shape is not regular or the elements have different data types, the dtype argument passed to np.array only can be object.
import numpy as np
# arr1 = np.array([[10, 20.], [30], [40]], dtype=np.float32) # error
arr2 = np.array([[10, 20.], [30], [40]]) # OK, and the dtype is object
arr3 = np.array([[10, 20.], 'hello']) # OK, and the dtype is also object
``
In my case, I had a nested list as the series that I wanted to use as an input.
First check: If
df['nestedList'][0]
outputs a list like [1,2,3], you have a nested list.
Then check if you still get the error when changing to input df['nestedList'][0].
Then your next step is probably to concatenate all nested lists into one unnested list, using
[item for sublist in df['nestedList'] for item in sublist]
This flattening of the nested list is borrowed from How to make a flat list out of list of lists?.
The error is because the dtype argument of the np.array function specifies the data type of the elements in the array, and it can only be set to a single data type that is compatible with all the elements. The value "abc" is not a valid float, so trying to convert it to a float results in a ValueError. To avoid this error, you can either remove the string element from the list, or choose a different data type that can handle both float values and string values, such as object.
numpy.array([1.2, "abc"], dtype=object)
>>> allData.shape
Out[72]: (8L, 161L)
>>> mask = allData[2,:]
>>> allData[[0,1,3],:][:,mask == 1] # works fine
>>> allData[[0,1,3],mask == 1] # error: ValueError: shape mismatch: objects cannot be broadcast to a single shape
Why is it that numpy arrays cannot be indexed within a single bracket []?
Changing allData[[0,1,3],mask == 1] to allData[[0,1,3],argwhere(mask == 1)] should fix it.
Brief explanation, if you slice an array by [[list1], [list2]], both lists are supposed to be lists of indices. If one of them is substituted by : to take all the element among that axis, then the other list can be a Boolean array of the same size. Only ONE of them need to be substituted by :. Consider the following 3D array example:
b=random.random((5, 10,2))
b[[0,1,3],:, some_mask==1] #works
b[:,:, some_mask==1] #works
b[[0,1,3],[2,4], some_mask==1] #ValueError
I have a numpy array and I want to delete the first 3 elements of the array. I tried this solution:
a = np.arange(0,10)
i=0
while(i<3):
del a[0]
i=i+1
This gives me an error that "ValueError: cannot delete array elements". I do not understand why this is the case. i'd appreciate the help thanks!
Numpy arrays have a fixed size, hence you cannot simply delete an element from them. The simplest way to achieve what you want is to use slicing:
a = a[3:]
This will create a new array starting with the 4th element of the original array.
For certain scenarios, slicing is just not enough. If you want to create a subarray consisting of specific elements from the original array, you can use another array to select the indices:
>>> a = arange(10, 20)
>>> a[[1, 4, 5]]
array([11, 14, 15])
So basically, a[[1,4,5]] will return an array that consists of the elements 1,4 and 5 of the original array.
It works for me:
import numpy as np
a = np.delete(a, k)
where "a" is your numpy arrays and k is the index position you want delete.
Hope it helps.
numpy arrays don't support element deletion. Why don't you just use slicing to achieve what you want?
a = a[3:]
You can convert it into a list and then try regular delete commands like pop, del, eg.
a = np.array([1,2,3,4,5])
l = list(a)
l.pop(3)
l
>>[1, 2, 3, 5]