We Keep Coding

What is the 'a' in numpy.arange?

What does the 'a' in numpy's numpy.arange method stand for, and how does it differ from a simple range produced by Python's builtin range method (definitionally, not in terms of performance and whatnot)?
I tried looking online for an answer to this, but all I find is tutorials for how to use numpy.arange by GeeksForGeeks and co.

You can inspect the return types and reason about what it could mean that way:
print(type(range(0,5)))
import numpy as np
print(type(np.arange(0,5)))
Which prints:
<class 'range'>
<class 'numpy.ndarray'>
Here's a related question: Why was the name "arange" chosen for the numpy function?
Some people do from numpy import * which would shadow range which causes problems.
Naming the function arrayrange was not chosen because it's too long to type.

From the previous SO we learn that the 'a' stands, in some sense, for 'array'. arange is a function that returns a numpy array that is similar, at least in simple cases, to the list produced by list(range(...)). From the official arange docs:
For integer arguments the function is roughly equivalent to the Python built-in range, but returns an ndarray rather than a range instance.
In [104]: list(range(-3,10,2))
Out[104]: [-3, -1, 1, 3, 5, 7, 9]
In [105]: np.arange(-3,10,2)
Out[105]: array([-3, -1, 1, 3, 5, 7, 9])
In py3, range by itself is "unevaluated", it's generator like. It's the equivalent of the py2 xrange.
The best "definition" is the official documentation page:
https://numpy.org/doc/stable/reference/generated/numpy.arange.html
But maybe you are wondering when to use one or the other. The simple answer is - if you are doing python level iteration, range is usually better. If you need an array, use arange (or np.linspace as suggested by the docs).
In [106]: [x**2 for x in range(5)]
Out[106]: [0, 1, 4, 9, 16]
In [107]: np.arange(5)**2
Out[107]: array([ 0, 1, 4, 9, 16])
I often use arange to create a example array, as in:
In [108]: np.arange(12).reshape(3,4)
Out[108]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
While it is possible to make an array from a range, e.g. np.array(range(5)), that is relatively slow. np.fromiter(range(5),int) is faster, but still not as good as the direct np.arange.

The 'a' stands for 'array' in numpy.arange. Numpy.arange is a function that produces an array of sequential numbers within a given interval. It differs from Python's builtin range() function in that it can handle floating-point numbers as well as arbitrary step sizes. Also, the output of numpy.arange is an array of elements instead of a range object.

Rationale for numpy.split returning a list and not an array

I was surprised that numpy.split yields a list and not an array. I would have thought it would be better to return an array, since numpy has put a lot of work into making arrays more useful than lists. Can anyone justify numpy returning a list instead of an array? Why would that be a better programming decision for the numpy developers to have made?

A comment pointed out that if the slit is uneven, the result can't be a array, at least not one that has the same dtype. At best it would be an object dtype.
But lets consider the case of equal length subarrays:
In [124]: x = np.arange(10)
In [125]: np.split(x,2)
Out[125]: [array([0, 1, 2, 3, 4]), array([5, 6, 7, 8, 9])]
In [126]: np.array(_) # make an array from that
Out[126]:
array([[0, 1, 2, 3, 4],
[5, 6, 7, 8, 9]])
But we can get the same array without split - just reshape:
In [127]: x.reshape(2,-1)
Out[127]:
array([[0, 1, 2, 3, 4],
[5, 6, 7, 8, 9]])
Now look at the code for split. It just passes the task to array_split. Ignoring the details about alternative axes, it just does
sub_arys = []
for i in range(Nsections):
# st and end from `div_points
sub_arys.append(sary[st:end])
return sub_arys
In other words, it just steps through array and returns successive slices. Those (often) are views of the original.
So split is not that sophisticate a function. You could generate such a list of subarrays yourself without a lot of numpy expertise.
Another point. Documentation notes that split can be reversed with an appropriate stack. concatenate (and family) takes a list of arrays. If give an array of arrays, or a higher dim array, it effectively iterates on the first dimension, e.g. concatenate(arr) => concatenate(list(arr)).

Actually you are right it returns a list
import numpy as np
a=np.random.randint(1,30,(2,2))
b=np.hsplit(a,2)
type(b)
it will return type(b) as list so, there is nothing wrong in the documentation, i also first thought that the documentation is wrong it doesn't return a array, but when i checked
type(b[0])
type(b[1])
it returned type as ndarray.
it means it returns a list of ndarrary's.

numpy array TypeError: only integer scalar arrays can be converted to a scalar index

i=np.arange(1,4,dtype=np.int)
a=np.arange(9).reshape(3,3)
and
a
>>>array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
a[:,0:1]
>>>array([[0],
[3],
[6]])
a[:,0:2]
>>>array([[0, 1],
[3, 4],
[6, 7]])
a[:,0:3]
>>>array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
Now I want to vectorize the array to print them all together. I try
a[:,0:i]
or
a[:,0:i[:,None]]
It gives TypeError: only integer scalar arrays can be converted to a scalar index

Short answer:
[a[:,:j] for j in i]
What you are trying to do is not a vectorizable operation. Wikipedia defines vectorization as a batch operation on a single array, instead of on individual scalars:
In computer science, array programming languages (also known as vector or multidimensional languages) generalize operations on scalars to apply transparently to vectors, matrices, and higher-dimensional arrays.
...
... an operation that operates on entire arrays can be called a vectorized operation...
In terms of CPU-level optimization, the definition of vectorization is:
"Vectorization" (simplified) is the process of rewriting a loop so that instead of processing a single element of an array N times, it processes (say) 4 elements of the array simultaneously N/4 times.
The problem with your case is that the result of each individual operation has a different shape: (3, 1), (3, 2) and (3, 3). They can not form the output of a single vectorized operation, because the output has to be one contiguous array. Of course, it can contain (3, 1), (3, 2) and (3, 3) arrays inside of it (as views), but that's what your original array a already does.
What you're really looking for is just a single expression that computes all of them:
[a[:,:j] for j in i]
... but it's not vectorized in a sense of performance optimization. Under the hood it's plain old for loop that computes each item one by one.

I ran into the problem when venturing to use numpy.concatenate to emulate a C++ like pushback for 2D-vectors; If A and B are two 2D numpy.arrays, then numpy.concatenate(A,B) yields the error.
The fix was to simply to add the missing brackets: numpy.concatenate( ( A,B ) ), which are required because the arrays to be concatenated constitute to a single argument

This could be unrelated to this specific problem, but I ran into a similar issue where I used NumPy indexing on a Python list and got the same exact error message:
# incorrect
weights = list(range(1, 129)) + list(range(128, 0, -1))
mapped_image = weights[image[:, :, band]] # image.shape = [800, 600, 3]
# TypeError: only integer scalar arrays can be converted to a scalar index
It turns out I needed to turn weights, a 1D Python list, into a NumPy array before I could apply multi-dimensional NumPy indexing. The code below works:
# correct
weights = np.array(list(range(1, 129)) + list(range(128, 0, -1)))
mapped_image = weights[image[:, :, band]] # image.shape = [800, 600, 3]

try the following to change your array to 1D
a.reshape((1, -1))

You can use numpy.ravel to return a flattened array from n-dimensional array:
>>> a
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
>>> a.ravel()
array([0, 1, 2, 3, 4, 5, 6, 7, 8])

I had a similar problem and solved it using list...not sure if this will help or not
classes = list(unique_labels(y_true, y_pred))

this problem arises when we use vectors in place of scalars
for example in a for loop the range should be a scalar, in case you have given a vector in that place you get error. So to avoid the problem use the length of the vector you have used

I ran across this error when while trying to access elements of a list using a 1-D array. I was suggested this page but I don't the answer I was looking for.
Let l be the list and myarray be my 1D array. The correct way to access list l using elements of myarray is
np.take(l,myarray)

Seemingly inconsistent slicing behavior in numpy arrays

I ran across something that seemed to me like inconsistent behavior in Numpy slices. Specifically, please consider the following example:
import numpy as np
a = np.arange(9).reshape(3,3) # a 2d numpy array
y = np.array([1,2,2]) # vector that will be used to index the array
b = a[np.arange(len(a)),y] # a vector (what I want)
c = a[:,y] # a matrix ??
I wanted to obtain a vector such that the i-th element is a[i,y[i]]. I tried two things (b and c above) and was surprised that b and c are not the same... in fact one is a vector and the other is a matrix! I was under the impression that : was shorthand for "all elements" but apparently the meaning is somewhat more subtle.
After trial and error I somewhat understand the difference now (b == np.diag(c)), but would appreciate clarification on why they are different, what exactly using : implies, and how to understand when to use either case.
Thanks!

It's hard to understand advanced indexing (with lists or arrays) without understanding broadcasting.
In [487]: a=np.arange(9).reshape(3,3)
In [488]: idx = np.array([1,2,2])
Index with a (3,) and (3,) producing shape (3,) result:
In [489]: a[np.arange(3),idx]
Out[489]: array([1, 5, 8])
Index with (3,1) and (3,), result is (3,3)
In [490]: a[np.arange(3)[:,None],idx]
Out[490]:
array([[1, 2, 2],
[4, 5, 5],
[7, 8, 8]])
The slice : does basically the same thing. There are subtle differences, but here it's the same.
In [491]: a[:,idx]
Out[491]:
array([[1, 2, 2],
[4, 5, 5],
[7, 8, 8]])
ix_ does the same thing, converting the (3,) & (3,) to (3,1) and (1,3):
In [492]: np.ix_(np.arange(3),idx)
Out[492]:
(array([[0],
[1],
[2]]), array([[1, 2, 2]]))
A broadcasted sum might help visualize the two cases:
In [495]: np.arange(3)*10+idx
Out[495]: array([ 1, 12, 22])
In [496]: np.sum(np.ix_(np.arange(3)*10,idx),axis=0)
Out[496]:
array([[ 1, 2, 2],
[11, 12, 12],
[21, 22, 22]])

When you pass
np.arange(len(a)), y
You can view the result as being all the indexed pairs for the zipped elements you passed. In this case, indexing by np.arange(len(a)) and y
np.arange(len(a))
# [0, 1, 2]
y
# [1, 2, 2]
effectively takes elements: (0, 1), (1, 2), and (2, 2).
print(a[0, 1], a[1, 2], a[2, 2]) # 0th, 1st, 2nd elements from each indexer
# 1 5 8
In the second case, you take the entire slice along the first dimension. (Nothing before the colon.) So this is all elements along the 0th axis. You then specify with y that you want the 1st, 2nd, and 2nd element along each row. (0-indexed.)
As you pointed out, it may seem a bit unintuitive that the results are different given that the individual elements of the slice are equivalent:
a[:] == a[np.arange(len(a))]
and
a[:y] == a[:y]
However, NumPy advanced indexing cares what type of data structure you pass when indexing (tuples, integers, etc). Things can become hairy very quickly.
The detail behind that is this: first consider all NumPy indexing to be of the general form x[obj], where obj is the evaluation of whatever you passed. How NumPy "behaves" depends on what type of object obj is:
Advanced indexing is triggered when the selection object, obj, is a
non-tuple sequence object, an ndarray (of data type integer or bool),
or a tuple with at least one sequence object or ndarray (of data type
integer or bool).
...
The definition of advanced indexing means that x[(1,2,3),] is
fundamentally different than x[(1,2,3)]. The latter is equivalent to
x[1,2,3] which will trigger basic selection while the former will
trigger advanced indexing. Be sure to understand why this occurs.
In your first case, obj = np.arange(len(a)),y, a tuple that fits the bill in bold above. This triggers advanced indexing and forces the behavior described above.
As for the second case, [:,y]
When there is at least one slice (:), ellipsis (...) or np.newaxis in
the index (or the array has more dimensions than there are advanced
indexes), then the behaviour can be more complicated. It is like
concatenating the indexing result for each advanced index element.
Demonstrated:
# Concatenate the indexing result for each advanced index element.
np.vstack((a[0, y], a[1, y], a[2, y]))

Element-wise addition of 1D and 2D numpy arrays

Situation
I have objects that have attributes which are represented by numpy arrays:
>> obj = numpy.array([1, 2, 3])
where 1, 2, 3 are the attributes' values.
I'm about to write a few methods that should work equally on both a single object and a group of objects. A group of objects is represented by a 2D numpy array:
>>> group = numpy.array([[11, 21, 31],
... [12, 22, 32],
... [13, 23, 33]])
where the first digit indicates the object and the second digit indicates the attribute. That is 12 is attribute 2 of object 1 and 21 is attribute 1 of object 2.
Why this way and not transposed? Because I want the array indices to correspond to the attributes. That is object_or_group[0] should yield the first attribute either as a single number or as a numpy array, so it can be used for further computations.
Alright, so when I want to compute the dot product for example this works out of the box:
>>> obj = numpy.array([1, 2, 3])
>>> obj.dot(object_or_group)
What doesn't work is element-wise addition.
Input:
>>> group
array([[1, 2, 3],
[4, 5, 6]])
>>> obj
array([10, 20])
The resulting array should be the sum of the first element of group and obj and similar for the second element:
>>> result = numpy.array([group[0] + obj[0],
... group[1] + obj[1]])
>>> result
array([[11, 12, 13],
[24, 25, 26]])
However:
>>> group + obj
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: operands could not be broadcast together with shapes (2,3) (2,)
Which makes sense considering numpy's broadcasting rules.
It seems that there is no numpy function which performs an addition (or equivalently the broadcasting) along a specified axis. While I could use
>>> (group.T + obj).T
array([[11, 12, 13],
[24, 25, 26]])
this feels very cumbersome (and if, instead of a group, I consider a single object this feels wrong indeed). Especially because numpy covered each and every corner case for its usage, I have the feeling that I might have gotten something conceptually wrong here.
To sum it up
Similarly to
>>> obj1
array([1, 2])
>>> obj2
array([10, 20])
>>> obj1 + obj2
array([11, 22])
(which performs an element-wise - or attribute-wise - addition) I want to do the same for groups of objects:
>>> group
array([[1, 2, 3],
[4, 5, 6]])
while the layout of such a 2D group array must be such that the single objects are listed along the 2nd axis (axis=1) in order to be able to request a certain attribute (or many) via normal indexing: obj[0] and group[0] should both yield the first attribute(s).

what you want to do seems to work with this simple code !!
>>> m
array([[1, 2, 3],
[4, 5, 6]])
>>> g = np.array([10,20])
>>> m + g[ : , None]
array([[11, 12, 13],
[24, 25, 26]])

You appear to be confused about which dimension of the matrix is an object and which is an attirbute, as evidenced by the changing object size in your examples. In fact, it it the fact that you are swapping dimensions to match that changing size that is throwing you off. You are also using the unfortunate example of a 3x3 group for your dot product, which is further throwing off your explanation.
In the examples below, objects will be three-element vectors, i.e., they will have three attributes each. The example group will have consistently two rows, meaning two objects in it, and three columns, because objects have three attributes.
The first row of the group, group[0], a.k.a. group[0, :], will be the first object in the group. The first column, group[:, 0] will be the first attribute.
Here are a couple of sample objects and groups to illustrate the points that follow:
>>> obj1 = np.array([1, 2, 3])
>>> obj2 = np.array([4, 5, 6])
>>> group1 = np.array([[7, 8, 9],
[0, 1, 2]])
>>> group2 = np.array([[3, 4, 5]])
Addition will work out of the box because of broadcasting now:
>>> obj1 + obj2
array([5, 7, 9])
>>> group1 + obj1
array([[ 8, 10, 12],
[ 1, 3, 5]])
As you can see, corresponding attributes are getting added just fine. You can even add together groups, but only if they are the same size or if one of them only contains a single object:
>>> group1 + group2
array([[10, 12, 14],
[ 3, 5, 7]])
>>> group1 + group1
array([[14, 16, 18],
[ 0, 2, 4]])
The same will be true for all the binary elementwise operators: *, -, /, np.bitwise_and, etc.
The only remaining question is how to make dot products not care if they are operating on a matrix or a vector. It just so happens that dot products don't care. Your common dimension is always the number of attributes, so the second operand (the multiplier) needs to be transposed so that the number of columns becomes the number of rows. np.dot(x1, x2.T), or equivalently x1.dot(x2.T) will work correctly whether x1 and x2 are groups or objects:
>>> obj1.dot(obj2.T)
32
>>> obj1.dot(group1.T)
array([50, 8])
>>> group1.dot(obj1.T)
array([50, 8])
You can use either np.atleast_1d or np.atleast_2d to always coerce the result into a particular shape so you don't end up with a scalar like the obj1.dot(obj2.T) case. I would recommend the latter, so you always have a consistent number of dimensions regardless of the inputs:
>>> np.atleast_2d(obj1.dot(obj2.T))
array([[32]])
>>> np.atleast_2d(obj1.dot(group1.T))
array([[50, 8]])
Just keep in mind that the dimensions of the dot product will be the the number of objects in the first operand by the number of objects in the second operand (everything will be treated as a group). The attributes will get multiplied and summed together. Whether or not that has a valid interpretation for your purposes is entirely for you to decide.
UPDATE
The only remaining problem at this point is attribute access. As stated above obj1[0] and group1[0] mean very different things. There are three ways to reconcile this difference, listed in the order that I personally prefer them, with 1 being the most preferable:
Use the Ellipsis indexing object to get the last index instead of the first
>>> obj1[..., 0]
array([1])
>>> group1[..., 0]
array([7, 0])
This is the most efficient way since it does not make any copies, just does a normal index on the original arrays. As you can see, there will be no difference between the result from a single object (1D array) and a group with only one object in it (2D array).
Make all your objects 2D. As you pointed out yourself, this can be done with a decorator, and/or using np.atleast_2d. Personally, I would prefer having the convenience of using 1D arrays as single objects without having to wrap them in 2D.
Always access attributes via a transpose:
>>> obj1.T[0]
1
>>> group1.T[0]
array([7, 0])
While this is functionally equivalent to #1, it is clunky and unsightly by comparison, in addition to doing something very different under-the-hood. This approach at the very least creates a new view of the underlying array, and may run the risk of making unnecessary copies in certain cases if the group arrays are not laid out just right. I would not recommend this approach even if it does solve the problem if uniform access.