We Keep Coding

Calling np.sum(np.fromiter(generator))

I have a generator that returns numpy arrays.
For example sake, let it be:
import numpy as np
a = np.arange(9).reshape(3,3)
gen = (x for x in a)
Calling:
np.sum(gen)
On numpy 1.17.4:
DeprecationWarning: Calling np.sum(generator) is deprecated, and in
the future will give a different result. Use
np.sum(np.fromiter(generator)) or the python sum builtin instead.
Trying to refactor the above:
np.sum(np.fromiter(gen, dtype=np.ndarray))
I get:
ValueError: cannot create object arrays from iterator
What is wrong in the above statement?

The problem is the second argument, np.ndarray in the fromiter(). Numpy fromiter expected a 1D and returns a 1D array:
Create a new 1-dimensional array from an iterable object.
Therefore, you cannot create object arrays from iterator. Furthermore the .reshape() will also raise an error, because of what I stated in the first line. All in all, this works:
import numpy as np
a = np.arange(9)
gen = (x for x in a)
print(np.sum(np.fromiter(gen,float)))
Output:
36

Since you're summing instances of arrays you can just use the built-in sum:
result = sum(gen)

What about simply converting your generator to a list and then passing it to the np.sum?
a = np.arange(9).reshape(3,3)
gen = (x for x in a)
Summing all the elements:
>>> np.sum(list(gen))
36
Summing column-wise:
>>> np.sum(list(gen), axis=0)
array([ 9, 12, 15])
Summing row-wise:
>>> np.sum(list(gen), axis=1)
array([ 3, 12, 21])

How to get element to element power in two arrays in python?

I have two arrays with 3 elements in each.
reduction_combs = [2, 3, 7]
elements = [3,6,8]
Is there a shortway to compute new array which is :
c = [2**3 , 3**6, 7**8]

This can be achieved using a simple list comprehension.
[x ** y for (x, y) in zip(elements, reduction_combs)]

Yes, you can just do [x**y for (x,y) in zip(reduction_combs, elements)]

You can also use map with lambda expressions passing two lists:
c = list(map(lambda x,y: x**y, reduction_combs, elements))
Where x and y will be values from reduction_combs and elements, respectively.

In addition to the zip method, this is another way using enumerate and list comprehension. Here j is the element of reduction_combs and i is the corresponding index using which you fetch the power to be raised from elements
c = [j**elements[i] for i, j in enumerate(reduction_combs)]

Using numpy arrays:
import numpy as np
a = np.array([2, 3, 7])
b = np.array([3, 6, 8])
a ** b
# output: array([8, 729,5764801])

You can do this either using numpy:
import numpy
reduction_combs = numpy.array([2, 3, 7])
elements = numpy.array([3, 6, 8])
c = reduction_combs ** elements
or if you want to do it with plain python, you might want to consider list comprehension:
c = [reduction_combs[i] ** elements[i] for i in range(len(reduction_combs))]
You should learn a bit more about what lists do in python and if you often work with arrays, get used to work with numpy!

If you like functional style as an alternative to the excellent list comprehension proposed by tda, here's a solution with operator.pow and itertools.starmap.
>>> from operator import pow
>>> from itertools import starmap
>>> list(starmap(pow, zip(reduction_combs, elements)))
[8, 729, 5764801]
In addition, since you tagged numpy, leveraging element-wise vectorized operations makes for a very straight forward solution.
>>> import numpy as np
>>> r = np.array(reduction_combs)
>>> e = np.array(elements)
>>>
>>> r**e
array([ 8, 729, 5764801])

You could use the numpy power function:
import numpy as np
reduction_combs = [2, 3, 7]
elements = [3, 6, 8]
print(np.power(reduction_combs, elements))
Output
[ 8 729 5764801]
If you want the output as a list simply do:
np.power(reduction_combs, elements).tolist()

One of the quick solution would be:
c = [a**b for a,b in zip(reduction_combs, elements)]
You can also try using numpy as below:
import numpy as np
c = np.power(reduction_combs, elements)

Using enumerate and pow
c = [pow(v, elements[i]) for i, v in enumerate(reduction_combs)]

Add values in numpy array successively, without looping [duplicate]

This question already has answers here:
Numpy sum elements in array based on its value
(2 answers)
Closed 4 years ago.
Maybe has been asked before, but I can't find it.
Sometimes I have an index I, and I want to add successively accordingly to this index to an numpy array, from another array. For example:
A = np.array([1,2,3])
B = np.array([10,20,30])
I = np.array([0,1,1])
for i in range(len(I)):
A[I[i]] += B[i]
print(A)
prints the expected (correct) value:
[11 52 3]
while
A[I] += B
print(A)
results in the expected (wrong) answer
[11 32 3].
Is there any way to do what I want in a vectorized way, without the loop?
If not, which is the fastest way to do this?

Use numpy.add.at:
>>> import numpy as np
>>> A = np.array([1,2,3])
>>> B = np.array([10,20,30])
>>> I = np.array([0,1,1])
>>>
>>> np.add.at(A, I, B)
>>> A
array([11, 52, 3])
Alternatively, np.bincount:
>>> A = np.array([1,2,3])
>>> B = np.array([10,20,30])
>>> I = np.array([0,1,1])
>>>
>>> A += np.bincount(I, B, minlength=A.size).astype(int)
>>> A
array([11, 52, 3])
Which is faster?
Depends. In this concrete example add.at seems marginally faster, presumably because we need to convert types in the bincount solution.
If OTOH A and B were float dtype then bincount would be faster.

You need to use np.add.at:
A = np.array([1,2,3])
B = np.array([10,20,30])
I = np.array([0,1,1])
np.add.at(A, I, B)
print(A)
prints
array([11, 52, 3])
This is noted in the doc:
ufunc.at(a, indices, b=None)
Performs unbuffered in place operation on operand ‘a’ for elements specified by ‘indices’. For addition ufunc, this method is equivalent to a[indices] += b, except that results are accumulated for elements that are indexed more than once. For example, a[[0,0]] += 1 will only increment the first element once because of buffering, whereas add.at(a, [0,0], 1) will increment the first element twice.

Python language/syntax usage

New to python but ran into something I don't understand. The following line of code:
diff = features[0:] - test[0] # <- this is what I don't get
is used thusly:
x = diff[i]
to return the element-wise difference between features[i] and test[0]. Can anyone point to a language reference for this or explain it? I understand how the result can be developed using "def" or "lambda" but I don't understand the construction.
thanks!

the answer depends on what features[0:] and test[0] evaluate to.
if test[0] is a number and features[0:] is a numpy array, then you may be using numpy to subtract a number from each element in a list:
>>> import numpy
>>> array = numpy.array([49, 51, 53, 56])
>>> array - 13
array([36, 38, 40, 43])

feature appears to be a Numpy array. Numpy arrays 'broadcast' scalar operations to the whole array.
import numpy as np
asd = np.full([10,10], 10, np.int64)
asd /= 5
print asd # prints a 10x10 array of 2s

Using searchsorted for 2D array

I would like to have the index like function :np.searchsorted([1,2,3,4,5], 3) returns 2 but in case 2-D dimension: np.searchsorted([[1,2],[3,4],[5,6]], [5,6]) should return 2. How can I do that?

a = np.array([[1,2],[3,4],[5,6]])
b = np.array([5,6])
np.where(np.all(a==b,axis=1))

The documentation for searchsorted indicates that it only works on 1-D arrays.
You can find the index location in a list using built-in methds:
>>> a = [[1,2],[3,4],[5,6]]
>>> a.index([5,6])
2
>>> a = [[1,2],[5,6],[3,4]]
>>> print(a.index([5,6]))
>>> a.sort()
>>> print(a.index([5,6]))
1
2