We Keep Coding

Numpy: Compute dot product of 2d arrays with a condition before summation

I have a problem where I have two arrays with dimensions
n x d and d x h, where n, d, and h are positive integers that may or may not be the same. I am trying to find the dot product of the two matrices but with the condition that for each multiplication, I apply a function g to the term.
For example, in a case where n=2, d=3, and h=3
If I had the following matrices:
a = [[1, 2, 3],
[4, 5, 6]]
b = [[1, 2, 3],
[4, 5, 6],
[1, 1, 1]]
I would want to find c such that
c = [[g(1*1)+g(2*4)+g(3*1), g(1*2)+g(2*5)+g(3*1), g(1*3)+g(2*6)+g(3*1)],
[g(4*1)+g(5*4)+g(6*1), g(4*2)+g(5*5)+g(6*1), g(4*3)+g(5*6)+g(6*1)]]
Any help would be appreciated

I was able to do this by first doing the multiplications with broadcasting, applying g(), and then summing across the correct axis:
import numpy as np
def g(x):
return 1 / (1 + np.exp(-x))
a = np.array([[1, 2, 3],
[4, 5, 6]])
b = np.array([[1, 2, 3],
[4, 5, 6],
[1, 1, 1]])
First, the multiplication a[:, None] * b.T (probably a nicer way to do this), then evaluate g(x), then sum across axis 2:
>>> g(a[:, None] * b.T).sum(axis=2)
array([[2.68329736, 2.83332581, 2.90514211],
[2.97954116, 2.99719203, 2.99752123]])
Verifying that the first row indeed matches your desired result:
>>> g(1*1) + g(2*4) + g(3*1)
2.683297355321972
>>> g(1*2) + g(2*5) + g(3*1)
2.8333258069316134
>>> g(1*3) + g(2*6) + g(3*1)
2.9051421094702645

Average difference between ints in two lists in one line - Python

There are two non-empty lists, containing only ints, both have the same length.
Our function needs to return the average absolute difference between ints of same index.
For example, for the lists [1, 2, 3, 4] and [1, 1, 1, 1], the answer will be 1.5.
The function needs to be completed in one line.
I had a little something that does that, but as you can probably guess, it's not a one-liner:
def avg_diff(a, b):
sd = 0.0
for x, y in zip(a, b):
sd += abs(x - y)
return sd / len(a)
Thanks.

In Python 3.4 we got some statistic functions in the standard library, including statistics.mean.
Using this function and a generator-expression:
from statistics import mean
a = [1, 2, 3, 4]
b = [1, 1, 1, 1]
mean(abs(x - y) for x, y in zip(a, b))
# 1.5

a = [1, 2, 3, 4]
b = [1, 1, 1, 1]
sum([abs(i - j) for i, j in zip(a,b)]) / float(len(a))

If you are happy to use a 3rd party library, numpy provides one way:
import numpy as np
A = np.array([1, 2, 3, 4])
B = np.array([1, 1, 1, 1])
res = np.mean(np.abs(A - B))
# 1.5

Using the in-built sum and len functions on list:
lst1 = [1, 2, 3, 4]
lst2 = [1, 1, 1, 1]
diff = [abs(x-y) for x, y in zip(lst1, lst2)] # find index-wise differences
print(sum(diff)/len(diff)) # divide sum of differences by total
# 1.5

Is there a function in numpy where it will return a matrix of each row of vector raised from a power of 0 to integer

Suppose I have a vertical vector X = [1,[2],[3]] and integer pow = 2.
Is there a function in numpy where it will return a matrix of each row of vector x raised from a power of 0 to pow (pow = 2)
The above example should return a matrix
[[1, 1, 1],
[1, 2, 4],
[1, 3, 9]]
I looked at numpy.power, however returns an array.

The function you are looking for is
numpy.vander

I don't think there is a specific function, but you could use array broadcasting.
X = np.array([[1],[2],[3]])
p = np.arange(0, 2+1) # powers
X**p # row vs column vector broadcasts to 2D matrix

Following the rules of broadcasting, we could extend to 2D and then raise to power with a ranged array -
X[:,None]**np.arange(3) # Or np.power(X[:,None], np.arange(3))
Sample run -
In [7]: X = np.array([1,2,3])
In [8]: X[:,None]**np.arange(3)
Out[8]:
array([[1, 1, 1],
[1, 2, 4],
[1, 3, 9]])
If X is already extended, just raise to power -
In [24]: X = np.array([[1],[2],[3]])
In [25]: X**np.arange(3)
Out[25]:
array([[1, 1, 1],
[1, 2, 4],
[1, 3, 9]])

If n is large, I would suggest you use nested loop instead. Calculating power is expensive. Do something like this:
arr = [1]
for i in range(1, p + 1):
arr.append(p * arr[-1])

Find nearest neighbour in a more pythonic way

A is a point, and P is a list of points.
I want to find which point P[i] is the closest to A, i.e. I want to find P[i_0] with:
i_0 = argmin_i || A - P[i]||^2
I do it this way:
import numpy as np
# P is a list of 4 points
P = [np.array([-1, 0, 7, 3]), np.array([5, -2, 8, 1]), np.array([0, 2, -3, 4]), np.array([-9, 11, 3, 4])]
A = np.array([1, 2, 3, 4])
distance = 1000000000 # better would be : +infinity
closest = None
for p in P:
delta = sum((p - A)**2)
if delta < distance:
distance = delta
closest = p
print closest # the closest point to A among all the points in P
It works, but how to do this in a shorter/more Pythonic way?
More generally in Python (and even without using Numpy), how to find k_0 such that D[k_0] = min D[k]? i.e. k_0 = argmin_k D[k]

A more Pythonic way of implementing the same algorithm you're using is to replace your loop with a call to min with a key function:
closest = min(P, key=lambda p: sum((p - A)**2))
Note that I'm using ** for exponentiation (^ is the binary-xor operator in Python).

A fully vectorized approach in numpy. Similar to the one of #MikeMüller, but using numpy's broadcasting to avoid lambda functions.
With the example data:
>>> P = [np.array([-1, 0, 7, 3]), np.array([5, -2, 8, 1]), np.array([0, 2, -3, 4]), np.array([-9, 11, 3, 4])]
>>> A = np.array([1, 2, 3, 4])
And making P a 2D numpy array:
>>> P = np.asarray(P)
>>> P
array([[-1, 0, 7, 3],
[ 5, -2, 8, 1],
[ 0, 2, -3, 4],
[-9, 11, 3, 4]])
It can be computed in one line using numpy:
>>> P[np.argmin(np.sum((P - A)**2, axis=1))]
Note that P - A, with P.shape = (N, 4) and A.shape = (4,) will brooadcast the substraction to all the rows of P (Pi = Pi - A).
For small N (number of rows in P), the pythonic approach is probably faster. For large values of N this should be significantly faster.

A NumPy version as one-liner:
clostest = P[np.argmin(np.apply_along_axis(lambda p: np.sum((p - A) **2), 1, P))]

Usage of the builtin min is the way for this:
import math
p1 = [1,2]
plst = [[1,3], [10,10], [5,5]]
res = min(plst, key=lambda x: math.sqrt(pow(p1[0]-x[0], 2) + pow(p1[1]-x[1], 2)))
print res
[1, 3]
Note that I just used plain python lists.

How to update a Numpy array sequentially without loop

I have a Numpy array v and I want to update each element using a function on the current element of the array :
v[i] = f(v, i)
A basic way to do this is to use a loop
for i in xrange(2, len(v)):
v[i] = f(v, i)
Hence the value used to update v[i] is the updated array v. Is there a way to do these updates without a loop ?
For example,
v = [f(v, i) for i in xrange(len(v))]
does not work since the v[i-1] is not updated when it is used in the comprehensive list.
IThe function f can depend on several elements on the list, those with index lower than i should be updated and those with an index greater than i are not yet updated, as in the following example :
v = [1, 2, 3, 4, 5]
f = lambda v, i: (v[i-1] + v[i]) / v[i+1] # for i = [1,3]
f = lambda v, i: v[i] # for i = {0,4}
it should return
v = [1, (1+2)/3, (1+4)/4, ((5/4)+4)/5, 5]

There is a function for this:
import numpy
v = numpy.array([1, 2, 3, 4, 5])
numpy.add.accumulate(v)
#>>> array([ 1, 3, 6, 10, 15])
This works on many different types of ufunc:
numpy.multiply.accumulate(v)
#>>> array([ 1, 2, 6, 24, 120])
For an arbitrary function doing this kind of accumulation, you can make your own ufunc, although this will be much slower:
myfunc = numpy.frompyfunc(lambda x, y: x + y, 2, 1)
myfunc.accumulate([1, 2, 3], dtype=object)
#>>> array([1, 3, 6], dtype=object)

you can use sum function for sum the numbers before v[i]:
>>> v = [v[i] + sum(v[:i]) for i in xrange(len(v))]
>>> v
[1, 3, 6, 10, 15]
or in a better way you can use np.cumsum()
>>> np.cumsum(v)
array([ 1, 3, 6, 10, 15])