We Keep Coding

Create lower triangular matrix from a vector in python

I want to create a python program which computes a matrix from a vector with some coefficients. So lets say we have the following vector of coefficients c = [c0, c1, c2] = [0, 1, 0], then I want to compute the matrix:
So how do I go from the vector c to creating a lower triangular matrix A. I know how to index it manually, but I need a program that can do it. I was maybe thinking about a for-loop inside another for-loop but I struggle with how it is done practically, what do you guys think should be done here?

One way (assuming you're using plain arrays and not numpy or anything):
src = [0, 1, 0]
dst = [
[
src[i-j] if i >= j else 0
for j in range(len(src))
] for i in range(len(src))
]

You can try the following:
import numpy as np
c = [1, 2, 3, 4, 5]
n = len(c)
a = np.zeros((n,n))
for i in range(n):
np.fill_diagonal(a[i:, :], c[i])
print(a)
It gives:
[[1. 0. 0. 0. 0.]
[2. 1. 0. 0. 0.]
[3. 2. 1. 0. 0.]
[4. 3. 2. 1. 0.]
[5. 4. 3. 2. 1.]]

numpy bincount - choose the max weight instead of sum all weights

Is it possible to use np.bincount but get the max instead of sum of weights? Here, bbb at index 3 has two values, 11.1 and 55.5. I want to have 55.5, not 66.6. I doubt I choose use other function but not so sure which one is good for this purpose.
bbb = np.array([ 3, 7, 11, 13, 3])
weight = np.array([ 11.1, 22.2, 33.3, 44.4, 55.5])
print np.bincount(bbb, weight, minlength=15)
OUT >> [ 0. 0. 0. 66.6 0. 0. 0. 22.2 0. 0. 0. 33.3 0. 44.4 0. ]
Note that, in fact, bbb and weight are very large (about 5e6 elements).

The solution to your 2D question is also valid for the 1D case, so you can use np.maxmimum.at
out = np.zeros(15)
np.maximum.at(out, bbb, weight)
# array([ 0. , 0. , 0. , 55.5, 0. , 0. , 0. , 22.2, 0. ,
# 0. , 0. , 33.3, 0. , 44.4, 0. ])

Approach #1 : Here's one way with np.maximum.reduceat to get the binned maximum values -
def binned_max(bbb, weight, minlength):
sidx = bbb.argsort()
weight_s = weight[sidx]
bbb_s = bbb[sidx]
cut_idx = np.flatnonzero(np.concatenate(([True], bbb_s[1:] != bbb_s[:-1])))
bbb_unq = bbb_s[cut_idx]
#Or bbb_unq, cut_idx = np.unique(bbb_s, return_index=1)
max_val = np.maximum.reduceat(weight_s, cut_idx)
out = np.zeros(minlength, dtype=weight.dtype)
out[bbb_unq] = max_val
return out
Sample run -
In [36]: bbb = np.array([ 3, 7, 11, 13, 3])
...: weight = np.array([ 11.1, 22.2, 33.3, 44.4, 55.5])
In [37]: binned_max(bbb, weight, minlength=15)
Out[37]:
array([ 0. , 0. , 0. , 55.5, 0. , 0. , 0. , 22.2, 0. ,
0. , 0. , 33.3, 0. , 44.4, 0. ])
Approach #2 : Well I was trying to check out/having fun with numba to solve this and it seems quite efficient. Here's one numba way -
from numba import njit
#njit
def numba_func(out, bins, weight, minlength):
l = len(bins)
for i in range(l):
if out[bins[i]] < weight[i]:
out[bins[i]] = weight[i]
return out
def maxat_numba(bins, weight, minlength):
out = np.zeros(minlength, dtype=weight.dtype)
out[bins] = weight.min()
numba_func(out, bins, weight, minlength)
return out
Runtime test -
The built-in with np.maximum.at looks quite neat and would be the preferred one in most scenarios, so testing the proposed one against it -
# #Nils Werner's soln with np.maximum.at
def maxat_numpy(bins, weight, minlength):
out = np.zeros(minlength)
np.maximum.at(out, bins, weight)
return out
Timings -
Case #1 :
In [155]: bbb = np.random.randint(1,1000, (10000))
In [156]: weight = np.random.rand(*bbb.shape)
In [157]: %timeit maxat_numpy(bbb, weight, minlength=bbb.max()+1)
1000 loops, best of 3: 686 µs per loop
In [158]: %timeit maxat_numba(bbb, weight, minlength=bbb.max()+1)
10000 loops, best of 3: 60.6 µs per loop
Case #2 :
In [159]: bbb = np.random.randint(1,10000, (1000000))
In [160]: weight = np.random.rand(*bbb.shape)
In [161]: %timeit maxat_numpy(bbb, weight, minlength=bbb.max()+1)
10 loops, best of 3: 66 ms per loop
In [162]: %timeit maxat_numba(bbb, weight, minlength=bbb.max()+1)
100 loops, best of 3: 5.42 ms per loop

Probably not quite as fast as the answer by Nils, but the numpy_indexed package (disclaimer: I am its author) has a more flexible syntax for performing these type of operations:
import numpy_indexed as npi
unique_keys, maxima_per_key = npi.group_by(bbb).max(weight)

Repeating numpy values and specifying dtype

I want to generate a numpy array of the form:
0.5*[[0, 0], [1, 1], [2, 2], ...]
I want the final array to have a dtype of numpy.float32.
Here is my attempt:
>>> import numpy as np
>>> N = 5
>>> x = np.array(np.repeat(0.5*np.arange(N), 2), np.float32)
>>> x
array([ 0. , 0. , 0.5, 0.5, 1. , 1. , 1.5, 1.5, 2. , 2. ], dtype=float32)
Is this a good way? Can I avoid the copy (if it is indeed copying) just for type conversion?

You only has to reshape your final result to obtain what you want:
x = x.reshape(-1, 2)
You could also run arange passing the dtype:
x = np.repeat(0.5*np.arange(N, dtype=np.float32), 2).reshape(-1, 2)
You can easily cast the array as another type using the astype method, which accepts an argument copy:
x.astype(np.int8, copy=False)
But, as explained in the documentation, numpy checks for some requirements in order to return the view. If those requirements are not satisfied, a copy is returned.
You can check if a given array is a copy or a view from another by checking the OWNDATA attribute, accessible through the flags property of the ndarray.
EDIT: more on checking if a given array is a copy...
Is there a way to check if numpy arrays share the same data?

An alternative:
np.array([0.5*np.arange(N, dtype=np.float32)]*2)
Gives:
array([[ 0. , 0.5, 1. , 1.5, 2. ],
[ 0. , 0.5, 1. , 1.5, 2. ]], dtype=float32)
You might want to rotate it:
np.rot90(np.array([0.5*np.arange(N, dtype=np.float32)]*2),3)
Giving:
array([[ 0. , 0. ],
[ 0.5, 0.5],
[ 1. , 1. ],
[ 1.5, 1.5],
[ 2. , 2. ]], dtype=float32)
Note, this is slower than #Saullo_Castro's answer:
np.rot90(np.array([0.5*np.arange(N, dtype=np.float32)]*2),3)
10000 loops, best of 3: 24.3us per loop
np.repeat(0.5*np.arange(N, dtype=np.float32), 2).reshape(-1, 2)
10000 loops, best of 3: 9.23 us per loop
np.array(np.repeat(0.5*np.arange(N), 2), np.float32).reshape(-1, 2)
10000 loops, best of 3: 10.4 us per loop
(using %%timeit on ipython)

Interleave rows of two numpy arrays in Python

I wanted to interleave the rows of two numpy arrays of the same size.
I came up with this solution.
# A and B are same-shaped arrays
A = numpy.ones((4,3))
B = numpy.zeros_like(A)
C = numpy.array(zip(A[::1], B[::1])).reshape(A.shape[0]*2, A.shape[1])
print(C)
Outputs
[[ 1. 1. 1.]
[ 0. 0. 0.]
[ 1. 1. 1.]
[ 0. 0. 0.]
[ 1. 1. 1.]
[ 0. 0. 0.]
[ 1. 1. 1.]
[ 0. 0. 0.]]
Is there a cleaner, faster, better, numpy-only way?

It is maybe a bit clearer to do:
A = np.ones((4,3))
B = np.zeros_like(A)
C = np.empty((A.shape[0]+B.shape[0],A.shape[1]))
C[::2,:] = A
C[1::2,:] = B
and it's probably a bit faster as well, I'm guessing.

I find the following approach using numpy.hstack() quite readable:
import numpy as np
a = np.ones((2,3))
b = np.zeros_like(a)
c = np.hstack([a, b]).reshape(4, 3)
print(c)
Output:
[[ 1. 1. 1.]
[ 0. 0. 0.]
[ 1. 1. 1.]
[ 0. 0. 0.]]
It is easy to generalize this to a list of arrays of the same shape:
arrays = [a, b, c,...]
shape = (len(arrays)*a.shape[0], a.shape[1])
interleaved_array = np.hstack(arrays).reshape(shape)
It seems to be a bit slower than the accepted answer of #JoshAdel on small arrays but equally fast or faster on large arrays:
a = np.random.random((3,100))
b = np.random.random((3,100))
%%timeit
...: C = np.empty((a.shape[0]+b.shape[0],a.shape[1]))
...: C[::2,:] = a
...: C[1::2,:] = b
...:
The slowest run took 9.29 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 3.3 µs per loop
%timeit c = np.hstack([a,b]).reshape(2*a.shape[0], a.shape[1])
The slowest run took 5.06 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 10.1 µs per loop
a = np.random.random((4,1000000))
b = np.random.random((4,1000000))
%%timeit
...: C = np.empty((a.shape[0]+b.shape[0],a.shape[1]))
...: C[::2,:] = a
...: C[1::2,:] = b
...:
10 loops, best of 3: 23.2 ms per loop
%timeit c = np.hstack([a,b]).reshape(2*a.shape[0], a.shape[1])
10 loops, best of 3: 21.3 ms per loop

You can stack, transpose, and reshape:
numpy.dstack((A, B)).transpose(0, 2, 1).reshape(A.shape[0]*2, A.shape[1])

Efficient standard basis vector with numpy

Given an index and a size, is there a more efficient way to produce the standard basis vector:
import numpy as np
np.array([1.0 if i == index else 0.0 for i in range(size)])

In [2]: import numpy as np
In [9]: size = 5
In [10]: index = 2
In [11]: np.eye(1,size,index)
Out[11]: array([[ 0., 0., 1., 0., 0.]])
Hm, unfortunately, using np.eye for this is rather slow:
In [12]: %timeit np.eye(1,size,index)
100000 loops, best of 3: 7.68 us per loop
In [13]: %timeit a = np.zeros(size); a[index] = 1.0
1000000 loops, best of 3: 1.53 us per loop
Wrapping np.zeros(size); a[index] = 1.0 in a function makes only a modest difference, and is still much faster than np.eye:
In [24]: def f(size, index):
....: arr = np.zeros(size)
....: arr[index] = 1.0
....: return arr
....:
In [27]: %timeit f(size, index)
1000000 loops, best of 3: 1.79 us per loop

x = np.zeros(size)
x[index] = 1.0
at least i think thats it...
>>> t = timeit.Timer('np.array([1.0 if i == index else 0.0 for i in range(size)]
)','import numpy as np;size=10000;index=5123')
>>> t.timeit(10)
0.039461429317952934 #original method
>>> t = timeit.Timer('x=np.zeros(size);x[index]=1.0','import numpy as np;size=10000;index=5123')
>>> t.timeit(10)
9.4077963240124518e-05 #zeros method
>>> t = timeit.Timer('x=np.eye(1.0,size,index)','import numpy as np;size=10000;index=5123')
>>> t.timeit(10)
0.0001398340635319073 #eye method
looks like np.zeros is fastest...

I'm not sure if this is faster, but it's definitely more clear to me.
a = np.zeros(size)
a[index] = 1.0

Often, you need not one but all basis vectors. If this is the case, consider np.eye:
basis = np.eye(3)
for vector in basis:
...
Not exactly the same, but closely related: This even works to get a set of basis matrices with a bit of tricks:
>>> d, e = 2, 3 # want 2x3 matrices
>>> basis = np.eye(d*e,d*e).reshape((d*e,d,e))
>>> print(basis)
[[[ 1. 0. 0.]
[ 0. 0. 0.]]
[[ 0. 1. 0.]
[ 0. 0. 0.]]
[[ 0. 0. 1.]
[ 0. 0. 0.]]
[[ 0. 0. 0.]
[ 1. 0. 0.]]
[[ 0. 0. 0.]
[ 0. 1. 0.]]
[[ 0. 0. 0.]
[ 0. 0. 1.]]]
and so on.

Another way to implement this is :
>>> def f(size, index):
... return (np.arange(size) == index).astype(float)
...
Which gives a slightly slower execution time :
>>> timeit.timeit('f(size, index)', 'from __main__ import f, size, index', number=1000000)
2.2554846050043125

It may not be the fastest, but the method scipy.signal.unit_impulse generalizes the above concept to numpy arrays of any shape.