We Keep Coding

Python Return closest number above and below given x

I have a problem. In my code I need to do the following:
I have an array like this: [1, 5, 7, 9, 10, 11, 14, 15]
Now I need to determine in the most efficient way what the closest values are for a given x. For example:
array = [1, 5, 7, 9, 10, 11, 14, 15]
above, below = myFunction(array, 8)
Should return 7 and 9
If there is no number higher or lower than the given number, the value that couldn't be determined should be None. If the same number is given as in the array, for example: 7. The values 5 and 9 should be returned.
I know there is a min() and max() function, but I couldn't find anything in my case. The only thing close to what I want, was manual looping through the array, but I was wondering if there was a more efficient way, because this code will be executed arround 500.000 times, so speed is important!
Is there a efficient way to determine those values?

This answer based on #Mark Tolonen hint about bisect
Let's say that you need to append one number on every iteration but initial array is the same.
All elements in array should be unique. Initial array may be not sorted.
import bisect as b
init_array = [5, 9, 10, 11, 14, 15, 1, 7]
init_array.sort()
numbers_to_append = [22, 4, 12, 88, 99, 109, 999, 1000, 1001]
numbers_to_check = [12, 55, 23, 55, 0, 55, 10, 999]
for (_, n) in enumerate(numbers_to_check):
# get index of closer right to below
i = b.bisect_left(init_array, n)
# get above
if i >= (len(init_array)):
above = None
else:
if init_array[i] == n:
try:
above = init_array[i + 1]
except IndexError:
above = None
else:
above = init_array[i]
# get below
below = init_array[i - 1] if i - 1 >= 0 and init_array[i - 1] - n < 0 else None
print('---------')
print('array is', init_array)
print('given x is', n)
print('below is', below)
print('above is', above)
print('appending', numbers_to_append[_])
# check if number trying to append doesn't exists in array
# WARNING: this check may be slow and added only for showing that we need to add only unique numbers
# real check should be based on real conditions and numbers that we suppose to add
i_to_append = b.bisect_left(init_array, numbers_to_append[_])
if numbers_to_append[_] not in init_array[i_to_append:i_to_append+1]:
init_array.insert(i_to_append, numbers_to_append[_])
output:
---------
array is [1, 5, 7, 9, 10, 11, 14, 15]
given x is 12
below is 11
above is 14
appending 22
---------
array is [1, 5, 7, 9, 10, 11, 14, 15, 22]
given x is 55
below is 22
above is None
appending 4
---------
array is [1, 4, 5, 7, 9, 10, 11, 14, 15, 22]
given x is 23
below is 22
above is None
appending 12
---------
array is [1, 4, 5, 7, 9, 10, 11, 12, 14, 15, 22]
given x is 55
below is 22
above is None
appending 88
---------
array is [1, 4, 5, 7, 9, 10, 11, 12, 14, 15, 22, 88]
given x is 0
below is None
above is 1
appending 99
---------
array is [1, 4, 5, 7, 9, 10, 11, 12, 14, 15, 22, 88, 99]
given x is 55
below is 22
above is 88
appending 109
---------
array is [1, 4, 5, 7, 9, 10, 11, 12, 14, 15, 22, 88, 99, 109]
given x is 10
below is 9
above is 11
appending 999
---------
array is [1, 4, 5, 7, 9, 10, 11, 12, 14, 15, 22, 88, 99, 109, 999]
given x is 999
below is 109
above is None
appending 1000

Assuming that your array is not sorted, you will be indeed forced to scan the entire array.
But if it is sorted you can use a binary search approach that will run in O(log(n)), something like this:
def search_nearest_elems(array, elem):
"""
Uses dichotomic search approach.
Runs in O(log(n)), with n = len(array)
"""
i = len(array) // 2
left, right = 0, len(array) - 1
while right - left > 1:
if array[i] == elem:
return elem, elem
elif array[i] > elem:
right, i = i, (left + i) // 2
else:
left, i = i, (right + i) // 2
return array[left], array[right]
array = [1, 5, 7, 9, 10, 11, 14, 15]
assert search_nearest_elems(array, 8) == (7, 9)
assert search_nearest_elems(array, 9) == (9, 9)
assert search_nearest_elems(array, 14.5) == (14, 15)
assert search_nearest_elems(array, 2) == (1, 5)

import bisect
def find_nearest(arr,value):
arr.sort()
idx = bisect.bisect_left(arr, value)
if idx == 0:
return None, arr[idx]
elif idx == len(arr):
return arr[idx - 1], None
elif arr[idx] == value:
return arr[idx], arr[idx]
else:
return arr[idx - 1], arr[idx]
array = [1, 5, 7, 9, 10, 11, 14, 15]
print(find_nearest(array, 0))
print(find_nearest(array, 4))
print(find_nearest(array, 8))
print(find_nearest(array, 10))
print(find_nearest(array, 20))
Output:
(None, 1)
(1, 5)
(7, 9)
(10, 10)
(15, None)
Helper source

Can my code be more simple?

An n-by-n square matrix (table of numbers) is a magic matrix if the sum of its row and the sum of each column are identical. For example, the 4-by-4 matrix below is a magic matrix. The sum of every row and the sum of every column are exactly the same value 34.
16 2 3 13
5 11 10 8
9 7 6 12
4 14 15 1
Write a function that takes a 4-by-4 matrix as an argument and then determine if the matrix is magic or not. The
#matrix should be stored as a two-dimensional list. Test your function with a magic matrix and a none magic matrix.
def magic(matrix = []):
magic_matrix = False
if len(matrix) != 4:
print('Enter a 4 * 4 matrix')
return magic_matrix
row1Sum = sum(matrix[0])
rowSum_ok = True
for row in range(1, 4):
if sum(matrix[row]) != row1Sum:
rowSum_ok = False
break
colSum_ok = True
for col in range(4):
s_col = 0
for row in range(4):
s_col += matrix[row][col]
if s_col != row1Sum:
colSum_ok = False
break
if rowSum_ok and colSum_ok:
magic_matrix = True
return magic_matrix
def mainMagic():
m1 = [[9, 6, 3, 16],
[4, 15, 10, 5],
[14, 1, 8, 11],
[7, 12, 13, 2]]
print('\nThe matrix:')
for i in range(4):
for j in m1[i]:
print(str(j).rjust(3), end =' ')
print()
if magic(m1):
print('is a magic matrix.')
else:
print('is not a magic matrix.')
m2 = [[6, 22, 44, 18],
[1, 11, 10, 13],
[3, 17, 6, 12],
[9, 14, 2, 1]]
print('\nThe matrix:')
for i in range(4):
for j in m2[i]:
print(repr(j).rjust(3), end = ' ')
print()
if magic(m2):
print('is a magic matrix.')
else:
print('is not a magic matrix.')
mainMagic()

With a couple of set comprehensions and a zip() that is fairly straight forward to cleanup like:
Code:
def is_magic(matrix):
sum_rows = {sum(row) for row in matrix}
sum_cols = {sum(col) for col in zip(*matrix)}
return len(sum_cols) == 1 and sum_cols == sum_rows
Test Code:
m1 = [[9, 6, 3, 16],
[4, 15, 10, 5],
[14, 1, 8, 11],
[7, 12, 13, 2]]
m2 = [[6, 22, 44, 18],
[1, 11, 10, 13],
[3, 17, 6, 12],
[9, 14, 2, 1]]
print(is_magic(m1))
print(is_magic(m2))
Results:
True
False

Skipping rows in numpy ndarrays slicing

Suppose I have a numpy array img, with img.shape == (468,832,3). What does img[::2, ::2] do? It reduces the shape to (234,416,3) Can you please explain the logic?

Let's read documentation together (Source).
(Just read the bold part first)
The basic slice syntax is i:j:k where i is the starting index, j is the stopping index, and k is the step (k \neq 0). This selects the m elements (in the corresponding dimension) with index values i, i + k, ..., i + (m - 1) k where m = q + (r\neq0) and q and r are the quotient and remainder obtained by dividing j - i by k: j - i = q k + r, so that i + (m - 1) k < j.
...
Assume n is the number of elements in the dimension being sliced.
Then, if i is not given it defaults to 0 for k > 0 and n - 1 for k < 0
. If j is not given it defaults to n for k > 0 and -n-1 for k < 0 . If
k is not given it defaults to 1. Note that :: is the same as : and
means select all indices along this axis.
Now looking at your part.
[::2, ::2] will be translated to [0:468:2, 0:832:2] because you do not specify the first two or i and j in the documentation. (You only specify k here. Recall the i:j:k notation above.) You select elements on these axes at the step size 2 which means you select every other elements along the axes specified.
Because you did not specify for the 3rd dimension, all will be selected.

It slices every alternate row, and then every alternate column, from an array, returning an array of size (n // 2, n // 2, ...).
Here's an example of slicing with a 2D array -
>>> a = np.arange(16).reshape(4, 4)
>>> a
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
>>> a[::2, ::2]
array([[ 0, 2],
[ 8, 10]])
And, here's another example with a 3D array -
>>> a = np.arange(27).reshape(3, 3, 3)
>>> a
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]],
[[18, 19, 20],
[21, 22, 23],
[24, 25, 26]]])
>>> a[::2, ::2] # same as a[::2, ::2, :]
array([[[ 0, 1, 2],
[ 6, 7, 8]],
[[18, 19, 20],
[24, 25, 26]]])

Well, we have the RGB image as a 3D array of shape:
img.shape=(468,832,3)
Now, what does img[::2, ::2] do?
we're just downsampling the image (i.e. we're shrinking the image size by half by taking only every other pixel from the original image and we do this by using a step size of 2, which means to skip one pixel). This should be clear from the example below.
Let's take a simple grayscale image for easier understanding.
In [13]: arr
Out[13]:
array([[10, 11, 12, 13, 14, 15],
[20, 21, 22, 23, 24, 25],
[30, 31, 32, 33, 34, 35],
[40, 41, 42, 43, 44, 45],
[50, 51, 52, 53, 54, 55],
[60, 61, 62, 63, 64, 65]])
In [14]: arr.shape
Out[14]: (6, 6)
In [15]: arr[::2, ::2]
Out[15]:
array([[10, 12, 14],
[30, 32, 34],
[50, 52, 54]])
In [16]: arr[::2, ::2].shape
Out[16]: (3, 3)
Notice which pixels are in the sliced version. Also, observe how the array shape changes after slicing (i.e. it is reduced by half).
Now, this downsampling happens for all three channels in the image since there's no slicing happening in the third axis. Thus, you will get the shape reduced only for the first two axis in your example.
(468, 832, 3)
. . |
. . |
(234, 416, 3)

Pythonic way to get both diagonals passing through a matrix entry (i,j)

What is the Pythonic way to get a list of diagonal elements in a matrix passing through entry (i,j)?
For e.g., given a matrix like:
[1 2 3 4 5]
[6 7 8 9 10]
[11 12 13 14 15]
[16 17 18 19 20]
[21 22 23 24 25]
and an entry, say, (1,3) (representing element 9) how can I get the elements in the diagonals passing through 9 in a Pythonic way? Basically, [3,9,15] and [5,9,13,17,21] both.

Using np.diagonal with a little offset logic.
import numpy as np
lst = np.array([[1, 2, 3, 4, 5],
[6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25]])
i, j = 1, 3
major = np.diagonal(lst, offset=(j - i))
print(major)
array([ 3, 9, 15])
minor = np.diagonal(np.rot90(lst), offset=-lst.shape[1] + (j + i) + 1)
print(minor)
array([ 5, 9, 13, 17, 21])
The indices i and j are the row and column. By specifying the offset, numpy knows from where to begin selecting elements for the diagonal.
For the major diagonal, You want to start collecting from 3 in the first row. So you need to take the current column index and subtract it by the current row index, to figure out the correct column index at the 0th row. Similarly for the minor diagonal, where the array is flipped (rotated by 90˚) and the process repeats.

As another alternative method, with raveling the array and for matrix with shape (n*n):
array = np.array([[1, 2, 3, 4, 5],
[6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25]])
x, y = 1, 3
a_mod = array.ravel()
size = array.shape[0]
if y >= x:
diag = a_mod[y-x:(x+size-y)*size:size+1]
else:
diag = a_mod[(x-y)*size::size+1]
if x-(size-1-y) >= 0:
reverse_diag = array[:, ::-1].ravel()[(x-(size-1-y))*size::size+1]
else:
reverse_diag = a_mod[x:x*size+1:size-1]
# diag --> [ 3 9 15]
# reverse_diag --> [ 5 9 13 17 21]
The correctness of the resulted arrays must be checked further. This can be developed to handle matrices with other shapes e.g. (n*m).

Python with numpy: How to delete an element from each row of a 2-D array according to a specific index

Say I have a 2-D numpy array A of size 20 x 10.
I also have an array of length 20, del_ind.
I want to delete an element from each row of A according to del_ind, to get a resultant array of size 20 x 9.
How can I do this?
I looked into np.delete with a specified axis = 1, but this only deletes element from the same position for each row.
Thanks for the help

You will probably have to build a new array.
Fortunately you can avoid python loops for this task, using fancy indexing:
h, w = 20, 10
A = np.arange(h*w).reshape(h, w)
del_ind = np.random.randint(0, w, size=h)
mask = np.ones((h,w), dtype=bool)
mask[range(h), del_ind] = False
A_ = A[mask].reshape(h, w-1)
Demo with a smaller dataset:
>>> h, w = 5, 4
>>> %paste
A = np.arange(h*w).reshape(h, w)
del_ind = np.random.randint(0, w, size=h)
mask = np.ones((h,w), dtype=bool)
mask[range(h), del_ind] = False
A_ = A[mask].reshape(h, w-1)
## -- End pasted text --
>>> A
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19]])
>>> del_ind
array([2, 2, 1, 1, 0])
>>> A_
array([[ 0, 1, 3],
[ 4, 5, 7],
[ 8, 10, 11],
[12, 14, 15],
[17, 18, 19]])

Numpy isn't known for inplace edits; it's mainly intended for statically sized matrices. For that reason, I'd recommend doing this by copying the intended elements to a new array.
Assuming that it's sufficient to delete one column from every row:
def remove_indices(arr, indices):
result = np.empty((arr.shape[0], arr.shape[1] - 1))
for i, (delete_index, row) in enumerate(zip(indices, arr)):
result[i] = np.delete(row, delete_index)
return result