We Keep Coding

Image Convolution with callback function in python

I want to loop over the pixels of a binary image in python and set the value of a pixel depending on a surrounding neighborhood of pixels. Similar to convolution but I want create a method that sets the value of the center pixel using a custom function rather than normal convolution that sets the center pixel to the arithmetic mean of the neighborhood.
In essence I would like to create a function that does the following:
def convolve(img, conv_function = lambda subImg: np.mean(subImg)):
newImage = emptyImage
for nxn_window in img:
newImage[center_pixel] = conv_function(nxn_window)
return newImage
At the moment I have a solution but it is very slow:
#B is the structuing array or convolution window/kernel
def convolve(func):
def wrapper(img, B):
#get dimensions of img
length, width = len(img), len(img[0])
#half width and length of dimensions
hw = (int)((len(B) - 1) / 2)
hh = (int)((len(B[0]) - 1) / 2)
#convert to npArray for fast operations
B = np.array(B)
#initialize empty return image
retVal = np.zeros([length, width])
#start loop over the values where the convolution window has a neighborhood
for row in range(hh, length - hh):
for pixel in range(hw, width - hw):
#window as subarray of pixels
window = [arr[pixel-hh:pixel+hh+1]
for arr in img[row-hw:row+hw+1]]
retVal[row][pixel] = func(window, B)
return retVal
return wrapper
with this function as a decorator I then do
# dilation
#convolve
def __add__(img, B):
return np.mean(np.logical_and(img, B)) > 0
# erosion
#convolve
def __sub__(img, B):
return np.mean(np.logical_and(img, B)) == 1
Is there a library that provides this type of function or is there a better way I can loop over the image?

Here's an idea: assign each pixel an array with its neighborhood and then simply apply your custom function to the extended image. It'll be fast BUT will consume more memory ( times more memory; if your B.shape is (3, 3) then you'll need 9 times more memory). Try this:
import numpy as np
def convolve2(func):
def conv(image, kernel):
""" Apply given filter on an image """
k = kernel.shape[0] # which is assumed equal to kernel.shape[1]
width = k//2 # note that width == 1 for k == 3 but also width == 1 for k == 2
a = framed(image, width) # create a frame around an image to compensate for kernel overlap when shifting
b = np.empty(image.shape + kernel.shape) # add two more dimensions for each pixel's neighbourhood
di, dj = image.shape[:2] # will be used as delta for slicing
# add the neighbourhood ('kernel size') to each pixel in preparation for the final step
# in other words: slide the image along the kernel rather than sliding the kernel along the image
for i in range(k):
for j in range(k):
b[..., i, j] = a[i:i+di, j:j+dj]
# apply the desired function
return func(b, kernel)
return conv
def framed(image, width):
a = np.zeros(np.array(image.shape) + [2 * width, 2 * width]) # only add the frame to the first two dimensions
a[width:-width, width:-width] = image # place the image centered inside the frame
return a
I've used a greyscale image 512x512 pixels and a filter 3x3 for testing:
embossing_kernel = np.array([
[-2, -1, 0],
[-1, 1, 1],
[0, 1, 2]
])
#convolve2
def filter2(img, B):
return np.sum(img * B, axis=(2,3))
#convolve2
def __add2__(img, B):
return np.mean(np.logical_and(img, B), axis=(2,3)) > 0
# image_gray is a 2D grayscale image (not color/RGB)
b = filter2(image_gray, embossing_kernel)
To compare with your convolve I've used:
#convolve
def filter(img, B):
return np.sum(img * B)
#convolve
def __add__(img, B):
return np.mean(np.logical_and(img, B)) > 0
b = filter2(image_gray, embossing_kernel)
The time for convolve was 4.3 s, for convolve2 0.05 s on my machine.
In my case the custom function needs to specify the axes over which to operate, i.e., the additional dimensions holding the neighborhood data. Perhaps the axes could be avoided too but I haven't tried.
Note: this works for 2D images (grayscale) (as you asked about binary images) but can be easily extended to 3D (color) images. In your case you could probably get rid of the frame (or fill it with zeros or ones e.g., in case of repeated application of the function).
In case memory was an issue you might want to adapt a fast implementation of convolve I've posted here: https://stackoverflow.com/a/74288118/20188124.

Most efficient way to transfrom a 2d array to a different coordinate system using a function, then interpolate the resultant holes

To start, Im basically trying to go from this:
To this:
Given that each coordinate [x,y] correspond with a given point in the second image after a function is applied to x and y. f(x,y)=coords of the second image for the value of [x,y]. The way Im handling this part as of now is to make a "map" array of x and y and the lookup in that array to find the new point. so mapArrayX[x] will give the new x value and mapArray[y] will give the new Y value. The Issue with this is that I have to iterate over the entire image (256,000 points) and that takes roughly .4 seconds. Is there a better way to do this?
The second issue is after transforming the coordinates I get an image with holes in it that looks like this:
which I make look like the image above without the holes by doing this:
dewarpedImage[dewarpedImage == 0] = np.nan
x = np.arange(0, dewarpedImage.shape[1])
y = np.arange(0, dewarpedImage.shape[0])
# mask invalid values
dewarpedImage = np.ma.masked_invalid(dewarpedImage)
xx, yy = np.meshgrid(x, y)
# get only the valid values
x1 = xx[~dewarpedImage.mask]
y1 = yy[~dewarpedImage.mask]
newarr = dewarpedImage[~dewarpedImage.mask]
startTime = time.time()
dewarpedImage = interpolate.griddata((x1, y1), newarr.ravel(),
(xx, yy),
method='linear')
This takes roughly 3 seconds to perform. Is there a faster way to do this maybe. I ideally need to get this whole process to go from taking 3+seconds to less than 1 second.
Here is my conversion function/how I generate my mapping:
RANGE_BIN_SIZE = .39
def rangeBinToRange(rangeBin):
return rangeBin * RANGE_BIN_SIZE
def azToDegree(azBin):
degree = math.degrees(math.asin((azBin - 127.5) * 0.3771/(0.19812*255)))
return degree
def makeWarpMap():
print("making warp maps")
xMap = np.zeros((1024, 256))
yMap = np.zeros((1024, 256))
for az in range(256):
for rang in range(1024):
azDegree = azToDegree(az)
dist = rangeBinToRange(rang)
x = round(dist * math.sin(math.radians(azDegree)) + 381)
y = round(dist * math.cos(math.radians(azDegree)))
xMap[rang][az] = x
yMap[rang][az] = y
np.save("warpmapX", xMap)
np.save("warpmapY", yMap)
print(azToDegree(0))
if not path.exists("warpmapX.npy") or not path.exists("warpmapY.npy"):
makeWarpMap()
data = np.load(filename)
xMap = np.load("warpmapX.npy")
yMap = np.load("warpmapY.npy")
dewarpedImage = np.zeros((400, 762))
print(data.shape)
for az in range(256):
azslice = data[:, az]
for rang in range(1024):
intensity = azslice[rang]
x = xMap[rang][az]
y = yMap[rang][az]
dewarpedImage[int(y)][int(x)] = intensity

You have holes in your converted image because your conversion does not span the entire polar image. I would recommend to do the reverse conversion. In other words, for each (X,Y) in polar image, find corresponding point (x,y) in cartesian image and get that color. That way you won't need to deal with holes at all and it will give you a full image (it will get rid of 3sec conversion). If you provide your conversion function, I can help you do the reverse conversion.

Compute a least square only on n best values of an array

I have :
print(self.L.T.shape)
print(self.M.T.shape)
(8, 3)
(8, 9082318)
self.N = np.linalg.lstsq(self.L.T, self.M.T, rcond=None)[0].T
which is working fine and return
(9082318, 3)
But
I want to perform a kind of sort on M and compute the solution only on the best 8 - n values of M.
Or ignore values of M below and/or higher than a certain value.
Any pointer on how to do that would be extremely appreciated.
Thank you.
Tried to copy this solution exactly but it return an error
The original working function but basically it's just one line.
M is a stack of 8 grayscale images reshaped.
L is a stack of 8 light direction vectors.
M contains shadows but not always at the same location in the image.
So I need to remove those pixel from the computation but L must retains its dimensions.
def _solve_l2(self):
"""
Lambertian Photometric stereo based on least-squares
Woodham 1980
:return: None
Compute surface normal : numpy array of surface normal (p \times 3)
"""
self.N = np.linalg.lstsq(self.L.T, M.T, rcond=None)[0].T
print(self.N.shape)
self.N = normalize(self.N, axis=1) # normalize to account for diffuse reflectance
Here is the borrowed code from the link for trying to resolve this :
L and M as previously used
Ma = self.M.copy()
thresh = 300
Ma[self.M <= thresh] = 0
Ma[self.M > thresh] = 1
Ma = Ma.T
self.M = self.M.T
self.L = self.L.T
print(self.L.shape)
print(self.M.shape)
print(Ma.shape)
A = self.L
B = self.M
M = Ma #http://alexhwilliams.info/itsneuronalblog/2018/02/26/censored-lstsq/
# else solve via tensor representation
rhs = np.dot(A.T, M * B).T[:,:,None] # n x r x 1 tensor
T = np.matmul(A.T[None,:,:], M.T[:,:,None] * A[None,:,:]) # n x r x r tensor
self.N = np.squeeze(np.linalg.solve(T, rhs)).T # transpose to get r x n
return
numpy.linalg.LinAlgError: Singular matrix

How to Transform Normalization Image Math Equation to Python?

I try to learn how to transform equation to python script.
I choose to start it from FingerPrint Enhancement from Academic resources here.
to start learn i search a fingerprint image to be enhance. I choose this image:
so, i do the first step is converting to gray:
import cv2
import numpy as np
input = 'PATH OF IMAGE'
img = cv2.imread(input)
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
and below is the result:
ok the problem start from here...
please try to understood me, I try to learn how to convert math equation to python script.
not try to looking for another / existing script in Github (for example).
the equation is:
all detail from the academic research. Told that:
Let I(i, j) denote the gray-level value at pixel (i, j), M and
VAR denote the estimated mean and variance of I, respectively, and G(i, j) denote the normalized gray-level value at pixel (i, j).
A gray-level fingerprint image, I is defined as an N x N matrix, where I(i, j) represents the intensity of the pixel at the
i-th row and j-th column. We assume that all the images are
scanned at a resolution of 500 dots per inch (dpi). The mean and variance of a gray-level fingerprint image, I, are defined as
and
respectively
ok, we start to transform the equation:
def mean(gray):
rows, cols = gray.shape
sum = 0
for i in range(0,rows):
for j in range(0, cols):
pix = (gray[i,j].item())
sum += pix
M = sum/N
return M
def var(gray, M):
rows, cols = gray.shape
N = gray.size
sum = 0
for i in range(0,rows):
for j in range(0, cols):
vix = ((img[i,j].item()) - M)**2
sum += vix
VAR = sum/N
return VAR
def normalize(img, M0, VAR0):
M = mean(img)
VAR = var(img, M)
rows,cols = img.shape
normim = np.zeros((rows, cols))
for i in range(0, rows):
for j in range(0, cols):
if (gray[i,j].item()) > M:
G0 = M0 + ((((VAR0)*(((gray[i,j].item())-(M))**2))/(VAR))**(1/2))
normim[i,j] = int(G0)
else:
G1 = M0 - ((((VAR0)*(((gray[i,j].item())-(M))**2))/(VAR))**(1/2))
normim[i,j] = int(G1)
return normim
M0 = 100 #follow the academic research document
VAR0 = 100 #follow the academic research document
normgray = normalize(gray, 100,100)
cv2.imshow('test', normgray)
cv2.waitKey(1)
the result is out of expected:
all is white.
can somebody help me? please your advise.
to remind you, I'm not try to looking for the another script / another example. I try to understood how to transform a math equation to python script. about another script, i already have, even i already map it here.

This is a simple problem of not respecting the data types in between transformations. Specifically, when you load in the image, it is going to be unsigned 8-bit integer so the expected values should be within [0, 255], yet your calculations for the mean and variance will exceed this dynamic range and thus your calculations will overflow. The quickest way to resolve this problem is to convert your image so that it will respect a data type that can handle the precision of the calculations you want, like floating-point. Perform the calculations, and when you're done convert the image back to the expected data type, so unsigned 8-bit integer.
In addition, there are several errors in your code. For one thing, you didn't provide the variable N, which should be the total number of pixels in the image. In addition, your var function accepts gray as the variable yet you are using img to try and access pixel data, so this will also give off an error when you try and run it. Finally, you omitted the packages you're using so I added these in.
I've also downloaded your image locally so I can run the code to verify that it works. I've patched up the end of your code so that the image window that displays the result properly closes after you push a key and I've written the output image to file.
Therefore:
# Added so the code can run
import cv2
import numpy as np
# Added so the code can run
gray = cv2.imread('gnN4Q.png', 0)
gray = gray.astype(np.float) # Change to floating-point
N = gray.shape[0]*gray.shape[1]
def mean(gray):
rows, cols = gray.shape
sum = 0
for i in range(0,rows):
for j in range(0, cols):
pix = (gray[i,j].item())
sum += pix
M = sum/N # Added above
return M
def var(gray, M):
rows, cols = gray.shape
N = gray.size
sum = 0
for i in range(0,rows):
for j in range(0, cols):
vix = ((gray[i,j].item()) - M)**2 # Change
sum += vix
VAR = sum/N
return VAR
def normalize(img, M0, VAR0):
M = mean(img)
VAR = var(img, M)
rows,cols = img.shape
normim = np.zeros((rows, cols))
for i in range(0, rows):
for j in range(0, cols):
if (gray[i,j].item()) > M:
G0 = M0 + ((((VAR0)*(((gray[i,j].item())-(M))**2))/(VAR))**(1/2))
normim[i,j] = int(G0)
else:
G1 = M0 - ((((VAR0)*(((gray[i,j].item())-(M))**2))/(VAR))**(1/2))
normim[i,j] = int(G1)
return normim
M0 = 100 #follow the academic research document
VAR0 = 100 #follow the academic research document
normgray = normalize(gray, 100,100)
normgray = normgray.astype(np.uint8) # Added - convert back to uint8
cv2.imshow('test', normgray)
cv2.waitKey(0)
cv2.destroyAllWindows()
cv2.imwrite('output.png', normgray)
The output image we get is:

I didn't run your code but make sure G0 or G1 doesn't get too big. It could be that your value is above 255, thus the resulting all-white image.

How to efficiently compute the heat map of two Gaussian distribution in Python?

I am trying to produce a heat map where the pixel values are governed by two independent 2D Gaussian distributions. Let them be Kernel1 (muX1, muY1, sigmaX1, sigmaY1) and Kernel2 (muX2, muY2, sigmaX2, sigmaY2) respectively. To be more specific, the length of each kernel is three times its standard deviation. The first Kernel has sigmaX1 = sigmaY1 and the second Kernel has sigmaX2 < sigmaY2. Covariance matrix of both kernels are diagonal (X and Y are independent). Kernel1 is usually completely inside Kernel2.
I tried the following two approaches and the results are both unsatisfactory. Can someone give me some advice?
Approach1:
Iterate over all pixel value pairs (i, j) on the map and compute the value of I(i,j) given by I(i,j) = P(i, j | Kernel1, Kernel2) = 1 - (1 - P(i, j | Kernel1)) * (1 - P(i, j | Kernel2)). Then I got the following result, which is good in terms of smoothness. But it takes 10 seconds to run on my computer, which is too slow.
Codes:
def genDensityBox(self, height, width, muY1, muX1, muY2, muX2, sigmaK1, sigmaY2, sigmaX2):
densityBox = np.zeros((height, width))
for y in range(height):
for x in range(width):
densityBox[y, x] += 1. - (1. - multivariateNormal(y, x, muY1, muX1, sigmaK1, sigmaK1)) * (1. - multivariateNormal(y, x, muY2, muX2, sigmaY2, sigmaX2))
return densityBox
def multivariateNormal(y, x, muY, muX, sigmaY, sigmaX):
return norm.pdf(y, loc=muY, scale=sigmaY) * norm.pdf(x, loc=muX, scale=sigmaX)
Approach2:
Generate two images corresponding to two kernels separately and then blend them together using certain alpha value. Each image is generated by taking the outer product of two one-dimensional Gaussian filter. Then I got the following result, which is very crude. But the advantage of this approach is that it is very fast due to the use of outer product between two vectors.
Since the first one is slow and the second on is crude, I am trying to find a new approach that achieves good smoothness and low time-complexity at the same time. Can someone give me some help?
Thanks!
For the second approach, the 2D Gaussian map can be easily generated as mentioned here:
def gkern(self, sigmaY, sigmaX, yKernelLen, xKernelLen, nsigma=3):
"""Returns a 2D Gaussian kernel array."""
yInterval = (2*nsigma+1.)/(yKernelLen)
yRow = np.linspace(-nsigma-yInterval/2.,nsigma+yInterval/2.,yKernelLen + 1)
kernelY = np.diff(st.norm.cdf(yRow, 0, sigmaY))
xInterval = (2*nsigma+1.)/(xKernelLen)
xRow = np.linspace(-nsigma-xInterval/2.,nsigma+xInterval/2.,xKernelLen + 1)
kernelX = np.diff(st.norm.cdf(xRow, 0, sigmaX))
kernelRaw = np.sqrt(np.outer(kernelY, kernelX))
kernel = kernelRaw / (kernelRaw.sum())
return kernel

Your approach is fine other than that you shouldn't loop over norm.pdf but just push all values at which you want the kernel(s) evaluated, and then reshape the output to the desired shape of the image.
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import multivariate_normal
# create 2 kernels
m1 = (-1,-1)
s1 = np.eye(2)
k1 = multivariate_normal(mean=m1, cov=s1)
m2 = (1,1)
s2 = np.eye(2)
k2 = multivariate_normal(mean=m2, cov=s2)
# create a grid of (x,y) coordinates at which to evaluate the kernels
xlim = (-3, 3)
ylim = (-3, 3)
xres = 100
yres = 100
x = np.linspace(xlim[0], xlim[1], xres)
y = np.linspace(ylim[0], ylim[1], yres)
xx, yy = np.meshgrid(x,y)
# evaluate kernels at grid points
xxyy = np.c_[xx.ravel(), yy.ravel()]
zz = k1.pdf(xxyy) + k2.pdf(xxyy)
# reshape and plot image
img = zz.reshape((xres,yres))
plt.imshow(img); plt.show()
This approach shouldn't take too long:
In [26]: %timeit zz = k1.pdf(xxyy) + k2.pdf(xxyy)
1000 loops, best of 3: 1.16 ms per loop

Based on Paul's answer, I made a function to make a heatmap of gaussians taking as input the centers of the gaussians (it could be helpful to others) :
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import multivariate_normal
def points_to_gaussian_heatmap(centers, height, width, scale):
gaussians = []
for y,x in centers:
s = np.eye(2)*scale
g = multivariate_normal(mean=(x,y), cov=s)
gaussians.append(g)
# create a grid of (x,y) coordinates at which to evaluate the kernels
x = np.arange(0, width)
y = np.arange(0, height)
xx, yy = np.meshgrid(x,y)
xxyy = np.stack([xx.ravel(), yy.ravel()]).T
# evaluate kernels at grid points
zz = sum(g.pdf(xxyy) for g in gaussians)
img = zz.reshape((height,width))
return img
W = 800 # width of heatmap
H = 400 # height of heatmap
SCALE = 64 # increase scale to make larger gaussians
CENTERS = [(100,100),
(100,300),
(300,100)] # center points of the gaussians
img = points_to_gaussian_heatmap(CENTERS, H, W, SCALE)
plt.imshow(img); plt.show()