Numpy point cloud to image - python

I have a point cloud which looks something like this:
The red dots are the points, the black dots are the red dots projected to the xy plane. Although it is not visible in the plot, each point also has a value, which is added to the given pixel when the point is moved to the xy plane. The points are represented by a numpy (np) array like so:
points=np.array([[x0,y0,z0,v0],[x1,y1,z1,v1],...[xn,yn,zn,vn]])
The obvious way to put these points into some image would be through a simple loop, like so:
image=np.zeros(img_size)
for point in points:
#each point = [x,y,z,v]
image[tuple(point[0:2])] += point[3]
Now this works fine, but it is very slow. So I was wondering if there is some way using vectorization, slicing and other clever numpy/python tricks of speeding it up, since in reality I have to this many times for large point clouds. I had come up with something using np.put:
def points_to_image(xs, ys, vs, img_size):
img = np.zeros(img_size)
coords = np.stack((ys, xs))
#put the 2D coordinates into linear array coordinates
abs_coords = np.ravel_multi_index(coords, img_size)
np.put(img, abs_coords, ps)
return img
(in this case the points are pre-split into vectors containing the x, y and v components). While this works fine, it of course only puts the last point to each given pixel, i.e. it is not additive.
Many thanks for your help!

Courtesy of #Paul Panzer:
def points_to_image(xs, ys, ps, img_size):
coords = np.stack((ys, xs))
abs_coords = np.ravel_multi_index(coords, img_size)
img = np.bincount(abs_coords, weights=ps, minlength=img_size[0]*img_size[1])
img = img.reshape(img_size)
On my machine, the loop version takes 0.4432s vs 0.0368s using vectorization. So a neat 12x speedup.
============ EDIT ============
Quick update: using torch...
def points_to_image_torch(xs, ys, ps, sensor_size=(180, 240)):
xt, yt, pt = torch.from_numpy(xs), torch.from_numpy(ys), torch.from_numpy(ps)
img = torch.zeros(sensor_size)
img.index_put_((yt, xt), pt, accumulate=True)
return img
I get all the way down to 0.00749. And that's still all happening on CPU, so 59x speedup vs python loop. I also had a go at running it on GPU, it doesn't seem to make a difference in speed, I guess with accumulate=True it's probably using some sort of atomics on the GPU that slows it all down.

Related

Why does Python Pillow swap x and y axis

I tried this example from GeeksForGeeks where you create a blue picture. And I wanted to try something more out of it, so I wanted to change a single pixel from blue to red. Meanwhile, I did that successfully, I notice that position of the red pixel is reversed. Instead of [1,4] i got [4,1] pixel to turn red, I noticed the same problem of switching x and y with function Image.frombytes. I tried reading the class(PixelAccess) documentation but haven't found anything. I am using Python 3.10.6 and 9.2.0 the latest version of PIL which makes this post not relevant.
The easiest solution is to switch x and y in code, but I can't find a reason why are they swapped.
from PIL import Image
input = Image.new(mode="RGB",size=(10, 10),
color="blue")
input.save("input", format="png")
pixel_map = input.load()
pixel_map[1,4] = (255,0,0)
input.save("path\\example.png", format="png")
edit:
I have added a red thick red line in the middle.
So regarding this code, the line should vertical, not horizontal like it is
// this code goes instead of line: pixel_map[1,4] = (255,0,0)
i = 0
j = 0
for i in range (10):
for j in range (10):
if j == 4 or j == 5:
pixel_map[i,j] = (255,0,0)
pixel_map[i,j] = (255,0,0)
Summary of my comments:
It is pretty standard to access digital images via [x, y] coordinates, as opposed to [y, x] or [y][x]. In mathematics, arrays are usually indexed by row and then column, but with images, the width component is conventionally first - hence why we say resolutions like "1920x1080", which is the X and then Y value. And, just like when accessing a cartesian coordinate plane in mathematics, X refers to the horizontal component, and is first, while Y is second and refers to the vertical component. So, images tend to be treated more like a coordinate system than a matrix, at least when a layer of abstraction is added like PIL is doing. Hence, I think this can be confusing for those who are used to how 2D arrays are typically indexed.
Here is a post which does a great job explaining why there's different coordinate systems. It's far more detailed and well-researched than what I'm capable of coming up with right now.
Like I said I think there's just some understandable confusion when it comes to transitioning from thinking of the first index as the row, and the second as the column, when with digital images it's the other way around usually. In the end, the order is just determined by the tool you are using. Some tools use pixel coordinates (x, y) while others use matrix coordinates (y, x). Matrix coordinates are indeed how images are usually internally stored, but I think the (x, y) order is a layer of "convenience" that is added sometimes. This thread has some related discussion: why should I use (y,x) instead of (x,y) to access a pixel in opencv?
If you look at the Pillow source code that actually gets the pixel when accessing the loaded image data, you'll see that it actually indexes self.pixels[y][x]. So, internally, it's being stored how you expect; it's just that the pixel access object deliberately chose the index to be in (x, y) order. The PixelAccess object can be indexed via [x, y] but internally it stores the pixels as [y][x], so internally it swaps the order. You do not have access to this internal representation as far as I know. That's just an internal implementation detail though.

Generating a scatterplot from a greyscale intensity map

Using matplotlib(or if there exists anything else), i want to populate a scatterplot image by using a grey scale image as its distribution. I have found many resource to create heat maps from images but not the other way around.
The input image will be like this one.
I think I understand what you're going for, but I'm not certain. I also don't really understand what this would be used for so I'm extra uncertain about this answer, but here goes:
So by loading the image we can evaluate each pixel position and its intensity. We can use that intensity as a "fitness" value and probabilistically add it to our plot so that we can get some of that "density" of points that you want to see. I picked a really simple equation as a decider (I just cubed the value), but feel free to replace that with whatever you want.
import cv2
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import random
# select func
def selection(value):
return value**3 >= random.randint(0, 255**3);
# populate the sample
def populate(img):
# get res
h, w = img.shape;
# go through and populate
sx = [];
sy = [];
for y in range(0, h):
for x in range(0, w):
val = img[y, x];
# use intensity to decide if it gets in
# replace with what you want this function to look like
if selection(val):
sx.append(x);
sy.append(h - y); # opencv is top-left origin
return sx, sy;
# I'm using opencv to pull the image into code, use whatever you like
# matplotlib can also do something similar, but I'm not familiar with its format
img = cv2.imread("circ.png");
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY);
# lets take a sample
sx, sy = populate(img);
# find the bigger square size
h, w = img.shape;
side = None;
if h > w:
side = h;
else:
side = w;
# make a square graph
fig, ax = plt.subplots();
ax.scatter(sx, sy, s = 4);
ax.set_xlim((0, side));
ax.set_ylim((0, side));
x0,x1 = ax.get_xlim();
y0,y1 = ax.get_ylim();
ax.set_aspect(abs(x1-x0)/abs(y1-y0));
fig.savefig("out.png", dpi=600);
plt.show();
Feel free to replace opencv with whatever image library you're comfortable with. I'm pretty sure matplotlib can open images as well, but openCV is what I'm most familiar with so I used that.
As far as I can tell, you're trying to generate random coordinates that follow a distribution described by a grayscale image: the brighter each point, the more likely that point's coordinates will be generated. Your problem can thus be solved by a rejection sampler, as follows.
Assume you know the width and height of the image in pixels, call them w and h.
Generate two random numbers: one in the interval [0, w), and [0, h). These are the x and y coordinates, respectively.
Get the pixel at the given coordinates x and y in the image. This can be done using interpolation, but describing interpolation techniques is beyond the scope of this answer. For this reason, we will use only the nearest pixel ("nearest neighbor") in the image: take the pixel at coordinate floor(x) and floor(y) (and step 1 devolves to generating random integers). Convert the pixel somehow to a number p in the interval [0, 1]; in this answer we will assume black is 0 and white is 1, to simplify matters.
With probability p, return the point (x, y). Otherwise, go to step 1.
Roughly speaking, the time complexity of this algorithm depends on the numbers of "bright points" the input image has, compared to the number of "dark points". In general, the "brighter" the image, the higher the acceptance rate (and the faster the algorithm runs).

reconstruction: why undistort image AND normalize coordinates?

From the Tutorial: https://programtalk.com/vs2/?source=python/8176/opencv-python-blueprints/chapter4/scene3D.py
I don't understand why they first undistort the images
# undistort the images
self.img1 = cv2.undistort(self.img1, self.K, self.d)
self.img2 = cv2.undistort(self.img2, self.K, self.d)
and: Compute the Essential Matrix
def _find_fundamental_matrix(self):
self.F, self.Fmask = cv2.findFundamentalMat(self.match_pts1,
self.match_pts2,
cv2.FM_RANSAC, 0.1,0.99)
def _find_essential_matrix(self):
self.E = self.K.T.dot(self.F).dot(self.K)
and also Normalize the coordinates:
first_inliers = []
second_inliers = []
for i in range(len(self.Fmask)):
if self.Fmask[i]:
# normalize and homogenize the image coordinates
first_inliers.append(self.K_inv.dot([self.match_pts1[i][0],
self.match_pts1[i][1], 1.0]))
second_inliers.append(self.K_inv.dot([self.match_pts2[i][0],
self.match_pts2[i][1], 1.0]))
Shouldn't it be either or? Or do I have some wrong understanding here?
Can please somone help me on that?
The first step, undistort, does a number of things to reverse the typical warping caused by small camera lenses. See the Wikipedia article on distortion (optics) for more background.
The last step, homogenizing the coordinates, is a completely different thing. The Wikipedia article on homogenous coordinates explains it, but the basic idea is that you add in an extra fake axis that lets you do all affine and projective transformations with chained simple matrix multiplication and then just project back to 3D at the end. Normalizing is just a step you do to make that math easier—basically, you want your extra coordinate to start off as 1.0 (multiply by the inverse of the projective norm).
The requirement for normalization is explained at page-107 of Multi-View Geometry (Hartley and Zisserman). The normalization is required in addition to the un-distortion.
If are using raw pixel values in homogeneous coordinates, the Z-coordinate which is 1 will be small compared to the X and Y co-coordinates. Eg: (X=320, Y=220, Z=1).
But if the homogenized coordinates are the image pixel positions normalized to a standard range, ie -1.0 to 1.0, then we are talking about coordinate values all of whom are kind of in the same range , Eg: (0.75, -0.89, 1.0).
If the image coordinates are of dramatically different ranges(as in the unnormalized case), then the DLT matrix produced will have a bad condition number, and consequently small variations in input image pixel positions, could produce wide variations in the result.
Please see page 107 for a very good explanation.

Apply rotation to HEALPix map in healpy

I have a HEALPix map that I have read in using healpy, however it is in galactic coordinates and I need it in celestial/equatorial coordinates. Does anybody know of a simple way to convert the map?
I have tried using healpy.Rotator to convert from (l,b) to (phi,theta) and then using healpy.ang2pix to reorder the pixels, but the map still looks strange.
It would be great if there was a function similar to Rotator that you could call like: map = AnotherRotator(map,coord=['G','C']). Anybody know of any such function??
Thanks,
Alex
I realise that this was asked ages ago, but I was having the same problem myself this week and found your post. I've found a couple of potential solutions, so I'll share incase someone else comes upon this and finds it useful.
Solution 1: This sort of depends on the format your data is coming in. Mine came in a (theta, phi) grid.
import numpy as np
import healpy as H
map = <your original map>
nside = <your map resolution, mine=256>
npix = H.nside2npix(nside)
pix = N.arange(npix)
t,p = H.pix2ang(nside,pix) #theta, phi
r = H.Rotator(deg=True, rot=[<THETA ROTATION>, <PHI ROTATION>])
map_rot = np.zeros(npix)
for i in pix:
trot, prot = r(t[i],p[i])
tpix = int(trot*180./np.pi) #my data came in a theta, phi grid -- this finds its location there
ppix = int(prot*180./np.pi)
map_rot[i] = map[ppix,tpix] #this being the rright way round may need double-checking
Solution 2: Haven't quite finished testing this, but just came across it AFTER doing the annoying work above...
map_rot = H.mollview(map,deg=True,rot=[<THETA>,<PHI>], return_projected_map=True)
which gives a 2D numpy array. I'm interested to know how to convert this back into a healpix map...
I found another potential solution, after searching off and on for a few months. I haven't tested it very much yet, so please be careful!
Saul's Solution 2, above, is the key (great suggestion!)
Basically, you combine the functionality of healpy.mollview (gnomview, cartview, and orthview work as well) with that of the reproject_to_healpix function in the reproject package (http://reproject.readthedocs.org/en/stable/).
The resulting map is suitable for my angular scales, but I can't say how accurate the transformation is compared to other methods.
-----Basic Outline----------
Step 1: Read-in the map and make the rectangular array via cartview. As Saul indicated above, this is also one way to do the rotation. If you're just doing a standard rotation/coordinate transformation, then all you need is the coord keyword. From Celestial to Galactic coordinates, set coord = ['C','G']
map_Gal = hp.cartview(map_Cel, coord=['C','G'], return_projected_map=True, xsize=desired_xsize, norm='hist',nest=False)
Step 2: Write a template all-sky FITS header (as in the example below). I wrote mine to have the same average pixel-scale as my desired HEALPix map.
Step 3: Use reproject.transform_to_healpix
reproject includes a function for mapping a "normal" array (or FITS file) into the HEALPix projection. Combine that with the ability to return the array created by healpy.mollview/cartview/orthview/gnomview, and you can rotate a HEALPix map of one coordinate system (Celestial) into another coordinate system (Galactic).
map_Gal_HP, footprint_Gal_HP = rp.reproject_to_healpix((map_Gal, target_header), coord_system_out= 'GALACTIC', nside=nside, nested=False)
It comes down, essentially, to those two commands. However you'll have to make a template header, giving the pixel scale and size corresponding to the intermediary all-sky map you want to make.
-----Full Working Example (iPython notebook format + FITS sample data)------
https://github.com/aaroncnb/healpix_coordtrans_example/tree/master
The code there should run very quickly, but that's because the maps are heavily degraded. I did the same for my NSIDE 1024 and 2048 maps, and it took
about an hour.
------Before and After Images------
This function seems to do the trick (reasonably slow, but should be better than the for loop):
def rotate_map(hmap, rot_theta, rot_phi):
"""
Take hmap (a healpix map array) and return another healpix map array
which is ordered such that it has been rotated in (theta, phi) by the
amounts given.
"""
nside = hp.npix2nside(len(hmap))
# Get theta, phi for non-rotated map
t,p = hp.pix2ang(nside, np.arange(hp.nside2npix(nside))) #theta, phi
# Define a rotator
r = hp.Rotator(deg=False, rot=[rot_phi,rot_theta])
# Get theta, phi under rotated co-ordinates
trot, prot = r(t,p)
# Interpolate map onto these co-ordinates
rot_map = hp.get_interp_val(hmap, trot, prot)
return rot_map
Using this on data from PyGSM gives the following:
hp.mollview(np.log(rotate_map(gsm.generated_map_data, 0,0)))
Upon rotation of phi:
hp.mollview(np.log(rotate_map(gsm.generated_map_data, 0,np.pi)))
Or rotating theta:
hp.mollview(np.log(rotate_map(gsm.generated_map_data, np.pi/4,0)))

calculate turning points / pivot points in trajectory (path)

I'm trying to come up with an algorithm that will determine turning points in a trajectory of x/y coordinates. The following figures illustrates what I mean: green indicates the starting point and red the final point of the trajectory (the entire trajectory consists of ~ 1500 points):
In the following figure, I added by hand the possible (global) turning points that an algorithm could return:
Obviously, the true turning point is always debatable and will depend on the angle that one specifies that has to lie between points. Furthermore a turning point can be defined on a global scale (what I tried to do with the black circles), but could also be defined on a high-resolution local scale. I'm interested in the global (overall) direction changes, but I'd love to see a discussion on the different approaches that one would use to tease apart global vs local solutions.
What I've tried so far:
calculate distance between subsequent points
calculate angle between subsequent points
look how distance / angle changes between subsequent points
Unfortunately this doesn't give me any robust results. I probably have too calculate the curvature along multiple points, but that's just an idea.
I'd really appreciate any algorithms / ideas that might help me here. The code can be in any programming language, matlab or python are preferred.
EDIT here's the raw data (in case somebody want's to play with it):
mat file
text file (x coordinate first, y coordinate in second line)
You could use the Ramer-Douglas-Peucker (RDP) algorithm to simplify the path. Then you could compute the change in directions along each segment of the simplified path. The points corresponding to the greatest change in direction could be called the turning points:
A Python implementation of the RDP algorithm can be found on github.
import matplotlib.pyplot as plt
import numpy as np
import os
import rdp
def angle(dir):
"""
Returns the angles between vectors.
Parameters:
dir is a 2D-array of shape (N,M) representing N vectors in M-dimensional space.
The return value is a 1D-array of values of shape (N-1,), with each value
between 0 and pi.
0 implies the vectors point in the same direction
pi/2 implies the vectors are orthogonal
pi implies the vectors point in opposite directions
"""
dir2 = dir[1:]
dir1 = dir[:-1]
return np.arccos((dir1*dir2).sum(axis=1)/(
np.sqrt((dir1**2).sum(axis=1)*(dir2**2).sum(axis=1))))
tolerance = 70
min_angle = np.pi*0.22
filename = os.path.expanduser('~/tmp/bla.data')
points = np.genfromtxt(filename).T
print(len(points))
x, y = points.T
# Use the Ramer-Douglas-Peucker algorithm to simplify the path
# http://en.wikipedia.org/wiki/Ramer-Douglas-Peucker_algorithm
# Python implementation: https://github.com/sebleier/RDP/
simplified = np.array(rdp.rdp(points.tolist(), tolerance))
print(len(simplified))
sx, sy = simplified.T
# compute the direction vectors on the simplified curve
directions = np.diff(simplified, axis=0)
theta = angle(directions)
# Select the index of the points with the greatest theta
# Large theta is associated with greatest change in direction.
idx = np.where(theta>min_angle)[0]+1
fig = plt.figure()
ax =fig.add_subplot(111)
ax.plot(x, y, 'b-', label='original path')
ax.plot(sx, sy, 'g--', label='simplified path')
ax.plot(sx[idx], sy[idx], 'ro', markersize = 10, label='turning points')
ax.invert_yaxis()
plt.legend(loc='best')
plt.show()
Two parameters were used above:
The RDP algorithm takes one parameter, the tolerance, which
represents the maximum distance the simplified path
can stray from the original path. The larger the tolerance, the cruder the simplified path.
The other parameter is the min_angle which defines what is considered a turning point. (I'm taking a turning point to be any point on the original path, whose angle between the entering and exiting vectors on the simplified path is greater than min_angle).
I will be giving numpy/scipy code below, as I have almost no Matlab experience.
If your curve is smooth enough, you could identify your turning points as those of highest curvature. Taking the point index number as the curve parameter, and a central differences scheme, you can compute the curvature with the following code
import numpy as np
import matplotlib.pyplot as plt
import scipy.ndimage
def first_derivative(x) :
return x[2:] - x[0:-2]
def second_derivative(x) :
return x[2:] - 2 * x[1:-1] + x[:-2]
def curvature(x, y) :
x_1 = first_derivative(x)
x_2 = second_derivative(x)
y_1 = first_derivative(y)
y_2 = second_derivative(y)
return np.abs(x_1 * y_2 - y_1 * x_2) / np.sqrt((x_1**2 + y_1**2)**3)
You will probably want to smooth your curve out first, then calculate the curvature, then identify the highest curvature points. The following function does just that:
def plot_turning_points(x, y, turning_points=10, smoothing_radius=3,
cluster_radius=10) :
if smoothing_radius :
weights = np.ones(2 * smoothing_radius + 1)
new_x = scipy.ndimage.convolve1d(x, weights, mode='constant', cval=0.0)
new_x = new_x[smoothing_radius:-smoothing_radius] / np.sum(weights)
new_y = scipy.ndimage.convolve1d(y, weights, mode='constant', cval=0.0)
new_y = new_y[smoothing_radius:-smoothing_radius] / np.sum(weights)
else :
new_x, new_y = x, y
k = curvature(new_x, new_y)
turn_point_idx = np.argsort(k)[::-1]
t_points = []
while len(t_points) < turning_points and len(turn_point_idx) > 0:
t_points += [turn_point_idx[0]]
idx = np.abs(turn_point_idx - turn_point_idx[0]) > cluster_radius
turn_point_idx = turn_point_idx[idx]
t_points = np.array(t_points)
t_points += smoothing_radius + 1
plt.plot(x,y, 'k-')
plt.plot(new_x, new_y, 'r-')
plt.plot(x[t_points], y[t_points], 'o')
plt.show()
Some explaining is in order:
turning_points is the number of points you want to identify
smoothing_radius is the radius of a smoothing convolution to be applied to your data before computing the curvature
cluster_radius is the distance from a point of high curvature selected as a turning point where no other point should be considered as a candidate.
You may have to play around with the parameters a little, but I got something like this:
>>> x, y = np.genfromtxt('bla.data')
>>> plot_turning_points(x, y, turning_points=20, smoothing_radius=15,
... cluster_radius=75)
Probably not good enough for a fully automated detection, but it's pretty close to what you wanted.
A very interesting question. Here is my solution, that allows for variable resolution. Although, fine-tuning it may not be simple, as it's mostly intended to narrow down
Every k points, calculate the convex hull and store it as a set. Go through the at most k points and remove any points that are not in the convex hull, in such a way that the points don't lose their original order.
The purpose here is that the convex hull will act as a filter, removing all of "unimportant points" leaving only the extreme points. Of course, if the k-value is too high, you'll end up with something too close to the actual convex hull, instead of what you actually want.
This should start with a small k, at least 4, then increase it until you get what you seek. You should also probably only include the middle point for every 3 points where the angle is below a certain amount, d. This would ensure that all of the turns are at least d degrees (not implemented in code below). However, this should probably be done incrementally to avoid loss of information, same as increasing the k-value. Another possible improvement would be to actually re-run with points that were removed, and and only remove points that were not in both convex hulls, though this requires a higher minimum k-value of at least 8.
The following code seems to work fairly well, but could still use improvements for efficiency and noise removal. It's also rather inelegant in determining when it should stop, thus the code really only works (as it stands) from around k=4 to k=14.
def convex_filter(points,k):
new_points = []
for pts in (points[i:i + k] for i in xrange(0, len(points), k)):
hull = set(convex_hull(pts))
for point in pts:
if point in hull:
new_points.append(point)
return new_points
# How the points are obtained is a minor point, but they need to be in the right order.
x_coords = [float(x) for x in x.split()]
y_coords = [float(y) for y in y.split()]
points = zip(x_coords,y_coords)
k = 10
prev_length = 0
new_points = points
# Filter using the convex hull until no more points are removed
while len(new_points) != prev_length:
prev_length = len(new_points)
new_points = convex_filter(new_points,k)
Here is a screen shot of the above code with k=14. The 61 red dots are the ones that remain after the filter.
The approach you took sounds promising but your data is heavily oversampled. You could filter the x and y coordinates first, for example with a wide Gaussian and then downsample.
In MATLAB, you could use x = conv(x, normpdf(-10 : 10, 0, 5)) and then x = x(1 : 5 : end). You will have to tweak those numbers depending on the intrinsic persistence of the objects you are tracking and the average distance between points.
Then, you will be able to detect changes in direction very reliably, using the same approach you tried before, based on the scalar product, I imagine.
Another idea is to examine the left and the right surroundings at every point. This may be done by creating a linear regression of N points before and after each point. If the intersecting angle between the points is below some threshold, then you have an corner.
This may be done efficiently by keeping a queue of the points currently in the linear regression and replacing old points with new points, similar to a running average.
You finally have to merge adjacent corners to a single corner. E.g. choosing the point with the strongest corner property.

Categories