We Keep Coding

covert rgb png and depth txt to point cloud

I have a series of rgb files in png format, as well as the corresponding depth file in txt format, which can be loaded with np.loadtxt. How could I merge these two files to point cloud using open3d?
I followed the procedure as obtain point cloud from depth numpy array using open3d - python, but the result is not readable for human.
The examples is listed here:
the source png:
the pcd result:
You can get the source file from this link ![google drive] to reproduce my result.
By the way, the depth and rgb are not registerd.
Thanks.

I had to play a bit with the settings and data and used mainly the answer of your SO link.
import cv2
import numpy as np
import open3d as o3d
color = o3d.io.read_image("a542c.png")
depth = np.loadtxt("a542d.txt")
vertices = []
for x in range(depth.shape[0]):
for y in range(depth.shape[1]):
vertices.append((float(x), float(y), depth[x][y]))
pcd = o3d.geometry.PointCloud()
point_cloud = np.asarray(np.array(vertices))
pcd.points = o3d.utility.Vector3dVector(point_cloud)
pcd.estimate_normals()
pcd = pcd.normalize_normals()
o3d.visualization.draw_geometries([pcd])
However, if you keep the code as provided, the whole scene looks very weird and unfamiliar. That is because your depth file contains data between 0 and almost 2.5 m.
I introduced a cut-off at 500 or 1000 mm plus removed all 0s as suggested in the other answer. Additionally I flipped the x-axis (float(-x) instead of float(x)) to resemble your photo.
# ...
vertices = []
for x in range(depth.shape[0]):
for y in range(depth.shape[1]):
if 0< depth[x][y]<500:
vertices.append((float(-x), float(y), depth[x][y]))
For a good perspective I had to rotate the images manually. Probably open3d provides methods to do it automatically (I quickly tried pcd.transform() from your SO link above, it can help you if needed).
Results
500 mm cut-off: and 1000 mm cut-off: .

I used laspy instead of open3d because wanted to give some colors to your image:
import imageio
import numpy as np
# first reading the image for RGB values
image = imageio.imread(".../a542c.png")
loading the depth file
depth = np.loadtxt("/home/shaig93/Documents/internship_FWF/a542d.txt")
# creating fake x, y coordinates with meshgrid
xv, yv = np.meshgrid(np.arange(400), np.arange(640), indexing='ij')
# save_las is a function based on laspy that was provided to me by my supervisor
save_las("fn.laz", image[:400, :, 0].flatten(), np.c_[yv.flatten(), xv.flatten(), depth.flatten()], cmap = plt.cm.magma_r)
and the result is this. As you can see objects are visible from front.
However from side they are not easy to distinguish.
This means to me to think that your depth file is not that good.
Another idea would be also getting rid off 0 values from your depth file so that you can get point cloud without a wall kind of structure in the front. But still does not solve depth issue of course.
ps. I know this is not a proper answer but I hope it was helpful on identifying the problem.

Why does Nibabel change the size of my file?

I am working with images in the format nii.gz. Therefore, I am using nibabel in order to open them. The problem is that when I open the images, transform them to numpy arrays and convert them back to Nifti format, the output size is changed. The sequence is:
nifti_image = nib.load('/my_path_to_image/image.nii.gz')
np_img = ct_images.get_fdata()
nifti_final = nib.Nifti1Image(data, affine=np.eye(4)) # Convert them to nifti
nib.save(nifti_final , 'image.nii.gz')
The initial file is ~45 MB, after running the code above, the image is ~65 MB. I know that the original images are 16-bit encoded. My initial theory was that when transforming to numpy array, they were encoded as 64-bit which is indeed the case. So I tried the following:
nifti_image = nib.load('/my_path_to_image/image.nii.gz')
np_img = ct_images.get_fdata()
np_img = np_img.astype(numpy.float16, copy=False)
nifti_final = nib.Nifti1Image(data, affine=np.eye(4)) # Convert them to nifti
nib.save(nifti_final , 'image.nii.gz')
However, the ouput is still the same size ~65MB. Any ideas why this is happening?

You should add the original nifti affine and header information to the output nifti. E.g., in your case:
nifti_final = nib.Nifti1Image(data, nifti_image.affine, nifti_image.header)

Reading MONO 16-bit images using PyCapture2

I am using the CMLN-13S2M-CS camera from PointGrey. This camera has a MONO 16-bit pixel format.Using the PyCapture2 wrapper from PointGrey I am unable to retrieve the image the camera is recording.
I have the following code:
import sys
import numpy
import PyCapture2
## Connect camera
bus = PyCapture2.BusManager()
c = PyCapture2.Camera()
c.connect(bus.getCameraFromIndex(0))
## Configure camera format7 settings
fmt7imgSet = PyCapture2.Format7ImageSettings(0, 0, 0, 1296, 964, PyCapture2.PIXEL_FORMAT.MONO16)
fmt7pktInf, isValid = c.validateFormat7Settings(fmt7imgSet)
c.setFormat7ConfigurationPacket(fmt7pktInf.recommendedBytesPerPacket, fmt7imgSet)
## Start capture and retrieve buffer
c.startCapture()
im = c.retrieveBuffer()
print im.getData().shape
print numpy.max(im.getData())
The following is returned by the print statements: (2498688,) and 240. The shape is exactly 2 x (964 x 1296). How should I reshape this? Also, the maximum value when saturated is 255. This is odd as this corresponds to MONO 8 Pixel format. What am I doing wrong?

Here's a quick demo that shows how to convert a 1D array of uint8 to a 2D array of uint16. The key function we need here is view.
import numpy as np
# Make 24 bytes of fake data
raw = np.arange(24, dtype=np.uint8)
#Convert
out = raw.view(np.uint16).reshape(3, 4)
print(out)
print(out.dtype)
output
[[ 256 770 1284 1798]
[2312 2826 3340 3854]
[4368 4882 5396 5910]]
uint16
Thanks to Andras Deak for his assistance!
If the resulting image doesn't look correct, you may need to swap the byte ordering of the 16 bit integers. You can read about byte ordering in Numpy here.
And if that still doesn't look correct, then the data may be organized as two planes, with one plane for the low-order bits of a pixel and the other plane for the high-order bits. That's also easy to deal with, but hopefully it won't come to that. ;)

Include WCS coordinates in a FITS image

I have a 2D numpy array 'ZEA_N_sky' initially created as
n=128
ZEA_N_sky=np.zeros((n,n))
and later some values were assigned to every pixel of this array. I can plot this array as FITS image (that can be opened by ds9) using the following code, however I wish to see the WCS coordinates as well. That is, when I roll my mouse cursor over the image, I should be able to see the gal-long, gal-lat; Right-Asc, Dec , etc. How can I achieve that ? Do i need to do it manually or there is some header() trick ?
The documentation given using WCS package and use all_pix2world() does not seem to work (rather I dont seem to understand it). If somebody can help me please with the code? Thanks !
from astropy.io import fits
import numpy as np
from astropy import wcs
from astropy.table import Table
out_file_name = 'FITS_image.fits'
hdr = fits.Header()
hdr['Projection'] = "ZEA"
hdr['nd_size'] = str(nd_size)
hdr['SCALE']="nd_size/2"
fits.writeto(out_file_name, ZEA_N_sky, hdr,clobber=True)
I have made some progress. This is the bit of code I am using right now using which I do obtain the WCS parameters, but the value of longitude that I am getting on my FITS image is wrong ! I think I am making mistake in the ZEA projection parameters values.
NSGP=1 #Parameter for North side projection, -1 for south side
w = wcs.WCS(naxis=2)
w.wcs.crpix = [SCALE,SCALE] # SCALE is half the value of my pixel range in the image. n/2=128/2=64
w.wcs.cdelt = np.array([-NSGP*90.0/float(SCALE) * 0.90032, NSGP*90.0/float(SCALE) * 0.90032]) # increments in degrees per pixel
w.wcs.crval = [NSGP*90.0, NSGP*90.0] #RA and dec values in hours and degrees
w.wcs.ctype = ["GLON-ZEA", "GLAT-ZEA"]
#w.wcs.set_pv([(180, NSGP, float(SCALE))])
out_file_name = 'N_Mateu_ZEA_stream_mask.fits'
# Now, write out the WCS object as a FITS header
header = w.to_header()
hdu = fits.PrimaryHDU(ZEA_N_sky,header=header)
hdu.writeto(out_file_name, clobber=True)

Try making a WCS object with the correct WCS parameters for your image, and then call header = wcs.to_header, and store the image data and header in the FITS file.
See example here.
Usually you don't create WCS or headers yourself, it's something you read from FITS files. In that case you do wcs = WCS(header) to create a WCS object from the FITS header.
See example here.

delete pixel from the raster image with specific range value

update :
any idea how to delete pixel from specific range value raster image with
using numpy/scipy or gdal?
or how to can create new raster with some class using raster calculation expressions(better)
for example i have a raster image with the
5 class :
1. 0-100
2. 100-200
3. 200-300
4. 300-500
5. 500-1000
and i want to delete class 1 range value
or maybe class 1,2,4,5
i begin with this script :
import numpy as np
from osgeo import gdal
ds = gdal.Open("raster3.tif")
myarray = np.array(ds.GetRasterBand(1).ReadAsArray())
#print myarray.shape
#print myarray.size
#print myarray
new=np.delete(myarray[::2], 1)
but i cant to complete
the image
White in class 5 and black class 1

Rasters are 2-D arrays of values, with each value being stored in a pixel (which stands for picture element). Each pixel must contain some information. It is not possible to delete or remove pixels from the array because rasters are usually encoded as a simple 1-dimensional string of bits. Metadata commonly helps explain where line breaks are and the length of the bitstring, so that the 1-D bitstring can be understood as a 2-D array. If you "remove" a pixel, then you break the raster. The 2-D grid is no longer valid.
Of course, there are many instances where you do want to effectively discard or clean the raster of data. Such an example might be to remove pixels that cover land from a raster of sea-surface temperatures. To accomplish this goal, many geospatial raster formats hold metadata describing what are called NoData values. Pixels containing a NoData value are interpreted as not existing. Recall that in a raster, each pixel must contain some information. The NoData paradigm allows the structure and format of rasters to be met, while also giving a way to mask pixels from being displayed or analyzed. There is still data (bits, 1s and 0s) at the masked pixels, but it only serves to identify the pixel as invalid.
With this in mind, here is an example using gdal which will mask values in the range of 0-100 so they are NoData, and "do not exist". The NoData value will be specified as 0.
from osgeo import gdal
# open dataset to read, and get a numpy array
ds = gdal.Open("raster3.tif", 'r')
myarray = ds.GetRasterBand(1).ReadAsArray()
# modify numpy array to mask values
myarray[myarray <= 100] = 0
# open output dataset, which is a copy of original
driver = gdal.GetDriverByName('GTiff')
ds_out = driver.CreateCopy("raster3_with_nodata.tif", ds)
# write the modified array to the raster
ds_out.GetRasterBand(1).WriteArray(myarray)
# set the NoData metadata flag
ds_out.GetRasterBand(1).SetNoDataValue(0)
# clear the buffer, and ensure file is written
ds_out.FlushCache()

I want to contribute with an example for landsat data.
In this qick guide, you will be able to exclude Landsat cloud pixels.
Landsat offers the Quality Assessment Band (BQA), which includes int32 values (classes) regarding natural features such as Clouds, Rocks, Ice, Water, Cloud Shadow etc.
We will use the BQA to clip the cloud pixels in the other bands.
# Import Packages
import rasterio as rio
import earthpy.plot as ep
from matplotlib import pyplot
import rioxarray as rxr
from numpy import ma
# Open the Landsat Band 3
Landsat_Image = rxr.open_rasterio(r"C:\...\LC08_L1TP_223075_20210311_20210317_01_T1_B3.tif")
# Open the Quality Assessment Band
BQA = rxr.open_rasterio(r"C:\...\LC08_L1TP_223075_20210311_20210317_01_T1_BQA.tif").squeeze()
# Create a list with the QA values that represent cloud, cloud_shadow, etc.
Cloud_Values = [6816, 6848, 6896, 7072]
# Mask the data using the pixel QA layer
landsat_masked = Landsat_Image.where(~BQA.isin(Cloud_Values))
landsat_masked
# Plot the masked data
landsat_masked_plot = ma.masked_array(landsat_masked.values,landsat_masked.isnull())
# Plot
ep.plot_rgb(landsat_masked_plot, rgb=[2, 1, 0], title = "Masked Data")
plt.show()
###############################################################################
# Export the masked Landsat Scenes to Directory "Masked_Bands_QA"
out_img = landsat_masked
out_img.shape
out_transform = landsat_masked.rio.transform()
# Get a Band of the same Scene for reference
rastDat = rio.open(r"C:\Dados_Espaciais\NDVI_Usinas\Adeco\Indices\Imagens\LC08_L1TP_223075_20210311_20210317_01_T1\LC08_L1TP_223075_20210311_20210317_01_T1_B3.tif")
#copying metadata from original raster
out_meta = rastDat.meta.copy()
#amending original metadata
out_meta.update({'nodata': 0,
'height' : out_img.shape[1],
'width' : out_img.shape[2],
'transform' : out_transform})
# writing and then re-reading the output data to see if it looks good
for i in range(out_img.shape[0]):
with rio.open(rf"C:\Dados_Espaciais\DSM\Bare_Soil_Landsat\Teste_{i+1}_masked.tif",'w',**out_meta) as dst:
dst.write(out_img[i,:,:],1)
This way you tell the program:
Check the areas in BQA with these "Cloud_Values" and exclude these areas, but in the landsat image that I provided.
I hope it works.