I am trying to convert image data saved in a rosbag file to numpy arrays and opencv images for further processing. I can not use cv_bridge or any of the other ROS utils.
I read the rosbag using the bagpy module here. And convert the data to a pandas dataframe:
import numpy as np
import cv2
import bagpy
from bagpy import bagreader
import matplotlib.pyplot as plt
import pandas as pd
import csv
b = bagreader('camera.bag')
image_csv = b.message_by_topic('/left/image')
df_limage = pd.read_csv('camera/left-image.csv')
Because the rosbag stores images as type bytestring, the df_limage dataframe looks like:
>>> df_limage.head()
time height width encoding is_bigendian data
1.593039e+09 1080 1920 rgb8 0 b' \'\n"*\x0c$\'\x14\x1f...
When I try to examine the image stored in the data column, I see that each image is stored as a string:
>>> type(df_limage['data'][0])
str
>>> len(df_limage['data'][0])
15547333
>>> print(df_limage['data'][0])
b' \'\n"*\x0c$\'\x14\x1f#\x0f\x1d!\x12 %\x16\x1f\'\x0e\x1c%\x0b\x1c&\x12\x19#\x10\x1e#\x13\x1f$\x14##\x16!!\x13$$"$$"&*\x12$(\x1...
When I try to decode this using code from this answer, I get warnings and NoneType returns:
>>> nparr = np.fromstring(df_limage['data'][0], np.uint8)
DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead
>>> img_np = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
>>> type(img_np)
NoneType
I think this is because the string isn't being read correctly as a bytestring and nparr hasn't been reshaped into a 3-channel RGB image of dimensions (1080 x 1920). The size of nparr is 15547333, so it can't be reshaped into a (1080 x 1920 x 3) image which leads me to believe that the np.fromstring call isn't correct.
How do I take a binarystring that is represented as string with a leading "b'", convert that back to a binarystring so I can then convert it into an array, and then an opencv image?
Thanks
Your image is pure rgb8 pixels in a bytes type. That means:
it isn't a str and you shouldn't treat it as such, and
it isn't JPEG-encoded or PNG-encoded, so you shouldn't be passing it to cv2.imdecode() because that decompresses images and turns them into Numpy arrays of pixels, which is nearly what you already have.
So, you have a number of contiguous bytes representing pixels. The length of your bytes should be 1920x1080x3, i.e. one byte per channel for 3 channels of 1080p dimensions. We need to make a Numpy array and then reshape it from a long line into 1080p:
na = np.frombuffer(YOURBYTES).reshape((1080,1920,3))
General rule:
Part 1
You should generally only be calling cv2.imdecode() on things that look like either a PNG:
b'\x89PNG\r\n\x1a\n\x00\x00...'
or a JPEG:
b'\xff\xd8\xff\xe0\x00\x10JFIF...'
or a TIFF ( b'II' or b'MM') or BMP (b'BM') magic signature.
Part 2
If your buffer begins with a base64-encoded version of either of the above, i.e. iVBORw0KGgo= (PNG) or /9 (JPEG), you need to base64-decode, then call cv2.imdecode() the result of that.
from base64 import b64decode
import numpy as np
import cv2
# Extract JPEG-encoded image from base64-encoded string
JPEG = b64decode(YOURDATA)
# Decode JPEG back into Numpy array
na = cv2.imdecode(np.frombuffer(JPEG,dtype=np.uint8), cv2.IMREAD_COLOR)
Part 3
If your data is bytes type and already has the same length as the dimensions of your image, i.e. len(YOURBYTES) == height*width*nChannels like you have, that means it is pure, uncompressed pixels, so you just need the first part of this answer:
na = np.frombuffer(YOURBYTES).reshape((1080,1920,3))
Note that, unlike in Parts 1 and 2 above, the reshaping is necessary here because there was no JPEG or PNG metadata telling us the height and width of the image.
This is what I ended up having to do (without using the ast library):
>>> import numpy as np
>>> import pandas as pd
>>> import matplotlib.pyplot as plt
# read image data as raw string from csv
>>> df = pd.read_csv('camera_data.csv')
>>> df.head()
Time data
0 11578 b' \'\n"*\x0c$\'\x14\x1f#\x0f\x1d!\x12 %\x16...
1 11579 b'\x19)\n\x15%\x07 (\x0f\x1d&\x0c\x16$\x18\x15...
2 11580 b'\x1a)\x04\x17&\x01\x17&\x13\x16%\x12\x1f...
3 11581 b'\x18%\x03\x19&\x04!$\x03\x1f"\x01\x1e#\x11\...
# access the raw string representation of first image string in column df['data']
# raw string appears as: 'b\' \\\'\\n"*\\x0c$\\\'\\x14\\x1f#...'
raw_string = df_left_image['data'][0]
# convert to byte string with escape characters included
byte_string = raw_string[2:-1].encode('latin1')
# remove escaped characters
escaped_string = byte_string.decode('unicode_escape')
# convert back to byte string without escaped characters
byte_string = escaped_string.encode('latin1')
# convert string to numpy array
# this will throw a warning to use np.frombuffer
nparr = np.fromstring(byte_string, np.uint8)
# convert to 3 channel rgb image array of (H x W x 3)
rgb = nparr.reshape((1080, 1920, -1))
# show image in matplotlib
plt.imshow(rgb)
Related
I am trying to save an altered z-stack .tif file in Python3. Here's my code where I checked that the the functions worked as intended.
#libraries
import imageio as ii
#import initial image
fname='101_nuc1syg1.tif'
adata = ii.volread(fname)
#check to make sure volread works
ii.volsave('temp.tif', adata)
Which results in this:
And now when I try to do a simple threshold, using the following code:
#now doing very simple thresholding
bdata = adata < adata[0].mean()
bdata = bdata +0
ii.volsave('temp.tif', bdata)
I get this:
Any idea how to save a tif file properly after performing image operators on it?
EDIT: Note that I am able to extract each stack and save them as separate .png files, but I would prefer to have them as a single .tif file.
Data from: https://www.nature.com/articles/s41467-020-15987-2
It looks like you need to convert bdata type to np.uint8 and multiply the result by 255.
The type of the expression (adata < adata[0].mean()) is np.bool.
The common image type is np.uint8.
Convert (adata < adata[0].mean()) to type np.uint8:
bdata = (adata < adata[0].mean()).astype(np.uint8)
When converting the result to np.uint8, all True elements are converted to 1 and False elements are converted to 0.
Multiply by 255 for converting the True elements to 255 (white color):
bdata = bdata * 255
Complete code:
import imageio as ii
import numpy as np
#import initial image
fname='101_nuc1syg1.tif'
adata = ii.volread(fname)
#now doing very simple thresholding
bdata = (adata < adata[0].mean()).astype(np.uint8)
bdata = bdata * 255 # Convert ones to 255 (255 is white color)
ii.volsave('temp.tif', bdata)
Note:
I could not test my answer - the link you have posted doesn't contain a link to an image.
I'm trying to load image from string like as PHP function imagecreatefromstring
How can I do that?
I have MySQL blob field image. I'm using MySQLdb and don't want create temporary file for working with images in PyOpenCV.
NOTE: need cv (not cv2) wrapper function
This is what I normally use to convert images stored in database to OpenCV images in Python.
import numpy as np
import cv2
from cv2 import cv
# Load image as string from file/database
fd = open('foo.jpg')
img_str = fd.read()
fd.close()
# CV2
nparr = np.fromstring(img_str, np.uint8)
img_np = cv2.imdecode(nparr, cv2.CV_LOAD_IMAGE_COLOR) # cv2.IMREAD_COLOR in OpenCV 3.1
# CV
img_ipl = cv.CreateImageHeader((img_np.shape[1], img_np.shape[0]), cv.IPL_DEPTH_8U, 3)
cv.SetData(img_ipl, img_np.tostring(), img_np.dtype.itemsize * 3 * img_np.shape[1])
# check types
print type(img_str)
print type(img_np)
print type(img_ipl)
I have added the conversion from numpy.ndarray to cv2.cv.iplimage, so the script above will print:
<type 'str'>
<type 'numpy.ndarray'>
<type 'cv2.cv.iplimage'>
EDIT:
As of latest numpy 1.18.5 +, the np.fromstring raise a warning, hence np.frombuffer shall be used in that place.
I think this answer provided on this stackoverflow question is a better answer for this question.
Quoting details (borrowed from #lamhoangtung from above linked answer)
import base64
import json
import cv2
import numpy as np
response = json.loads(open('./0.json', 'r').read())
string = response['img']
jpg_original = base64.b64decode(string)
jpg_as_np = np.frombuffer(jpg_original, dtype=np.uint8)
img = cv2.imdecode(jpg_as_np, flags=1)
cv2.imwrite('./0.jpg', img)
I've try to use this code to create an opencv from a string containing a raw buffer (plain pixel data) and it doesn't work in that peculiar case.
So here's how to do that for this kind of data:
image = np.fromstring(im_str, np.uint8).reshape( h, w, nb_planes )
(but yes you need to know your image properties)
if your B and G channel is permuted, here's how to fix it:
image = cv2.cvtColor(image, cv2.cv.CV_BGR2RGB)
I was following the solution from #jabaldonedo but it seems it's a bit old and need some adjustments.
I am using OpenCV 3.4.8.29 by the way.
im_path = 'path/to/foo.jpg'
with open(im_path, 'rb') as fp:
im_b = fp.read()
image_np = np.frombuffer(im_b, np.uint8)
img_np = cv2.imdecode(image_np, cv2.IMREAD_COLOR)
im_cv = cv2.imread(im_path)
print('Same image: {}'.format(np.all(im_cv == img_np)))
Same image: True
One gotcha of imdecode:
If the buffer is too short or contains invalid data, the function returns [None]
This feels uncharacteristically lenient from OpenCV. Here's a function that accommodates for this:
import numpy as np
import cv2 as cv
def read_image(content: bytes) -> np.ndarray:
"""
Image bytes to OpenCV image
:param content: Image bytes
:returns OpenCV image
:raises TypeError: If content is not bytes
:raises ValueError: If content does not represent an image
"""
if not isinstance(content, bytes):
raise TypeError(f"Expected 'content' to be bytes, received: {type(content)}")
image = cv.imdecode(np.frombuffer(content, dtype=np.uint8), cv.IMREAD_COLOR)
if image is None:
raise ValueError(f"Expected 'content' to be image bytes")
return image
In the input of a program is given height amount of lines that have width amount of RRGGBB values in them, with RR/GG/BB being a hexadecimal value of the corresponding color in an RGB format.
I need to take the input and convert it to an OpenCV image so that I could interact with it using the OpenCV library. How would I accomplish this?
Example of input:
https://drive.google.com/file/d/1XuKRuAiQLUv4rbVxl2xTgqYr_8JQeu63/view?usp=sharing
The first number is height, second is width, the rest of the text file is the image itself.
That is a really inefficient way to store an image, and this is a correspondingly inefficient way to unpack it!
#!/usr/bin/env python3
import numpy as np
import re
import cv2
# Read in entire file
with open('in.txt') as f:
s = f.read()
# Find anything that looks like numbers
l=re.findall(r'[0-9a-f]+',s)
# Determine height and width
height = int(l[0])
width = int(l[1])
# Create numpy array of BGR triplets
im = np.zeros((height,width,3), dtype=np.uint8)
i = 2
for row in range (height):
for col in range(width):
hex = l[i]
R = int(hex[0:2],16)
G = int(hex[2:4],16)
B = int(hex[4:6],16)
im[row,col] = (B,G,R)
i = i+1
# Save to disk
cv2.imwrite('result.png', im)
In case the data file disappears in future, this is how the first few lines look:
1080 1920
232215 18180b 18170b 18180b 18170a 181609 181708 171708 15160c 14170d
15170d 16170d 16160d 16170d 16170d 16170d 15160d 15160d 17170e 17180f
17180f 18180f 191a11 191a12 1c1c0f 1d1d0f 1e1d0f 1f1e10 1e1e10 1f1f12
202013 202113 212214 242413 242413 242413 242412 242410 242611 272610
272612 262712 262710 282811 27290f 2a2b10 2b2c12 2c2d12 2e3012 303210
Keywords: Python, Numpy, OpenCV, parse, hex, hexadecimal, image, image processing, regex
When I look at imread function in skimage.io, it doesn't say what calculation is used when as_grey=True is set, is there a way to find the calculation going on behind the scenes?
Link to lib:
(http://scikit-image.org/docs/dev/api/skimage.io.html#skimage.io.imread)
Text from link above:
as_grey : bool
If True, convert color images to grey-scale (64-bit floats). Images that are already in grey-scale format are not converted.
Example:
RGB - [108 123 128]
When I use convert('L'), it gets converted to 119 and that's inline with the formula on this post How can I convert an RGB image into grayscale in Python?
But when I use imread(img, as_grey = True), it gives me a value of 0.47126667, which is lower than the value if I were to divide the 119 value with the max value of pixel in that image to convert the values to 0-1 scale.
If you want to view the results, here's sample code:
from __future__ import division
from skimage.io import imread
import numpy as np
image_open = Image.open(image).convert('L')
np_image_open = np.array(image_open)
print (np_image_open[:10,0])
print (np_image_open[:10,0]/np.max(np_image_open))
image_open = imread(image, as_grey = True)
print (image_open[:10,0])
image_open = imread(image)
print (image_open[:10,0])
I have a .jpg image that I would like to convert to Python array, because I implemented treatment routines handling plain Python arrays.
It seems that PIL images support conversion to numpy array, and according to the documentation I have written this:
from PIL import Image
im = Image.open("D:\Prototype\Bikesgray.jpg")
im.show()
print(list(np.asarray(im)))
This is returning a list of numpy arrays. Also, I tried with
list([list(x) for x in np.asarray(im)])
which is returning nothing at all since it is failing.
How can I convert from PIL to array, or simply from numpy array to Python array?
I highly recommend you use the tobytes function of the Image object. After some timing checks this is much more efficient.
def jpg_image_to_array(image_path):
"""
Loads JPEG image into 3D Numpy array of shape
(width, height, channels)
"""
with Image.open(image_path) as image:
im_arr = np.fromstring(image.tobytes(), dtype=np.uint8)
im_arr = im_arr.reshape((image.size[1], image.size[0], 3))
return im_arr
The timings I ran on my laptop show
In [76]: %timeit np.fromstring(im.tobytes(), dtype=np.uint8)
1000 loops, best of 3: 230 µs per loop
In [77]: %timeit np.array(im.getdata(), dtype=np.uint8)
10 loops, best of 3: 114 ms per loop
```
I think what you are looking for is:
list(im.getdata())
or, if the image is too big to load entirely into memory, so something like that:
for pixel in iter(im.getdata()):
print pixel
from PIL documentation:
getdata
im.getdata() => sequence
Returns the contents of an image as a sequence object containing pixel
values. The sequence object is flattened, so that values for line one
follow directly after the values of line zero, and so on.
Note that the sequence object returned by this method is an internal
PIL data type, which only supports certain sequence operations,
including iteration and basic sequence access. To convert it to an
ordinary sequence (e.g. for printing), use list(im.getdata()).
Based on zenpoy's answer:
import Image
import numpy
def image2pixelarray(filepath):
"""
Parameters
----------
filepath : str
Path to an image file
Returns
-------
list
A list of lists which make it simple to access the greyscale value by
im[y][x]
"""
im = Image.open(filepath).convert('L')
(width, height) = im.size
greyscale_map = list(im.getdata())
greyscale_map = numpy.array(greyscale_map)
greyscale_map = greyscale_map.reshape((height, width))
return greyscale_map
I use numpy.fromiter to invert a 8-greyscale bitmap, yet no signs of side-effects
import Image
import numpy as np
im = Image.load('foo.jpg')
im = im.convert('L')
arr = np.fromiter(iter(im.getdata()), np.uint8)
arr.resize(im.height, im.width)
arr ^= 0xFF # invert
inverted_im = Image.fromarray(arr, mode='L')
inverted_im.show()