I am trying to convert image data saved in a rosbag file to numpy arrays and opencv images for further processing. I can not use cv_bridge or any of the other ROS utils.
I read the rosbag using the bagpy module here. And convert the data to a pandas dataframe:
import numpy as np
import cv2
import bagpy
from bagpy import bagreader
import matplotlib.pyplot as plt
import pandas as pd
import csv
b = bagreader('camera.bag')
image_csv = b.message_by_topic('/left/image')
df_limage = pd.read_csv('camera/left-image.csv')
Because the rosbag stores images as type bytestring, the df_limage dataframe looks like:
>>> df_limage.head()
time height width encoding is_bigendian data
1.593039e+09 1080 1920 rgb8 0 b' \'\n"*\x0c$\'\x14\x1f...
When I try to examine the image stored in the data column, I see that each image is stored as a string:
>>> type(df_limage['data'][0])
str
>>> len(df_limage['data'][0])
15547333
>>> print(df_limage['data'][0])
b' \'\n"*\x0c$\'\x14\x1f#\x0f\x1d!\x12 %\x16\x1f\'\x0e\x1c%\x0b\x1c&\x12\x19#\x10\x1e#\x13\x1f$\x14##\x16!!\x13$$"$$"&*\x12$(\x1...
When I try to decode this using code from this answer, I get warnings and NoneType returns:
>>> nparr = np.fromstring(df_limage['data'][0], np.uint8)
DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead
>>> img_np = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
>>> type(img_np)
NoneType
I think this is because the string isn't being read correctly as a bytestring and nparr hasn't been reshaped into a 3-channel RGB image of dimensions (1080 x 1920). The size of nparr is 15547333, so it can't be reshaped into a (1080 x 1920 x 3) image which leads me to believe that the np.fromstring call isn't correct.
How do I take a binarystring that is represented as string with a leading "b'", convert that back to a binarystring so I can then convert it into an array, and then an opencv image?
Thanks
Your image is pure rgb8 pixels in a bytes type. That means:
it isn't a str and you shouldn't treat it as such, and
it isn't JPEG-encoded or PNG-encoded, so you shouldn't be passing it to cv2.imdecode() because that decompresses images and turns them into Numpy arrays of pixels, which is nearly what you already have.
So, you have a number of contiguous bytes representing pixels. The length of your bytes should be 1920x1080x3, i.e. one byte per channel for 3 channels of 1080p dimensions. We need to make a Numpy array and then reshape it from a long line into 1080p:
na = np.frombuffer(YOURBYTES).reshape((1080,1920,3))
General rule:
Part 1
You should generally only be calling cv2.imdecode() on things that look like either a PNG:
b'\x89PNG\r\n\x1a\n\x00\x00...'
or a JPEG:
b'\xff\xd8\xff\xe0\x00\x10JFIF...'
or a TIFF ( b'II' or b'MM') or BMP (b'BM') magic signature.
Part 2
If your buffer begins with a base64-encoded version of either of the above, i.e. iVBORw0KGgo= (PNG) or /9 (JPEG), you need to base64-decode, then call cv2.imdecode() the result of that.
from base64 import b64decode
import numpy as np
import cv2
# Extract JPEG-encoded image from base64-encoded string
JPEG = b64decode(YOURDATA)
# Decode JPEG back into Numpy array
na = cv2.imdecode(np.frombuffer(JPEG,dtype=np.uint8), cv2.IMREAD_COLOR)
Part 3
If your data is bytes type and already has the same length as the dimensions of your image, i.e. len(YOURBYTES) == height*width*nChannels like you have, that means it is pure, uncompressed pixels, so you just need the first part of this answer:
na = np.frombuffer(YOURBYTES).reshape((1080,1920,3))
Note that, unlike in Parts 1 and 2 above, the reshaping is necessary here because there was no JPEG or PNG metadata telling us the height and width of the image.
This is what I ended up having to do (without using the ast library):
>>> import numpy as np
>>> import pandas as pd
>>> import matplotlib.pyplot as plt
# read image data as raw string from csv
>>> df = pd.read_csv('camera_data.csv')
>>> df.head()
Time data
0 11578 b' \'\n"*\x0c$\'\x14\x1f#\x0f\x1d!\x12 %\x16...
1 11579 b'\x19)\n\x15%\x07 (\x0f\x1d&\x0c\x16$\x18\x15...
2 11580 b'\x1a)\x04\x17&\x01\x17&\x13\x16%\x12\x1f...
3 11581 b'\x18%\x03\x19&\x04!$\x03\x1f"\x01\x1e#\x11\...
# access the raw string representation of first image string in column df['data']
# raw string appears as: 'b\' \\\'\\n"*\\x0c$\\\'\\x14\\x1f#...'
raw_string = df_left_image['data'][0]
# convert to byte string with escape characters included
byte_string = raw_string[2:-1].encode('latin1')
# remove escaped characters
escaped_string = byte_string.decode('unicode_escape')
# convert back to byte string without escaped characters
byte_string = escaped_string.encode('latin1')
# convert string to numpy array
# this will throw a warning to use np.frombuffer
nparr = np.fromstring(byte_string, np.uint8)
# convert to 3 channel rgb image array of (H x W x 3)
rgb = nparr.reshape((1080, 1920, -1))
# show image in matplotlib
plt.imshow(rgb)
Disaster!
As you can see, the image isn't quite loaded correctly. The original:
The code:
import cv2
import imutils
a=imutils.url_to_image("https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_272x92dp.png", readFlag=-1)
cv2.imshow("goog", a)
cv2.waitKey()
The implementation of url_to_image in imutils:
def url_to_image(url, readFlag=cv2.IMREAD_COLOR):
# download the image, convert it to a NumPy array, and then read
# it into OpenCV format
resp = urlopen(url)
image = np.asarray(bytearray(resp.read()), dtype="uint8")
image = cv2.imdecode(image, readFlag)
# return the image
return image
I also tried readFlag=cv2.IMREAD_UNCHANGED, but that didn't do the trick either.
please send help
alright gang we did it
so I tried another version of displaying:
plt.figure("Correct")
plt.imshow(imutils.opencv2matplotlib(a))
plt.show()
No luck it would appear. But then, looking into the opencv2matplotlib source, we find:
def opencv2matplotlib(image):
# OpenCV represents images in BGR order; however, Matplotlib
# expects the image in RGB order, so simply convert from BGR
# to RGB and return
return cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
Aha, but we have 4 channel color (alpha), so by common sense we need cv2.COLOR_BGRA2RGBA not cv2.COLOR_BGR2RGB!!
Testing this theory:
plt.figure("Correct")
plt.imshow(cv2.cvtColor(a, cv2.COLOR_BGRA2RGBA))
plt.show()
We get...
Whoop dee doop!
# import the necessary packages
import numpy as np
import urllib
import cv2
def url_to_image(url):
# download the image, convert it to a NumPy array, and then read
# it into OpenCV format
resp = urllib.request.urlopen(url)
image = np.asarray(bytearray(resp.read()), dtype="uint8")
image = cv2.imdecode(image, cv2.IMREAD_COLOR)
# return the image
return image
# initialize the list of image URLs to download
url="http://i.dailymail.co.uk/i/pix/2015/09/01/18/2BE1E88B00000578-3218613-image-m-5_1441127035222.jpg"
print ("downloading %s" % (url))
image = url_to_image(url)
cv2.imshow("Image", image)
cv2.waitKey(0)
And the output is:
So basically I'm trying to convert a set of RGB images to grayscale using cv2.cvtColor and python is throwing the following error:
Traceback (most recent call last):
File "MCG.py", line 53, in
gray = cv2.cvtColor(data, cv2.COLOR_BGR2GRAY)
TypeError: src is not a numpy array, neither a scalar.
This here is the code:
import numpy as np
import cv2
import dlib
import sys
import skimage
from PIL import Image
import os
import glob
folderpath = sys.argv[1]
cascPath = sys.argv[2]
imageformat = ".tif"
path = folderpath
imfilelist = [os.path.join(path,f) for f in os.listdir(path) if f.endswith(imageformat)]
data = []
for IMG in imfilelist:
print IMG
image = cv2.imread(IMG)
data.append(image)
cv2.imshow('Image', image)
cv2.waitKey(0)
faceCascade = cv2.CascadeClassifier(cascPath)
predictor = dlib.shape_predictor(PREDICTOR_PATH)
gray = cv2.cvtColor(data, cv2.COLOR_BGR2GRAY)
faces = faceCascade.detectMultiScale(
gray,
scaleFactor=1.05,
minNeighbors=5,
minSize=(100,100)
)
As you can see, I'm trying to append all these images to a list, which will then be converted using the cv2.cvtColor function. However, that error is thrown. What am I doing wrong? Thank you.
P.S if anyone is wondering why I imported modules that don't seem to be used in this code, this code is just a segment of the whole thing and all of those modules have are being utilized in one way or the other.
If you read the cv2.cvtColor documentation, you can see that the first parameter is the Src 8-bit single channel image. However, in your case you are giving an entire list of images.
So change the code as
gray = []
for j in range(0,len(data)):
gray.append(cv2.cvtColor(np.array(data[j]), cv2.COLOR_BGR2GRAY))
I guess this should work.
You are collecting the images into a list with
data = []
for IMG in imfilelist:
...
data.append(image)
....
and then trying to convert the list with
gray = cv2.cvtColor(data, cv2.COLOR_BGR2GRAY)
This is why you are getting the error - the error is telling you that data is not an image (numpy array) but is a list. You need to convert one image at a time with cv2.cvtColor().
You could try
gray = []
for img in data:
gray.append(cv2.cvtColor(img, cv2.COLOR_BGR2GRAY))
This would give you a list of greyscaled images, which is what I think you want to do.
I have a Block blob which is a png image. When I get the blob to bytes and try to create an image I get not enough data.
I am new to python and image analysis. The problem is Size of the blob is 1714562 and Width ad height if the image is 2024*1512 respectively.
Below is the code which I retrive the image:
#Get Required image
def findImage(containerName, imageName):
return block_blob_service.get_blob_to_bytes(
containerName,
input_folder + '/' + imageName,
max_connections=1).content;
And I have to do such a thing as described below to create an image:
np.array(imageToProcess).reshape(imageSize)
Or
PIL.Image.frombytes('L', imageSize, np.byte(imageToProcess), 'raw', 'L', 0, 1)
imageToProcess is the output of the findImage function.
I tried to get blob to stream or path still have the same issue. This Blob needs to be reshaped to actual image size and pass to another process function.
I will appreciate it if someone help me to undrestand the problem and possible solutions.
The PNG file you downloaded from Azure Blob Storage is a png-format binary byte string, you need to convert it to numpy array using OpenCV & Numpy or to PIL.Image object.
Here are my sample code using OpenCV & numpy or PIL.Image.
Using OpenCV & numpy
import numpy as np
import cv2
data = findImage(containerName, imageName)
print type(data)
# <type 'str'>
nparr = np.fromstring(data, np.uint8)
# if the numpy version is 2.x
img_np = cv2.imdecode(nparr, cv2.CV_LOAD_IMAGE_COLOR)
# if the numpy version is 3.x
img_np = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
print type(img_np)
# <type 'numpy.ndarray'>
Using PIL.Image & StringIO
from PIl import Image
from StringIO import StringIO
data = findImage(containerName, imageName)
print type(data)
# <type 'str'>
img = Image.open(StringIO(data))
print type(img)
# <class 'PIL.PngImagePlugin.PngImageFile'>
Hope it helps.
I have an RGB image. I want to convert it to numpy array. I did the following
im = cv.LoadImage("abc.tiff")
a = numpy.asarray(im)
It creates an array with no shape. I assume it is a iplimage object.
You can use newer OpenCV python interface (if I'm not mistaken it is available since OpenCV 2.2). It natively uses numpy arrays:
import cv2
im = cv2.imread("abc.tiff",mode='RGB')
print(type(im))
result:
<type 'numpy.ndarray'>
PIL (Python Imaging Library) and Numpy work well together.
I use the following functions.
from PIL import Image
import numpy as np
def load_image( infilename ) :
img = Image.open( infilename )
img.load()
data = np.asarray( img, dtype="int32" )
return data
def save_image( npdata, outfilename ) :
img = Image.fromarray( np.asarray( np.clip(npdata,0,255), dtype="uint8"), "L" )
img.save( outfilename )
The 'Image.fromarray' is a little ugly because I clip incoming data to [0,255], convert to bytes, then create a grayscale image. I mostly work in gray.
An RGB image would be something like:
out_img = Image.fromarray( ycc_uint8, "RGB" )
out_img.save( "ycc.tif" )
You can also use matplotlib for this.
from matplotlib.image import imread
img = imread('abc.tiff')
print(type(img))
output:
<class 'numpy.ndarray'>
As of today, your best bet is to use:
img = cv2.imread(image_path) # reads an image in the BGR format
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) # BGR -> RGB
You'll see img will be a numpy array of type:
<class 'numpy.ndarray'>
Late answer, but I've come to prefer the imageio module to the other alternatives
import imageio
im = imageio.imread('abc.tiff')
Similar to cv2.imread(), it produces a numpy array by default, but in RGB form.
You need to use cv.LoadImageM instead of cv.LoadImage:
In [1]: import cv
In [2]: import numpy as np
In [3]: x = cv.LoadImageM('im.tif')
In [4]: im = np.asarray(x)
In [5]: im.shape
Out[5]: (487, 650, 3)
You can get numpy array of rgb image easily by using numpy and Image from PIL
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt
im = Image.open('*image_name*') #These two lines
im_arr = np.array(im) #are all you need
plt.imshow(im_arr) #Just to verify that image array has been constructed properly
When using the answer from David Poole I get a SystemError with gray scale PNGs and maybe other files. My solution is:
import numpy as np
from PIL import Image
img = Image.open( filename )
try:
data = np.asarray( img, dtype='uint8' )
except SystemError:
data = np.asarray( img.getdata(), dtype='uint8' )
Actually img.getdata() would work for all files, but it's slower, so I use it only when the other method fails.
load the image by using following syntax:-
from keras.preprocessing import image
X_test=image.load_img('four.png',target_size=(28,28),color_mode="grayscale"); #loading image and then convert it into grayscale and with it's target size
X_test=image.img_to_array(X_test); #convert image into array
OpenCV image format supports the numpy array interface. A helper function can be made to support either grayscale or color images. This means the BGR -> RGB conversion can be conveniently done with a numpy slice, not a full copy of image data.
Note: this is a stride trick, so modifying the output array will also change the OpenCV image data. If you want a copy, use .copy() method on the array!
import numpy as np
def img_as_array(im):
"""OpenCV's native format to a numpy array view"""
w, h, n = im.width, im.height, im.channels
modes = {1: "L", 3: "RGB", 4: "RGBA"}
if n not in modes:
raise Exception('unsupported number of channels: {0}'.format(n))
out = np.asarray(im)
if n != 1:
out = out[:, :, ::-1] # BGR -> RGB conversion
return out
I also adopted imageio, but I found the following machinery useful for pre- and post-processing:
import imageio
import numpy as np
def imload(*a, **k):
i = imageio.imread(*a, **k)
i = i.transpose((1, 0, 2)) # x and y are mixed up for some reason...
i = np.flip(i, 1) # make coordinate system right-handed!!!!!!
return i/255
def imsave(i, url, *a, **k):
# Original order of arguments was counterintuitive. It should
# read verbally "Save the image to the URL" — not "Save to the
# URL the image."
i = np.flip(i, 1)
i = i.transpose((1, 0, 2))
i *= 255
i = i.round()
i = np.maximum(i, 0)
i = np.minimum(i, 255)
i = np.asarray(i, dtype=np.uint8)
imageio.imwrite(url, i, *a, **k)
The rationale is that I am using numpy for image processing, not just image displaying. For this purpose, uint8s are awkward, so I convert to floating point values ranging from 0 to 1.
When saving images, I noticed I had to cut the out-of-range values myself, or else I ended up with a really gray output. (The gray output was the result of imageio compressing the full range, which was outside of [0, 256), to values that were inside the range.)
There were a couple other oddities, too, which I mentioned in the comments.
We can use following function of open CV2 to convert BGR 2 RGB format.
RBG_Image = cv2.cvtColor(Image, cv.COLOR_BGR2RGB)
Using Keras:
from keras.preprocessing import image
img = image.load_img('path_to_image', target_size=(300, 300))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
images = np.vstack([x])
Try timing the options to load an image to numpy array, they are quite similar. Go for plt.imread for simplicity and speed.
def time_this(function, times=100):
cum_time = 0
for t in range(times):
st = time.time()
function()
cum_time += time.time() - st
return cum_time / times
import matplotlib.pyplot as plt
def load_img_matplotlib(img_path):
return plt.imread(img_path)
import cv2
def load_img_cv2(img_path):
return cv2.cvtColor(cv2.imread(img_path), cv2.COLOR_BGR2RGB)
from PIL import Image
import numpy as np
def load_img_pil(img_path):
img = Image.open(img_path)
img.load()
return np.asarray( img, dtype="int32" )
if __name__=='__main__':
img_path = 'your_image_path'
for load_fn in [load_img_pil, load_img_cv2, load_img_matplotlib]:
print('-'*20)
print(time_this(lambda: load_fn(img_path)), 10000)
Result:
--------------------
0.0065201687812805175 10000 PIL, as in [the second answer][1]https://stackoverflow.com/a/7769424/16083419)
--------------------
0.0053211402893066405 10000 CV2
--------------------
0.005320906639099121 10000 matplotlib
You can try the following method. Here is a link to the docs.
tf.keras.preprocessing.image.img_to_array(img, data_format=None, dtype=None)
from PIL import Image
img_data = np.random.random(size=(100, 100, 3))
img = tf.keras.preprocessing.image.array_to_img(img_data)
array = tf.keras.preprocessing.image.img_to_array(img)