I am reading in an image one byte at a time with with read(1), and appending it to a list. The image data is all hex data. When I print out the list with the print function it is in the format '\xd7'
['\xd7', '\xd7', '\xd7', '\xd7', '\xd7', '\xd7', '\xd7',...]
The problem is that now I need to perform some calculations on this hex data, however, it is in string format, and this '\xd' string format isn't supported by any of the int or hex conversion functions in python. They require a '0xd7' or just a 'd7'.
Thanks for the help
It's interpreting them as characters, so use ord to turn them into numbers. I.e. ord('\xd7') gives 215.
Also if you use Windows, or the program might have to run on Windows, make sure that you've got the file open in binary mode: open("imagefile.png","rb"). Makes no difference on other operating systems.
If you require 'd7' or '0xd7', rather than simply 0xd7 (viz, 215), hex() or '%x' are your friend.
>>> ord('\xd7')
215
>>> ord('\xd7') == 215 == 0xd7
True
>>> hex(ord('\xd7'))
'0xd7'
>>> '%x' % ord('\xd7')
'd7'
Also as observed in other answers, do make sure you open with the 'b' in the mode, otherwise it can get messed up, thinking it's UTF-8 or something like that, on certain sequences of bytes.
You could do something like this to get them into a numeric array:
import array
data = array.array('B') # array of unsigned bytes
with open("test.dat", 'rb') as input:
data = input.read(100)
data.fromstring(data)
print data
# array('B', [215, 215, 215, 215, 215, 215, 215])
read() can take a size value larger than 1: read(1024) will read 1K worth of bytes from the stream. That will be a lot faster than reading a byte at a time and appending it to the previous bytes.
What are you trying to do when printing the data? See the byte values, or display the image?
The data isn't in "string format", it's just bytes, but when you print them the print routine will escape non-printing values into something that will mean more to human eyes and brains. If you want to see the values without the escaping you can iterate over the bytes and convert them to their hexadecimal values, or decimal, or binary - whatever works for you and your application. The string formatting mini-language will be a good starting place.
If you are doing image processing, then you probably want to look at numpy.
There are a few packages that will help you read your image into memory too (PIL is mentioned above, another is my own mahotas or scikits.image).
If the data is in a file as raw data an you know the dimensions, you can do the following
import numpy as np
img = np.empty( (n_rows, n_cols), dtype=np.uint8) # create an empty image
img.data[:] = input_file.read()
to get your data into img.
An introductory website for image processing in python is http://pythonvision.org.
Related
s1 = wave.open('sound.wav', 'r')
frames_1 = []
for i in range(44100):
frames_1.append(struct.unpack('<h', s1.readframes(1)))
Basically, I am trying to unpack s1.readframes(1), which should return an integer value between ~-32000 and 32000. readframes() returns a byte object. It gives me the message "struct.error: unpack requires a buffer of 2 bytes".
I am not sure if I am even supposed to use struct.unpack() or if there is something else that I should do. I assume I am because to write the frames to the .wav file I have to first convert the frames to bytes using data = value.writeframesraw() and then use struct.pack('<h', data), however I am not sure if I am using unpack() correctly. If I try to change '<h' to something else, it just increases the number of bytes in the error message. Optimally I want to only use the wave library for this code, but if there are other libraries that I could import that can read and write .wav files easily then please recommend them.
This function below reads an image and converts it into base64 image string
def getBase64Image(filePath):
with open(filePath, "rb") as img_file:
my_string = base64.b64encode(img_file.read())
my_string = my_string.decode('utf-8')
return my_string
However, the function below take image as array(loaded from OpenCV) and converts it into base64 image string
def convertToBase64(image):
image.tobytes()
my_string = base64.b64encode(image)
my_string = my_string.decode('utf-8')
return my_string
The output string from the first function differs from the string produced by the second function. Why is that?
Ideally, I want the second function to produce the same base64 string as the first function.
Please, can someone guide me on how I can achieve this?
You first function uses the PNG/JPG image data "as-is" and encodes it.
Your second function uses RAW bytes from the RGB or grayscale representation of the image and encodes that. If you want to convert RAW RGB to an image, you may use cv2.imencode() which will ouput PNG or JPG or whatever you like.
def convertToBase64(image):
#image.tobytes() -- don't need this
_, converted = cv2.imencode( '.png', image) # here the magic happens
my_string = base64.b64encode(converted) # and here
my_string = my_string.decode('utf-8')
return my_string
And yeah, just in case it's not clear. You DON'T have to save the encoded image anywhere, it's all happening in memory.
You are encoding fundamentally different structures. The first method is reading in the bytes of a compressed image format such as jpeg or png. Even if it is a bitmap image, there is a lot of extra data stored in the image that is not in the raw array data.
The second method is taking an h x w x 3 array of the pixel data, converting it to a byte string, and then 64bit encoding it. You can see the difference by comparing the byte strings of black and white data array compared to the saved bitmap image.
I am using Python 2.7
I am trying to use a large binary file that specifies the latitude/longitude of each pixel of an image.
Using this code: open('input', 'rb').read(50)
The file looks like this:
\x80\x0c\x00\x00\x00\x06\x00\x00.\x18\xca\xe4.\x18\xcc\xe4.\x18\xcf\xe4.\x18\xd1\xe4.\x18\xd3\xe4.\x18\xd5\xe4.\x18\xd7\xe4.\x18\xd9\xe4/\x18\xdb\xe4/\x18\xdd\xe4/\x18...
The read-me for the file gives the following information for decoding but I am unsure of how to apply them:
The files are in LSBF byte order. Files start with 2 4-byte integer values giving the pixel and line (x,y) size of the file. After the files have succeeding pairs of elements are 2-byte integer values of latitude and longitude multiplied 100 and truncated (e.g. 75.324 E is "-7532").
Thanks for any help.
Note the reason for doing this ultimately would be to draw/alter an image based on lat/long and not the pixel # in case anyone was wondering.
Generally when working with binary files you'll want to use the pack and unpack (https://docs.python.org/2/library/struct.html) functions. That will allow you to control the endianess of the data as well as the data types. In your specific case you'll probably want to read the header info first, something like
with open('input', 'rb') as fp:
# read the header bytes to get the number of elements
header_bytes = fp.read(8)
# convert the bytes to two unsigned integers.
x, y = unpack("II", header_bytes)
# Loop through the file getting the rest of the data
# read Y lines of X pixels
...
The above was not actually run or test, just trying to give you a general sense of an approach.
I'm reading an image file of dpx format, and want to extract the "Orientation" in the image section of the header, and also modify it. I have never tried to interpret binary data, so I'm a bit at a loss. I'm trying to use the struct module, but I really don't know how to do it properly. The file header specification is here:
http://www.fileformat.info/format/dpx/egff.htm
Thanks.
There seems to be a constant offset to the Orientation so if this is all you want to change then I wouldn't bother trying to parse it all, just work out the offset (which I think is just the size of the GENERICFILEHEADER plus one byte for the high byte of the orientation word) and read / manipulate it directly.
Using a bytearray would be my first choice. The offset varies by one depending on if it's in a big or little endian format, so something like this might work for you:
b = bytearray(your_byte_data)
big_endian = (b[0] == 0x52)
offset = 768 + big_endian
current_orientation = b[offset] # get current orientation
b[offset] = new_offset # set it to something new
open('out_file', 'wb').write(b)
You might want to consider using Imagemagic to do that. Open source and supports the dpx format.
The Python Imaging Library PIL has am attribute .info that might return the relevant data
I'm trying to write a video application in PyQt4 and I've used Python ctypes to hook into an old legacy video decoder library. The library gives me 32-bit ARGB data and I need to turn that into a QImage. I've got it working as follows:
# Copy the rgb image data from the pointer into the buffer
memmove(self.rgb_buffer, self.rgb_buffer_ptr, self.buffer_size)
# Copy the buffer to a python string
imgdata = ""
for a in self.rgb_buffer:
imgdata = imgdata + a
# Create a QImage from the string data
img = QImage(imgdata, 720, 288, QImage.Format_ARGB32)
The problem is that ctypes outputs the data as type "ctypes.c_char_Array_829440" and I need to turn it into a python string so that I can construct a QImage. My copying mechanism is currently taking almost 300ms per image so it's painfully slow. The decode and display part of the process is only taking about 50ms.
Can anyone think of any cunning shortcuts I can take to speed up this process and avoid the need to copy the buffer twice as I'm currently doing?
The ctypes.c_char_Array_829400 instance has the property .raw which returns a string possibly containing NUL bytes, and the property .value which returns the string up to the first NUL byte if it contains one or more.
However, you can also use ctypes the access the string at self.rgb_buffer_ptr, like this:
ctypes.string_at(self.rgb_buffer_ptr, self.buffer_size); this would avoid the need for the memmove call.