We Keep Coding

C++ format pixels data for PySide2 QImage.fromData()

I am developing an Image viewer.
My main goal is to make something able to easily load .EXR files along with all other main formats.
I based it around :
PySide2 (Qt) for the UI.
OpenImageIO for .exr files (and some others) loading. I built a custom python module based around OpenImageIO (made with C++ and PyBind11)
On the c++ side, the output is pretty straight forward, my custom module return a simple struct
( nicely 'converted' into python by pybind11) :
struct img_data
{
bool is_valid;
int width;
int height;
int nchannels;
std::vector<unsigned char> pixels;
};
It's less straightforward when in python:
# get image_data from custom module
image_data = custom_exr_loader.load_image(image_path)
# convert pixels to a numpy array
np_array = np.uint8(image_data.pixels)
np_array.shape = (image_data.height, image_data.width, image_data.nchannels)
# convert to a PIL Image
pil_image = Image.fromarray(np_array).convert("RGBA")
# convert to a QImage
image_qt : ImageQt = ImageQt(pil_image)
# and finally convert to a QPixmap for final consumption ...
pixmap = QPixmap.fromImage(image_qt, Qt.ImageConversionFlag.ColorOnly)
Performance is pretty bad. Especially when caching a whole directory of big EXRs
I bet there is a better way to do this ...
Of course the idea is to offload some of the work from python to C++ land. But I'm not sure how.
I tried using QImage.fromData() method, but I never got a working result.
It requires a QByteArray as a param.
How should I format pixels data in c++ to be able to use QImage.fromData() from UI code ?

The problem here are the copies of the data. You are actually using 5 representations:
mage_data from load_image
numpy array
pil image with conversion
QImage
QPixmap again with conversion
All your conversion as far as I can see make a complete copy of the image. Sometimes you might even convert the data. This is detrimental for performance.
What you need is are converters that don't make a copy but just memory map the data in some array. Ideally you would need to do this for your binding as well. I am not sure how PyBind11 works, but you need to somehow give access to the raw C++ memory used there. Also you might be able to use QImage from the C++ side as such a wrapper. For numpy you need to use np.frombuffer. To go to Qt you can do something like
im_np = np.array(img)
qimage = QImage(im_np.data, im_np.shape[1], im_np.shape[0],
QImage.Format_BGR888)
Also you need to find a representation (column, pixel format) that is supported by all those entities. It's not an easy task. You would need to research a copy free mapping for each of your conversion. Examples are qimage_from_numpy, library_qimage2ndarray, pil_from_numpy. And there are many more.

Adding custom extratags with tifffile

I'm trying to write a script to simplify my everyday life in the lab. I operate one ThermoFisher / FEI scanning electron microscope and I save all my pictures in the TIFF format.
The microscope software is adding an extensive custom TiffTag (code 34682) containing all the microscope / image parameters.
In my script, I would like to open an image, perform some manipulations and then save the data in a new file, including the original FEI metadata. To do so, I would like to use a python script using the tifffile module.
I can open the image file and perform the needed manipulations without problems. Retrieving the FEI metadata from the input file is also working fine.
I was thinking to use the imwrite function to save the output file and using the extratags optional argument to transfer to the output file the original FEI metadata.
This is an extract of the tifffile documentation about the extratags:
extratags : sequence of tuples
Additional tags as [(code, dtype, count, value, writeonce)].
code : int
The TIFF tag Id.
dtype : int or str
Data type of items in 'value'. One of TIFF.DATATYPES.
count : int
Number of data values. Not used for string or bytes values.
value : sequence
'Count' values compatible with 'dtype'.
Bytes must contain count values of dtype packed as binary data.
writeonce : bool
If True, the tag is written to the first page of a series only.
Here is a snippet of my code.
my_extratags = [(input_tags['FEI_HELIOS'].code,
input_tags['FEI_HELIOS'].dtype,
input_tags['FEI_HELIOS'].count,
input_tags['FEI_HELIOS'].value, True)]
tifffile.imwrite('output.tif', data, extratags = my_extratags)
This code is not working and complaining that the value of the extra tag should be ASCII 7-bit encoded. This looks already very strange to me because I haven't touched the metadata and I am just copying it to the output file.
If I convert the metadata tag value in a string as below:
my_extratags = [(input_tags['FEI_HELIOS'].code,
input_tags['FEI_HELIOS'].dtype,
input_tags['FEI_HELIOS'].count,
str(input_tags['FEI_HELIOS'].value), True)]
tifffile.imwrite('output.tif', data, extratags = my_extratags)
the code is working, the image is saved, the metadata corresponding to 'FEI_HELIOS' is created but it is empty!
Can you help me in finding what I am doing wrongly?
I don't need to use tifffile, but I would prefer to use python rather than ImageJ because I have already several other python scripts and I would like to integrate this new one with the others.
Thanks a lot in advance!
toto
ps. I'm a frequent user of stackoverflow, but this is actually my first question!

In principle the approach is correct. However, tifffile parses the raw values of certain tags, including FEI_HELIOS, to dictionaries or other Python types. To get the raw tag value for rewriting, it needs to be read from file again. In these cases, use the internal TiffTag._astuple function to get an extratag compatible tuple of the tag, e.g.:
import tifffile
with tifffile.TiffFile('FEI_SEM.tif') as tif:
assert tif.is_fei
page = tif.pages[0]
image = page.asarray()
... # process image
with tifffile.TiffWriter('copy1.tif') as out:
out.write(
image,
photometric=page.photometric,
compression=page.compression,
planarconfig=page.planarconfig,
rowsperstrip=page.rowsperstrip,
resolution=(
page.tags['XResolution'].value,
page.tags['YResolution'].value,
page.tags['ResolutionUnit'].value,
),
extratags=[page.tags['FEI_HELIOS']._astuple()],
)
This approach does not preserve Exif metadata, which tifffile cannot write.
Another approach, since FEI files seem to be written uncompressed, is to directly memory map the image data in the file to a numpy array and manipulate that array:
import shutil
import tifffile
shutil.copyfile('FEI_SEM.tif', 'copy2.tif')
image = tifffile.memmap('copy2.tif')
... # process image
image.flush()
Finally, consider tifftools for rewriting TIFF files where tifffile is currently failing, e.g. Exif metadata.

pdf2image conversion of multi page PDFs to images returns the last page on all images

So when I use the pdf2image python import, and pass a multi page PDF into the convert_from_bytes()- or convert_from_path() method, the output array does contain multiple images - but all images are of the last PDF page (whereas I would've expected that each image represented one of the PDF pages).
The output looks something like this:
Any idea on why this would occur? I can't find any solution to this online. I've found some vague suggestion that the use_cropbox argument might be used, but modifying it has no effect.
def convert(opened_file)
# Read PDF and convert pages to PPM image objects
try:
_ppm_pages = self.pdf2image.convert_from_bytes(
opened_file.read(),
grayscale = True
)
except Exception as e:
print(f"[CreateJPEG] Could not convert PDF pages to JPEG image due to error: \n '{e}'")
return
# Do stuff with _ppm_pages
for img in _ppm_pages:
img.show() # ...all images in that list are of the last page
Sometimes the output is an empty 1x1 image, instead, which I also haven't found a reason for. So if you have any idea what that is about, please do let me know!
Thanks in advance,
Simon
EDIT: Added code.
EDIT: So, when I try this in a random notebook, it actually works fine.
I've removed a few detours I used in my original code, and now it works. Still not sure what the underlying reason was though...
All the same, thanks for your help, everyone!

I'm using this right now....
from pdf2image import convert_from_path
imgSet = convert_from_path(pathToPDF, 500)
That gives me a list of images within imgSet

I guess you have to do something like this as described in the unit tests of the package.
with open("./tests/test.pdf", "rb") as pdf_file:
images_from_bytes = convert_from_bytes(pdf_file.read(), fmt="jpg")
self.assertTrue(images_from_bytes[0].format == "JPEG")

Read binary data off Windows clipboard, in Blender (python)

EDIT: Figured THIS part out, but see 2nd post below for another question.
(a little backstory here, skip ahead for the TLDR :) )
I'm currently trying to write a few scripts for Blender to help improve the level creation workflow for a game that I play (Natural Selection 2). Currently, to move geometry from the level editor to Blender, I have to 1) Save a file from the editor as an .obj 2) import obj into blender, and make my changes. Then I 3) export to the game's level format using an exporter script I wrote, and 4) re-open the file in a new instance of the editor. 5) copy the level data from the new instance. 6) paste into the main level file. This is quite a pain to do, and quite clearly discourages even using the tool at all but for major edits. My idea for an improved workflow: 1) Copy data to clipboard in editor 2) Run importer script in Blender to load data. 3) Run exporter script in blender to save data. 4) Paste back into original file. This not only cuts out two whole steps in the tedious process, but also eliminates the need for extra files cluttering up my desktop. Currently though, I haven't found a way to read in clipboard data from the Windows clipboard into Blender... at least not without having to go through some really elaborate installation steps (eg install python 3.1, install pywin32, move x,y,z to the blender directory, uninstall python 3.1... etc...)
TLDR
I need help finding a way to write/read BINARY data to/from the clipboard in Blender. I'm not concerned about cross-platform capability -- the game tools are Windows only.
Ideally -- though obviously beggars can't be choosers here -- the solution would not make it too difficult to install the script for the layman. I'm (hopefully) not the only person who is going to be using this, so I'd like to keep the installation instructions as simple as possible. If there's a solution available in the python standard library, that'd be awesome!
Things I've looked at already/am looking at now
Pyperclip -- plaintext ONLY. I need to be able to read BINARY data off the clipboard.
pywin32 -- Kept getting missing DLL file errors, so I'm sure I'm doing something wrong. Need to take another stab at this, but the steps I had to take were pretty involved (see last sentence above TLDR section :) )
TKinter -- didn't read too far into this one as it seemed to only read plain-text.
ctypes -- actually just discovered this in the process of writing this post. Looks scary as hell, but I'll give it a shot.

Okay I finally got this working. Here's the code for those interested:
from ctypes import *
from binascii import hexlify
kernel32 = windll.kernel32
user32 = windll.user32
user32.OpenClipboard(0)
CF_SPARK = user32.RegisterClipboardFormatW("application/spark editor")
if user32.IsClipboardFormatAvailable(CF_SPARK):
data = user32.GetClipboardData(CF_SPARK)
size = kernel32.GlobalSize(data)
data_locked = kernel32.GlobalLock(data)
text = string_at(data_locked,size)
kernel32.GlobalUnlock(data)
else:
print('No spark data in clipboard!')
user32.CloseClipboard()

Welp... this is a new record for me (posting a question and almost immediately finding an answer).
For those interested, I found this: How do I read text from the (windows) clipboard from python?
It's exactly what I'm after... sort of. I used that code as a jumping-off point.
Instead of CF_TEXT = 1
I used CF_SPARK = user32.RegisterClipboardFormatW("application/spark editor")
Here's where I got that function name from: http://msdn.microsoft.com/en-us/library/windows/desktop/ms649049(v=vs.85).aspx
The 'W' is there because for whatever reason, Blender doesn't see the plain-old "RegisterClipboardFormat" function, you have to use "...FormatW" or "...FormatA". Not sure why that is. If somebody knows, I'd love to hear about it! :)
Anyways, haven't gotten it actually working yet: still need to find a way to break this "data" object up into bytes so I can actually work with it, but that shouldn't be too hard.
Scratch that, it's giving me quite a bit of difficulty.
Here's my code
from ctypes import *
from binascii import hexlify
kernel32 = windll.kernel32
user32 = windll.user32
user32.OpenClipboard(0)
CF_SPARK = user32.RegisterClipboardFormatW("application/spark editor")
if user32.IsClipboardFormatAvailable(CF_SPARK):
data = user32.GetClipboardData(CF_SPARK)
data_locked = kernel32.GlobalLock(data)
print(data_locked)
text = c_char_p(data_locked)
print(text)
print(hexlify(text))
kernel32.GlobalUnlock(data_locked)
else:
print('No spark data in clipboard!')
user32.CloseClipboard()
There aren't any errors, but the output is wrong. The line print(hexlify(text)) yields b'e0cb0c1100000000', when I should be getting something that's 946 bytes long, the first 4 of which should be 01 00 00 00. (Here's the clipboard data, saved out from InsideClipboard as a .bin file: https://www.dropbox.com/s/bf8yhi1h5z5xvzv/testLevel.bin?dl=1 )

Where does Python Imaging LIbrary Save Objects

I am using the Image.save method from PIL and I cannot find where the file is being placed. I have done a system search yet still no luck.
My code looks like this:
print imageObj.save(fileName, "JPEG")
and gives the proper None response to say that it is working. Any idea where they go and how I can find them?
Thanks!

You should pick a location to save the image when setting the filename variable.
filename = "/Users/clifgray/Desktop/filename.jpeg"
imgObj.save(filename)