We Keep Coding

Strange behavior in pyvips, impossible to write some images

I'm currently trying to make pyvips work for a project where I need to manipulate images of a "big but still sensible" size, between 1920x1080 and 40000x40000.
The installation worked well, but these particular 2 lines sometime work, sometime don't.
img = pyvips.Image.new_from_file('global_maps/MapBigBig.png')
img.write_to_file('global_maps/MapTest.png')
It seems that for the biggest images, I get the following error message when writing back the image (the loading works fine):
pyvips.error.Error: unable to call VipsForeignSavePngFile
pngload: arithmetic overflow
vips2png: unable to write to target global_maps/MapFishermansRowHexTest.png
I say it seems, because the following lines work perfectly well (with a size of 100 000 x 100 000, far bigger than the problematic images):
size = 100000
test = pyvips.Image.black(size, size, bands=3)
test.write_to_file('global_maps/Test.png')
I could not find an answer anywhere, do you have any idea what I'm doing wrong ?
EDIT:
Here is a link to an image that does not work (it weights 102 Mo).
This image was created using pyvips and a 40 time smaller image, this way:
img = pyvips.Image.new_from_file('global_maps/MapNormal.png')
out = img.resize(40, kernel='linear')
out.write_to_file('global_maps/MapBigBig.png')
And it can be read using paint3D or gimp.

I found your error message in libspng:
https://github.com/randy408/libspng/blob/master/spng/spng.c#L5989
It looks like it's being triggered if the decompressed image size would go over your process pointer size. If I try a 32-bit libvips on Windows I see:
$ ~/w32/vips-dev-8.12/bin/vips.exe copy MapFishermansRowHexBigBig.png x2.png
pngload: arithmetic overflow
vips2png: unable to write to target x2.png
But a 64-bit libvips on Windows works fine:
$ ~/vips-dev-8.12/bin/vips.exe copy MapFishermansRowHexBigBig.png x.png
$
So I think switching to a 64-bit libvips build would probably fix your problem. You'll need a 64-bit python too, of course.
I also think this is probably a libspng bug (or misfeature?) since you can read >4gb images on a 32-bit machine as long as you don't try to read them all in one go (libvips reads in chunks, so it should be fine). I'll open an issue on the libspng repo.

Using the opencv stitcher module to stitch blurry images

I am using python 3.5 and opencv 3.4.1.
I have a set of 19 images that I need to stitch. They are blurry and the stitching module is unable to stitch them. I have read this post, but am wondering if I can find a way to stitch. I would appreciate some specific suggestions and solutions. I want to stitch these images.
I have tried changing the match_conf as reccomended by this post. How would I edit this as the source code states that it is a flag? I have tried using the line below to change the match_conf to 0.1, but it does not work and I get the error below.
stitcher = cv2.createStitcher(False)
stitcher.setFeaturesMatcher(detail = BestOf2NearestMatcher(false, 0.1))
result = np.empty(shape=[2048, 2048])
ret, result = stitcher.stitch(imgs, result)
'cv2.Stitcher' object has no attribute 'setFeaturesMatcher'

Check this post out.
Possibly gig into pipeline and change opencv C++ code.
"This is full pipleline of OPencv Stitching code. You can see that there are lot of parameters you can change to make your code give some good stitching result. Also I would suggest using a small image (640 X480) for the feature detection step. Using small images is better than using very large images"
Technically you should be able to change the parameter from python, but hopeful somebody else knows how to do that.s

Python: Manipulating a 16-bit .tiff image in PIL &/or pygame: convert to 8-bit somehow?

Hello all,
I am working on a program which determines the average colony size of yeast from a photograph, and it is working fine with the .bmp images I tested it on. The program uses pygame, and might use PIL later.
However, the camera/software combo we use in my lab will only save 16-bit grayscale tiff's, and pygame does not seem to be able to recognize 16-bit tiff's, only 8-bit. I have been reading up for the last few hours on easy ways around this, but even the Python Imaging Library does not seem to be able to work with 16-bit .tiff's, I've tried and I get "IOError: cannot identify image file".
import Image
img = Image.open("01 WT mm.tif")
My ultimate goal is to have this program be user-friendly and easy to install, so I'm trying to avoid adding additional modules or requiring people to install ImageMagick or something.
Does anyone know a simple workaround to this problem using freeware or pure python? I don't know too much about images: bit-depth manipulation is out of my scope. But I am fairly sure that I don't need all 16 bits, and that probably only around 8 actually have real data anyway. In fact, I once used ImageMagick to try to convert them, and this resulted in an all-white image: I've since read that I should use the command "-auto-levels" because the data does not actually encompass the 16-bit range.
I greatly appreciate your help, and apologize for my lack of knowledge.
P.S.: Does anyone have any tips on how to make my Python program easy for non-programmers to install? Is there a way, for example, to somehow bundle it with Python and pygame so it's only one install? Can this be done for both Windows and Mac? Thank you.
EDIT: I tried to open it in GIMP, and got 3 errors:
1) Incorrect count for field "DateTime" (27, expecting 20); tag trimmed
2) Sorry, can not handle images with 12-bit samples
3) Unsupported layout, no RGBA loader
What does this mean and how do I fit it?

py2exe is the way to go for packaging up your application if you are on a windows system.
Regarding the 16bit tiff issue:
This example http://ubuntuforums.org/showthread.php?t=1483265 shows how to convert for display using PIL.
Now for the unasked portion question: When doing image analysis, you want to maintain the highest dynamic range possible for as long as possible in your image manipulations - you lose less information that way. As you may or may not be aware, PIL provides you with many filters/transforms that would allow you enhance the contrast of an image, even out light levels, or perform edge detection. A future direction you might want to consider is displaying the original image (scaled to 8 bit of course) along side a scaled image that has been processed for edge detection.
Check out http://code.google.com/p/pyimp/wiki/screenshots for some more examples and sample code.

I would look at pylibtiff, which has a pure python tiff reader.
For bundling, your best bet is probably py2exe and py2app.

This is actually a 2 part question:
1) 16 bit image data mangling for Python - I usually use GDAL + Numpy. This might be a bit too much for your requirements, you can use PIL + Numpy instead.
2) Release engineering Python apps can get messy. Depending on how complex your app is you can get away with py2deb, py2app and py2exe. Learning distutils will help too.

Reading Alpha of a PNG Pixel. Fast way via pure Python?

I am having an issue with an embedded 64bit Python instance not liking PIL. Before i start exhausting more methods to get a compiled image editor to read the pixels for me (such as ImageMagick) i am hoping perhaps anyone here can think of a purely Python solution that will be comparable in speeds to the compiled counterparts.
Now i am aware that the compiled friends will always be much faster, but i am hoping that because i "just" want to read the alpha of a group of pixels, that perhaps a fast enough pure Python solution can be conjured up. Anyone have any bright ideas?
Though, i have tried PyPNG and that is far too slow, so i'm not expecting any magic solutions. None the less, i had to ask.
Thanks to any replies!
And just for reference, the images i'll be reading will be on average around 512*512 to 2048*2048, and i'll be reading anywhere from one to all of the pixels alpha (multiplied by a few million times, but the values can be stored so reading twice isn't done).

Getting data out of a PNG requires unpacking data and decompressing it. These are likely going to be too slow in Python for your application. One possibility is to start with PyPNG and get rid of anything in it that you don't need. For example, it is probably storing all of the data it reads from the PNG, and some of the slow speed you see may be due to the memory allocations.

When you say PyPNG is too slow, how slow is it? To put it another way, how fast would be fast enough? PyPNG doesn't do anything stupid to make itself slow, but it is written in Python.
Make sure you're using read() to read the image row by row, and make sure you're using row[3::4] to extract the alpha channel. Extracting the alpha channel by using slice notation is no slower than reading the whole image.
I've added some notes to the PyPNG documentation about its speed.

How to scale an image without occasionally inverting it (with the Python Imaging Library)

When resizing images along the lines shown in this question occasionally the resulting image is inverted. About 1% of the images I resize are inverted, the rest is fine. So far I was unable to find out what is different about these images.
See resized example and original image for examples.
Any suggestions on how to track down that problem?

I was finally able to find someone experienced in JPEG and with some additional knowledge was able to find a solution.
JPEG is a very underspecified
Format.
The second image is a valid JPEG but it is in CMYK color space, not in RGB color space.
Design minded tools (read: things from Apple) can process CMYK JPEGs, other stuff (Firefox, IE) can't.
CMYK JPEG is very under specified and the way Adobe Photoshop writes it to disk is borderline to buggy.
Best of it all there is a patch to fix the issue.

Your original image won't display for me; Firefox says
The image “http://images.hudora.de/o/NIRV2MRR3XJGR52JATL6BOVMQMFSV54I01.jpeg”
cannot be displayed, because it contains errors.
This suggests that the problem arises when you attempt to resize a corrupted JPEG, and indeed your resized example shows what looks like JPEG corruption to my eye (Ever cracked open a JPEG image and twiddled a few bits to see what it does to the output? I have, and a few of my abominable creations looked like that). There are a few JPEG repair tools out there, but I've never seriously tried any of them and don't know if they might be able to help you out.