I want to filter an image with a simple convolution kernel in python-pillow. However, to achieve optimal results, I need a 9x9 kernel. This is not possible in pillow, at least when using ImageFilter.Kernel and the built-in filter() method, which are limited to 5x5 kernels.
Short of implementing my own convolution code, is there a way to filter/convolve an image with a kernel size larger than 5x5?
I'm quite surprised to see that PIL doesn't have support beyond 5 x 5 kernels. As such, it may be prudent to look at other Python packages, such as OpenCV or scipy... for the interest of saving time, let's use scipy. OpenCV is a pain to configure even though it's quite powerful.
I would recommend using scipy to load in your image with imread from the ndimage package, convolve the image with your kernel, then convert to a PIL image when you're done. Use convolve from the ndimage package, then convert back to a PIL image by Image.fromArray. It does have support to convert a numpy.ndarray (which is what is loaded in when you use scipy.ndimage.imread), which is great.
Something like this, assuming a 9 x 9 averaging filter:
# Import relevant packages
import numpy as np
from scipy import ndimage
from PIL import Image
# Read in image - change filename to whatever you want
img = ndimage.imread('image.jpg')
# Create kernel
ker = (1/81.0)*np.ones((9,9))
# Convolve
out = ndimage.convolve(img, ker)
# Convert back to PIL image
out = Image.fromArray(out, 'RGB')
pyvips is another option, if you're not tied to pillow, numpy or scipy. It's quite a bit faster and needs a lot less memory, especially for larger images. It'll beat opencv too, at least on some benchmarks.
I tried on this laptop:
import sys
import numpy as np
from scipy import ndimage
from PIL import Image
img = ndimage.imread(sys.argv[1])
ker = (1 / 81.0) * np.ones((9, 9))
out = ndimage.convolve(img, ker)
out = Image.fromarray(out)
out.save(sys.argv[2])
I can run it like this:
$ /usr/bin/time -f %M:%e ./try257.py ~/pics/wtc-mono.jpg x.jpg
300352:22.47
So a 10k x 10k pixel mono jpg on a 2015 i5 laptop takes about 22 seconds and needs a peak of 300mb of memory.
In pyvips it's:
import sys
import pyvips
im = pyvips.Image.new_from_file(sys.argv[1], access="sequential")
size = 9
kernel = size * [size * [1.0 / (size * size)]]
im = im.conv(kernel)
im.write_to_file(sys.argv[2])
I see:
$ /usr/bin/time -f %M:%e ./try258.py ~/pics/wtc-mono.jpg x.jpg
44336:4.76
About 5 seconds and 45mb of memory.
That's a float convolution. You can swap it to int precision like this:
im = im.conv(kernel, precision="integer")
And I see:
$ /usr/bin/time -f %M:%e ./try258.py ~/pics/wtc-mono.jpg x.jpg
44888:1.79
1.8 seconds.
Related
I have a program that processes huge RGB images in the range of 30000x30000 px.
To load I use Pillow, which works good.
Then I process it with NumPy and then I need to save it lossless as tiff.
However, whether I'm using Pillow or OpenCV, this takes very long compared to the runtime of all the other stuff. I think this is because of the image compression. Without compression, the saving does not take long at all but my files are >2 GB.
I found the module tifffile but it takes just as long as OpenCV, unless I missed a parameter.
Is there a module that can compress faster? The ones I tried only use one CPU core.
It also seems, that it's faster on an intel machine with i7-9700k 16GB than on my PC with AMD Ryzen 5600X 32GB?
Here is the code I used to test:
from PIL import Image
import cv2
import tifffile
import numpy as np
import time
arr = np.random.default_rng().integers(0, 255, size=(30000,30000,3), endpoint=True, dtype=np.uint8)
st = time.time()
Image.fromarray(arr).save("test_pil.tiff", compression="tiff_adobe_deflate")
print(f"Pil took {time.time()-st} s")
st = time.time()
cv2.imwrite("test_cv2.tiff", arr, params=(cv2.IMWRITE_TIFF_COMPRESSION, 32946))
print(f"Opencv took {time.time()-st} s")
st = time.time()
tifffile.imwrite("test_tifff.tiff", arr, compression="zlib", compressionargs={'level':5}, predictor=True, tile=(64,64))
print(f"Tifffile took {time.time()-st} s")
I know these also use different compression algorithms, but I haven't found matching parameters. This feature is generally very poorly documented.
Result (intel):
Pil took 32.01173210144043 s
Opencv took 60.46461296081543 s
Tifffile took 59.410102128982544 s
I want to convert .nii images to .tif to train my model using U-Net.
1-I looped through all images in the folder.
2-I looped through all slices within each image.
3-I saved each slice as .tif.
The training images are converted successfully. However, the labels (masks) are all saved as black images. I want to successfully convert those masks from .nii to .tif, but I don't know how. I read that it could be something with brightness, but I didn't get the idea clearly, so I couldn't solve the problem until now.
The only reason for this conversion is to be able to train my model. Feel free to suggest a better idea, if anyone can share a way to feed the network with the .nii format directly.
import nibabel as nib
import matplotlib.pyplot as plt
import imageio
import numpy as np
import glob
import os
import nibabel as nib
import numpy as np
from tifffile import imsave
import tifffile as tiff
for filepath in glob.iglob('data/Task04_Hippocampus/labelsTr/*.nii.gz'):
a = nib.load(filepath).get_fdata()
a = a.astype('int8')
base = Path(filepath).stem
base = re.sub('.nii', '', base)
x,y,z = a.shape
for i in range(0,z):
newimage = a[:, :, i]
imageio.imwrite('data/Task04_Hippocampus/masks/'+base+'_'+str(i)+'.tif', newimage)
Unless you absolutely have to use TIFF, I would strongly suggest using the NiFTI format for a number of important reasons:
Image values are often not arbitrary. For example, in CT images the values correspond to x-ray attenuation (check out this Wikipedia page). TIFF, which is likely to scale the values in some way, is not suitable for this.
NIfTI also contains a header which has crucial geometric information needed to correctly interpret the image, such as the resolution, slice thickness, and direction.
You can directly extract a numpy.ndarray from NIfTI images using SimpleITK. Here is a code snippet:
import SimpleITK as sitk
import numpy as np
img = sitk.ReadImage("your_image.nii")
arr = sitk.GetArrayFromImage(img)
slice_0 = arr[0,:,:] # this is a 2D axial slice as a np.ndarray
As an aside: the reason the images where you stored your masks look black is because in NIfTI format labels have a value of 1 (and background is 0). If you directly convert to TIFF, a value of 1 is very close to black when interpreted as an RGB value - another reason to avoid TIFF!
I'm using OpenCV version 4.1.1 in Python and cannot get a legitimate reading for a 32-bit image, even when I use cv.IMREAD_ANYDEPTH. Without cv.IMREAD_ANYDEPTH, it returns as None type; with it, I get a matrix of zeros. The issue persists after reinstalling OpenCV. os.path.isfile returns True. The error was replicated on another computer. The images open in ImageJ, so I wouldn't think they're corrupted. I would rather use Skimage since it reads the images just fine, but I have to use OpenCV for what I'm working on. Any advice is appreciated.
img = cv2.imread(file,cv2.IMREAD_ANYDEPTH)
Link for the image: https://drive.google.com/file/d/1IiHbemsmn2gLW12RG3i9fLYZQW2u8sQw/view?usp=sharing
It appears to be some bug in how OpenCV loads such TIFF images. Pillow seems to load the image in a sensible way. Running
from PIL import Image
import numpy as np
img_pil = Image.open('example_image.tiff')
img_pil_cv = np.array(img_pil)
print(img_pil_cv.dtype)
print(img_pil_cv.max())
I get
int32
40950
as an output, which looks reasonable enough.
When I do
import cv2
img_cv = cv2.imread('example_image.tiff', cv2.IMREAD_ANYDEPTH)
print(img_cv.dtype)
print(img_cv.max())
I get
float32
5.73832e-41
which is obviously wrong.
Nevertheless, the byte array holding the pixel data is correct, it's just not being interpreted correctly. You can use numpy.ndarray.view to reinterpret the datatype of a numpy array, so that it's treated as an array if 32bit integers instead.
img_cv = cv2.imread('example_image.tiff', cv2.IMREAD_ANYDEPTH)
img_cv = img_cv.view(np.int32)
print(img_cv.dtype)
print(img_cv.max())
Which prints out
int32
40950
Since the maximum value is small enough for 16bit integer, let's convert the array and see what it looks like
img_cv_16bit = img_cv.astype(np.uint16)
cv2.imwrite('output_cv_16bit.png', img_cv_16bit)
OK, there are some bright spots, and a barely visible pattern. With a little adjustment, we can get something visible:
img_cv_8bit = np.clip(img_cv_16bit // 16, 0, 255).astype(np.uint8)
cv2.imwrite('output_cv_8bit.png', img_cv_8bit)
That looks quite reasonable now.
I'm trying to display animations in Google Colab. Specifically, I would like to animate a numpy array with cv2, eg drawing lines in a frame-based manner, and show the output in the cell. The closest I got was this, you can try this code in Colab:
from google.colab.patches import cv2_imshow
import IPython
from PIL import Image
import numpy as np
import cv2 as cv
import time
# Create a black image
img = np.zeros((512,512,3), np.uint8)
# Draw a diagonal blue line with thickness of 5 px
cv.line(img,(0,0),(511,511),(255,0,0),5)
cv2_imshow(img)
for i in range(100):
cv.line(img,(i,0),(511,511),(255,0,0),5)
cv2_imshow(img)
IPython.display.clear_output(wait=True)
time.sleep(1/60.0)
At some point of course this should happen without time.sleep, but with repeated callbacks so we don't block any other code execution. However, as you can see, the output flickers and is not smooth at all.
Here are a couple things I've tried:
ipycanvas. This is great in a local Jupyter notebook and is based on HTML5 canvas. It is a bit annoying to get the image data from javascript back to python, but it's possible. However, this does not run in Google Colab.
https://ipycanvas.readthedocs.io/
Matplotlib animations. eg this (not mine):
https://colab.research.google.com/drive/1lnl5UPFWVPrryaZZgEzd0theI6S94c3X#scrollTo=QLRBwgFqdr83
This is alright. However, it renders the whole animation before displaying it, which is not what I want. Especially, I want to be able to add some interactivity to animations, which this limitation rules out (eg clicking in the image or some button to make something happen in the animation).
Some way of explicitly creating an HTML5 canvas in javascript, eg as suggested here:
IPython: Adding Javascript scripts to IPython notebook
However, I'd like all my code to be python, especially my data to be numpy arrays or PIL images.
Any suggestions?
Here's an example using ipywidgets.Image. This approach doesn't flicker like using clear_output, but the updates seem pretty slow. This might be to do with the fact we're running remotely from Colab - it has to send image updates over the net. Looks like I'm getting 2 or 3 per second, and it seems like it "batches up" or discards intervening updates, rather than waiting for each one.
It's pretty smooth running locally on regular Jupyter.
Hope someone can improve on this - it's something we want to do as well :)
import ipywidgets as ipw
from IPython import display
import numpy as np
import PIL
from io import BytesIO
import time
# image size
h,w = 200,300
# Make an Image Widget and display it
wIm = ipw.Image()
display.display(wIm)
# Make an RGBA array for the image
g3 = np.zeros((h,w,4), dtype=np.uint8)
g3[:,:,3] = 255 # opacity
g3[:,:,0:3] = 0 # color black
p = np.array([h//2,w//2], dtype=int)
for i in range(1000):
# Draw a coloured spiral
r = i/10
theta=i/20
p2 = p + r * np.array([ np.cos(theta), np.sin(theta) ])
(y,x) = p2.astype(int)
rgb = np.array([100+r, 100*(1+np.sin(theta)), 100*(1+np.cos(theta))], dtype=np.uint8)
g3[y:y+8, x:x+2, 0:3] = rgb
# convert numpy to PIL to png-format bytes
pilIm = PIL.Image.fromarray(g3, mode="RGBA")
with BytesIO() as fOut:
pilIm.save(fOut, format="png")
byPng = fOut.getvalue()
# set the png bytes as the image value;
# this updates the image in the browser.
wIm.value=byPng
time.sleep(1/60)
I am trying to load a data set of 1.000.000 images into memory. As standard numpy arrays (uint8) all images combined fill around 100 GB of RAM, but I need to get this down to < 50 GB while still being able to quickly read the images back into numpy (that's the whole point of keeping everything in memory). Lossless compression like blosc only reduces file size by around 10%, so I went to JPEG compression. Minimum example:
import io
from PIL import Image
numpy_array = (255 * np.random.rand(256, 256, 3)).astype(np.uint8)
image = Image.fromarray(numpy_array)
output = io.BytesIO()
image.save(output, format='JPEG')
At runtime I am reading the images with:
[np.array(Image.open(output)) for _ in range(1000)]
JPEG compression is very effective (< 10 GB), but the time it takes to read 1000 images back into numpy array is around 2.3 seconds, which seriously hurts the performance of my experiments. I am searching for suggestions that give a better trade-off between compression and read-speed.
I am still not certain I understand what you are trying to do, but I created some dummy images and did some tests as follows. I'll show how I did that in case other folks feel like trying other methods and want a data set.
First, I created 1,000 images using GNU Parallel and ImageMagick like this:
parallel convert -depth 8 -size 256x256 xc:red +noise random -fill white -gravity center -pointsize 72 -annotate 0 "{}" -alpha off s_{}.png ::: {0..999}
That gives me 1,000 images called s_0.png through s_999.png and image 663 looks like this:
Then I did what I think you are trying to do - though it is hard to tell from your code:
#!/usr/local/bin/python3
import io
import time
import numpy as np
from PIL import Image
# Create BytesIO object
output = io.BytesIO()
# Load all 1,000 images and write into BytesIO object
for i in range(1000):
name="s_{}.png".format(i)
print("Opening image: {}".format(name))
im = Image.open(name)
im.save(output, format='JPEG',quality=50)
nbytes = output.getbuffer().nbytes
print("BytesIO size: {}".format(nbytes))
# Read back images from BytesIO ito list
start=time.clock()
l=[np.array(Image.open(output)) for _ in range(1000)]
diff=time.clock()-start
print("Time: {}".format(diff))
And that takes 2.4 seconds to read all 1,000 images from the BytesIO object and turn them into numpy arrays.
Then, I palettised the images by reducing to 256 colours (which I agree is lossy - just as your method) and saved a list of palettised image objects which I can readily later convert back to numpy arrays by simply calling:
np.array(ImageList[i].convert('RGB'))
Storing the data as a palettised image saves 66% of the space because you only store one byte of palette index per pixel rather than 3 bytes of RGB, so it is better than the 50% compression you seek.
#!/usr/local/bin/python3
import io
import time
import numpy as np
from PIL import Image
# Empty list of images
ImageList = []
# Load all 1,000 images
for i in range(1000):
name="s_{}.png".format(i)
print("Opening image: {}".format(name))
im = Image.open(name)
# Add palettised image to list
ImageList.append(im.quantize(colors=256, method=2))
# Read back images into numpy arrays
start=time.clock()
l=[np.array(ImageList[i].convert('RGB')) for i in range(1000)]
diff=time.clock()-start
print("Time: {}".format(diff))
# Quick test
# Image.fromarray(l[999]).save("result.png")
That now takes 0.2s instead of 2.4s - let's hope the loss of colour accuracy is acceptable to your unstated application :-)