I'm trying to display animations in Google Colab. Specifically, I would like to animate a numpy array with cv2, eg drawing lines in a frame-based manner, and show the output in the cell. The closest I got was this, you can try this code in Colab:
from google.colab.patches import cv2_imshow
import IPython
from PIL import Image
import numpy as np
import cv2 as cv
import time
# Create a black image
img = np.zeros((512,512,3), np.uint8)
# Draw a diagonal blue line with thickness of 5 px
cv.line(img,(0,0),(511,511),(255,0,0),5)
cv2_imshow(img)
for i in range(100):
cv.line(img,(i,0),(511,511),(255,0,0),5)
cv2_imshow(img)
IPython.display.clear_output(wait=True)
time.sleep(1/60.0)
At some point of course this should happen without time.sleep, but with repeated callbacks so we don't block any other code execution. However, as you can see, the output flickers and is not smooth at all.
Here are a couple things I've tried:
ipycanvas. This is great in a local Jupyter notebook and is based on HTML5 canvas. It is a bit annoying to get the image data from javascript back to python, but it's possible. However, this does not run in Google Colab.
https://ipycanvas.readthedocs.io/
Matplotlib animations. eg this (not mine):
https://colab.research.google.com/drive/1lnl5UPFWVPrryaZZgEzd0theI6S94c3X#scrollTo=QLRBwgFqdr83
This is alright. However, it renders the whole animation before displaying it, which is not what I want. Especially, I want to be able to add some interactivity to animations, which this limitation rules out (eg clicking in the image or some button to make something happen in the animation).
Some way of explicitly creating an HTML5 canvas in javascript, eg as suggested here:
IPython: Adding Javascript scripts to IPython notebook
However, I'd like all my code to be python, especially my data to be numpy arrays or PIL images.
Any suggestions?
Here's an example using ipywidgets.Image. This approach doesn't flicker like using clear_output, but the updates seem pretty slow. This might be to do with the fact we're running remotely from Colab - it has to send image updates over the net. Looks like I'm getting 2 or 3 per second, and it seems like it "batches up" or discards intervening updates, rather than waiting for each one.
It's pretty smooth running locally on regular Jupyter.
Hope someone can improve on this - it's something we want to do as well :)
import ipywidgets as ipw
from IPython import display
import numpy as np
import PIL
from io import BytesIO
import time
# image size
h,w = 200,300
# Make an Image Widget and display it
wIm = ipw.Image()
display.display(wIm)
# Make an RGBA array for the image
g3 = np.zeros((h,w,4), dtype=np.uint8)
g3[:,:,3] = 255 # opacity
g3[:,:,0:3] = 0 # color black
p = np.array([h//2,w//2], dtype=int)
for i in range(1000):
# Draw a coloured spiral
r = i/10
theta=i/20
p2 = p + r * np.array([ np.cos(theta), np.sin(theta) ])
(y,x) = p2.astype(int)
rgb = np.array([100+r, 100*(1+np.sin(theta)), 100*(1+np.cos(theta))], dtype=np.uint8)
g3[y:y+8, x:x+2, 0:3] = rgb
# convert numpy to PIL to png-format bytes
pilIm = PIL.Image.fromarray(g3, mode="RGBA")
with BytesIO() as fOut:
pilIm.save(fOut, format="png")
byPng = fOut.getvalue()
# set the png bytes as the image value;
# this updates the image in the browser.
wIm.value=byPng
time.sleep(1/60)
Related
I am working with this data analysis pipeline that makes gifs of video recordings, the function used for this is:
def make_gif(self, datafile, save_file, frame_limit:int=20, fps:int=10, verbose=True):
h5 = h5py.File(datafile, "r")
frames = h5['frames'][::2][:frame_limit]
imageio.mimwrite(save_file, frames, fps=fps)
if verbose:
print(f"Saved gif version: fps={fps}, nframes={frame_limit}", flush=True)
The only necessary imports for this are h5py and imageio.
I'm needing to append some text to these gifs. There's some metadata we need displayed for quick reading. For example, I have a stack of frames that look like this image:
Image 1 - no text
But what I need is something like this:
Image 2 - with text
How would I go about doing that with Python and imageio? I should note that I cannot save the individual images as jpgs for reuploading later, I need to create the gifs as part of the pipeline.
Our focus with ImageIO is on the IO side of images. If you want to add text to an image or perform any other processing of the image, you will want to bring in an image processing library. Common choices here are scikit-image or opencv.
Here is an example of how you can do this using cv2. Note that I am using a standard image here for better reproducibility, but the same logic works with HDF5 and other video/image formats.
import imageio.v3 as iio
import cv2
frames = iio.imread("imageio:newtonscradle.gif") # example image/frames
# add the text
for frame in frames:
foo = cv2.putText(
frame,
"Hey look some metadata",
(5, 25),
cv2.FONT_HERSHEY_SIMPLEX,
.4,
(0, 0, 0)
)
# write the output
iio.imwrite("annotated.gif", frames, loop=0)
Output:
Weirdly, scikit-image doesn't allow rendering text onto an image, but there is an age-old issue to track that feature here.
Alternatively, if visualization is what you are after, you could use matplotlib. This comes with the advantage of giving you all the power of the MPL but comes with the drawback of losing control over individual pixels. This is not ideal for scientific processing (where pixel values matter), but great for quick annotations, human consumption, and qualitative data.
Here is an example how you could recreate the above:
import imageio.v3 as iio
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.figure import figaspect
frames = iio.imread("imageio:newtonscradle.gif")
aspect_ratio = figaspect(frames[0, ..., 0])
annotated_frames = list()
for frame in frames:
fig, ax = plt.subplots(figsize=aspect_ratio, dpi=50)
ax.imshow(frame)
ax.text(5, 5, "Hey look some metadata", fontsize="xx-large", va="top")
ax.set_axis_off()
ax.set_position([0, 0, 1, 1])
fig.canvas.draw()
annotated_frames.append(np.asarray(fig.canvas.renderer.buffer_rgba()))
plt.close(fig)
iio.imwrite("annotated.gif", annotated_frames, loop=0)
Output:
Probably an unusual question, but I am currently looking for a solution to display image files with PIL slower.
Ideally so that you can see how the image builds up, pixel by pixel from left to right.
Does anyone have an idea how to implement something like this?
It is a purely optical thing, so it is not essential.
Here an example:
from PIL import Image
im = Image.open("sample-image.png")
im.show()
Is there a way to "slow down" im.show()?
AFAIK, you cannot do this directly with PIL's Image.show() because it actually saves your image as a file to /var/tmp/XXX and then passes that file to your OS's standard image viewer to display on the screen and there is no further interaction with the viewer process after that. So, if you draw in another pixel, the viewer will not be aware and if you call Image.show() again, it will save a new copy of your image and invoke another viewer which will give you a second window rather than updating the first!
There are several possibilities to get around it:
use OpenCV's cv2.imshow() which does allow updates
use tkinter to display the changing image
create an animated GIF and start a new process to display that
I chose the first, using OpenCV, as the path of least resistance:
#!/usr/bin/env python3
import cv2
import numpy as np
from PIL import Image
# Open image
im = Image.open('paddington.png')
# Make BGR Numpy version for OpenCV
BGR = np.array(im)[:,:,::-1]
h, w = BGR.shape[:2]
# Make empty image to fill in slowly and display
d = np.zeros_like(BGR)
# Use "x" to avoid drawing and waiting for every single pixel
x=0
for y in range(h):
for x in range(w):
d[y,x] = BGR[y,x]
if x%400==0:
cv2.imshow("SlowLoader",d)
cv2.waitKey(1)
x += 1
# Wait for one final keypress to exit
cv2.waitKey(0)
Increase the 400 near the end to make it faster and update the screen after a greater number of pixels, or decrease it to make it update the screen after a smaller number of pixels meaning you will see them appear more slowly.
As I cannot share a movie on StackOverflow, I made an animated GIF to show how that looks:
I decided to try and do it with tkinter as well. I am no expert on tkinter but the following works just the same as the code above. If anyone knows tkinter better, please feel free to point out my inadequacies - I am happy to learn! Thank you.
#!/usr/bin/env python3
import numpy as np
from tkinter import *
from PIL import Image, ImageTk
# Create Tkinter Window and Label
root = Tk()
video = Label(root)
video.pack()
# Open image
im = Image.open('paddington.png')
# Make Numpy version for simpler pixel access
RGB = np.array(im)
h, w = RGB.shape[:2]
# Make empty image to fill in slowly and display
d = np.zeros_like(RGB)
# Use "x" to avoid drawing and waiting for every single pixel
x=0
for y in range(h):
for x in range(w):
d[y,x] = RGB[y,x]
if x%400==0:
# Convert the video for Tkinter
img = Image.fromarray(d)
imgtk = ImageTk.PhotoImage(image=img)
# Set the image on the label
video.config(image=imgtk)
# Update the window
root.update()
x += 1
I am trying to create a pipeline in which I first render an image using the blender python API (I am using Blender 2.90) and then perform some image processing in python. I want to fetch the image directly from blender without first writing the rendered image to disk and then loading it again. I ran the following code within the blender GUI to do so:
import bpy
import numpy as np
import PIL.Image as Image
from skimage.util import img_as_ubyte
resolution_x = 512
resolution_y = 512
# render settings
scene = bpy.context.scene
scene.render.engine = 'BLENDER_EEVEE'
scene.render.resolution_x = resolution_x
scene.render.resolution_y = resolution_y
scene.render.image_settings.file_format = 'PNG'
scene.render.filepath = "path/to/good_image.png"
# create Viewer Layer in Compositor
scene.use_nodes = True
tree = scene.node_tree
nodes = tree.nodes
links = tree.links
for node in nodes:
nodes.remove(node)
render_layer_node = nodes.new('CompositorNodeRLayers')
viewer_node = nodes.new('CompositorNodeViewer')
links.new(viewer_node.inputs[0], render_layer_node.outputs[0])
# render scene and get pixels from Viewer Node
bpy.ops.render.render(write_still=True)
pixels = bpy.data.images['Viewer Node'].pixels
# do some processing and save
img = np.flip(img_as_ubyte(np.array(pixels[:]).reshape((resolution_y, resolution_x, 4))), axis=0)
Image.fromarray(img).save("path/to/bad_image.png")
Problem: The image I get from the Viewer Node is much darker (bad image) than the image saved in the conventional way (good image). Does anyone have an idea why this happens and how to fix it? Does blender maybe treat pixel values differently than I expect?
Some additional information:
Before conversion to uint8, the values of the alpha channel within the dark image are 1.0 (as they actually should be). Background values in the dark image are not 0.0 or negative (as one might guess from appearance), but 0.05...
What I tried:
I thought that pixels might be scaled within range -1 to 1, so I rescaled the pixels to range 0 to 1 before transforming to uint8... Did not lead to the correct image either :(
It's because the image that you get from the Viewer Node is the one "straight from compositing" before color management takes place. You can have a look at the documentation here: this image is still in the linear space.
Your good_image.png on the other hand is obtained after transformation into the "Display Space" (see diagram in the doc). Hence it was transformed into a log-space, maybe gamma-corrected, etc.
Finally, you can get an image that is close to (but slightly different though) to the good image from the viewer node by calling bpy.data.images['Viewer Node'].save_render(filepath) instead, but there is no way to directly extract the color-managed version without rendering to a file first. You can probably do it yourself by adding PyOpenColorIO to your script and applying the color management from this module.
I'm looking for a library that enables to "create pictures" (or even videos) with the following functions:
Accepting picture inputs
Resizing said inputs to fit given template / scheme
Positioning the pictures in pre-set up layers or coordinates
A rather schematic approach to look at this:
whereas the red spots are supposed to represent e.g. text, picture (or if possible video) elements.
The end goal would be to give the .py script multiple input pictures and the .py creating a finished version like mentioned above.
Solutions I tried were looking into Python PIL, but I wasn't able to find what I was looking for.
Yes, it is possible to do this with Python.
The library you are looking for is OpenCV([https://opencv.org][1]/).
Some basic OpenCV python tutorials (https://docs.opencv.org/master/d9/df8/tutorial_root.html).
1) You can use imread() function to read images from files.
2) You can use resize() function to resize the images.
3) You can create a empty master numpy array matching the size and depth(color depth) of the black rectangle in the figure you have shown, resize your image and copy the contents into the empty array starting from the position you want.
Below is a sample code which does something close to what you might need, you can modify this to suit your actual needs. (Since your requirements are not clear I have written the code like this so that it can at least guide you.)
import numpy as np
import cv2
import matplotlib.pyplot as plt
# You can store most of these values in another file and load them.
# You can modify this to set the dimensions of the background image.
BG_IMAGE_WIDTH = 100
BG_IMAGE_HEIGHT = 100
BG_IMAGE_COLOR_DEPTH = 3
# This will act as the black bounding box you have shown in your figure.
# You can also load another image instead of creating empty background image.
empty_background_image = np.zeros(
(BG_IMAGE_HEIGHT, BG_IMAGE_WIDTH, BG_IMAGE_COLOR_DEPTH),
dtype=np.int
)
# Loading an image.
# This will be copied later into one of those red boxes you have shown.
IMAGE_PATH = "./image1.jpg"
foreground_image = cv2.imread(IMAGE_PATH)
# Setting the resize target and top left position with respect to bg image.
X_POS = 4
Y_POS = 10
RESIZE_TARGET_WIDTH = 30
RESIZE_TARGET_HEIGHT = 30
# Resizing
foreground_image= cv2.resize(
src=foreground_image,
dsize=(RESIZE_TARGET_WIDTH, RESIZE_TARGET_HEIGHT),
)
# Copying this into background image
empty_background_image[
Y_POS: Y_POS + RESIZE_TARGET_HEIGHT,
X_POS: X_POS + RESIZE_TARGET_WIDTH
] = foreground_image
plt.imshow(empty_background_image)
plt.show()
I'm learning some very basic steganography using python.
So far, I've been opening the file using:
from matplotlib import pyplot as plt
import numpy as np
file_location = '...'.png
rgb_data = np.array(plt.imread(file_location))
...
manually or otherwise edit some RGB values
...
plt.axis('off')
plt.imshow(rgb_data)
plt.savefig('image.jpg', dpi=96)
Note: In the above code, I've left out the particular edits I've been doing to the image. This is because I don't think they are part of my issue. Even if I make no changes at all, and just load then resave the image, I still encounter the issues below.
I've encountered three issues trying to save images using matplotlib in this way.
1. Getting the dpi to match
I'm not certain how to get the dpi of the output image to automatically match the dpi of the read image.
2. Getting the resolution to match
The resolutions of the input and output image by default don't match. Even if I manually match the dpi of the two images, they still don't have matching resolutions.
3. Getting the file sizes to match
My end goal is to produce an image that has the same dpi, resolution, and file size as the original. That way, when I start playing around with the RGB values, it should superficially appear to be the exact same image.
My question is how to save the file so the dpi and resolution (and presumably by extension the size...) of the output image match the input?
Of course, the number of pixels in the image, along with the dpi, should fix the resolution. However, it appears as though the output image is saved with a white border surrounding it, which is throwing off the resolution too.
Solutions using any python library are appreciated. However, edits to the existing code are preferable. Since I use matplotlib a lot, it would be useful to know how to circumvent this problem in the future too. Your help would be really appreciated!
plt.save(...) will save a Matplotlib figure. If you want to use only Matplotlib, please refer this post.
from matplotlib import pyplot as plt
import numpy as np
file_location = '...'.png
rgb_data = np.array(plt.imread(file_location))
# modify your image
fig = plt.figure(frameon=False)
h, w, _ = rgb_data.shape
fig.set_size_inches(w, h)
ax = plt.Axes(fig, [0., 0., 1., 1.])
ax.set_axis_off()
fig.add_axes(ax)
ax.imshow(rgb_data, aspect='auto')
path_to_save = "..."
fig.savefig(path_to_save)
The output image should match the input image.
But I think it will be better to save your image by using pillow.
import PIL.Image as Image
import numpy as np
file_location = "img.png"
path_to_save = "out.png"
img = Image.open(file_location)
rgb_data = np.array(img)
# modify your image
rgb_data = Image.fromarray(rgb_data)
rgb_data.save(path_to_save, format="PNG")
# you can also specify dpi, quality and etc.
# for example rgb_data.save(path_to_save, format="PNG", dpi=(300,300))
(Note: it is possible that due to compression settings of the input image size, the output size will be different, refer to this post)