Jupyter notebook: let a user inputs a drawing - python

Simple question but I'm not able to find something out there...
Is there a simple and user friendly tool that can be used within a jupyter-notebook to let the user draw something in black on a white space (let say of size (x,y) pixels) after running a cell?
The drawing has to be returned (or even temporarily saved) as an array/image which can then be used by numpy for example.

you can do that using PIL and tkinter libraries, like:
from PIL import ImageTk, Image, ImageDraw
import PIL
from tkinter import *
width = 200 # canvas width
height = 200 # canvas height
center = height//2
white = (255, 255, 255) # canvas back
def save():
# save image to hard drive
filename = "user_input.jpg"
output_image.save(filename)
def paint(event):
x1, y1 = (event.x - 1), (event.y - 1)
x2, y2 = (event.x + 1), (event.y + 1)
canvas.create_oval(x1, y1, x2, y2, fill="black",width=5)
draw.line([x1, y1, x2, y2],fill="black",width=5)
master = Tk()
# create a tkinter canvas to draw on
canvas = Canvas(master, width=width, height=height, bg='white')
canvas.pack()
# create an empty PIL image and draw object to draw on
output_image = PIL.Image.new("RGB", (width, height), white)
draw = ImageDraw.Draw(output_image)
canvas.pack(expand=YES, fill=BOTH)
canvas.bind("<B1-Motion>", paint)
# add a button to save the image
button=Button(text="save",command=save)
button.pack()
master.mainloop()
You can modify the save function to read the image using PIL and numpy to have it as an numpy array.
hope this helps!

Try ipycanvas with interactive drawing mode.
It actually has a demo notebook with interactive drawing that could be easily modified to do exactly what you want. It also has numpy support.

There is a discussion on jupyterlab's github page on this issue: https://github.com/jupyterlab/jupyterlab/issues/9194. Apparently, it is planned to add Excalidraw at some point and until then, https://github.com/nicknytko/notebook-drawing was recommended.

Related

How would I use PIL to "extend" an image, and then draw a black rectangle with text on it?

Basically, what I want to do with Pillow is:
I want to get an image, and then extend the size of the image from the bottom so I'm able to fit a black rectangle with a four digit code on it. How would I do this? I tried to, but my text ended up being, for some reason, extremely small and unreadable and my rectangle wasn't perfect.
If it makes it easier, here's my image: https://i.stack.imgur.com/o9eYr.jpg
And here's what I want the end result to be: https://i.stack.imgur.com/GZ4uu.jpg (take a look at the bottom of the image)
I would suggest ImageOps.expand to expand your canvas:
#!/usr/bin/env python3
from PIL import Image, ImageDraw, ImageFont, ImageOps
# Load image
im = Image.open('o9eYr.jpg')
# Define font size, and annotation and height of padding above and below annotation
fontSize = 130
annotation = "GVVL"
padding = 20
# Load font and work out size of annotation
font = ImageFont.truetype("/System/Library/Fonts//Menlo.ttc", fontSize)
tw, th = font.getsize(annotation)
# Extend image at bottom and get height and width of new canvas
extended = ImageOps.expand(im, border=(0,0,0,th+2*padding), fill=(0,0,0))
w, h = extended.size
# Get drawing context and annotate
draw = ImageDraw.Draw(extended)
draw.text(((w-tw)//2, h-th-padding), annotation,(255,255,255),font=font)
extended.save('result.jpg')
You could create a new black image, paste the desired image and add text.
from PIL import Image, ImageFont, ImageDraw
base_img = Image.open('tmp.jpg')
base_size = base_img.size
new_size = (base_size[0], base_size[1] + 150)
img = Image.new("RGB", new_size)
img.paste(base_img, (0, 0))
draw = ImageDraw.Draw(img)
font = ImageFont.truetype("microsoftsansserif.ttf", 145) # (<font-file>, <font-size>)d
draw.text((base_size[0] // 2 - 150, base_size[1]),"GVVL",(255,255,255),font=font) # (x, y),"text",(r,g,b)
img.save('out.jpg')
Result:

Reconciling pixel coordinates on tkinter canvas and PIL image (Python)

I have a form (in .png format) with blank text boxes (usually filled out by hand). I would like to populate the boxes with text.
To do this, I am using tkinter to display the form on the screen, I then use the mouse (with finer positioning using arrow keys) to get the pixel coordinates of a box, then use PIL to write text to that box. A working example is below.
My primary issue is I am struggling to align pixel coordinates in the tkinter canvas and pixel coordinates in the PIL image.
Some additional background. The image is in high resolution and is circa 4961 by 7016 pixels. My screen resolution is 1920 x 1080. I had problems writing text where I needed it to write like this and found greater success if I scaled the image to fit entirely within my screen. I can only assume this is because I was / am confusing screen pixels with picture pixels (and resolved this when I fit the image to the screen to align these two - but understanding the differences here and how to do this task without scaling would be most helpful also).
But I am also having trouble reconciling the pixel coordinates on the tkinter canvas with the PIL picture. For example, the code below is designed to write (x, y) pixel coordinates and then its page relativity {x% across the page, y% up the page} to the box (the reason for this is because it is an input into another process). An example is: (346, 481) >> {49.856, 51.018}
But if (using a scaling factor of 0.14) I click very low to the bottom of the image, I get (209, 986) >> {30.115, -0.407}. The relativities should be bounded between 0 and 100% so should not be negative and I cannot see this on my PIL produced .png file.
If I use a scaling factor of 0.125, I can write text to the tkinter canvas box fine, but the text appears quite a bit lower (ie outside the box) on the PIL .png file which is saved to the drives. So something is clearly not working between these two systems.
How can I reconcile PIL and tkinter pixel coordinates?
As an aside, I have a four separate functions to handle finer key adjustments. Ideally these would be one function, but I could not get the arrow buttons ( etc) to work inside an if elif block (eg, I tried this and some more derivatives of left, right etc)
def mouseMovement(event):
moveSpeed = 1
try:
int(event.char)
moveSpeed = max(1, int(event.char)*5)
return True
except ValueError:
return False
x, y = pyautogui.position()
if event.char == '<Left>':
pyautogui.moveTo(x-moveSpeed, y)
elif event.char == '<Right>':
pyautogui.moveTo(x+moveSpeed, y)
root.bind('<Key>' , mouseMovement)
Any help greatly appreciated!
Almost working example below:
from tkinter import *
from PIL import Image, ImageDraw, ImageFont, ImageTk
import pyautogui
# Form
formName = '2013+ MCS4'
# PIL image'
formImage = Image.open(formName+'.png')
wForm, hForm = formImage.size
scale = 0.14
formImage = formImage.resize((int(scale*wForm), int(scale*hForm)), Image.ANTIALIAS)
draw = ImageDraw.Draw(formImage)
font = ImageFont.truetype('arial.ttf', 10)
textColor = (255, 40, 40)
# tkinter canvas
def colorConversion(RGB):
def hexadecimalScale(RGB):
hexadecimalSystem = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 'A', 'B', 'C', 'D', 'E', 'F')
return str(hexadecimalSystem[RGB//16]) + str(hexadecimalSystem[RGB%16])
return '#' + hexadecimalScale(RGB[0]) + hexadecimalScale(RGB[1]) + hexadecimalScale(RGB[2])
fontCanvas = 'arial 7'
textColorCanvas = colorConversion(textColor)
# generate canvas
if __name__ == '__main__':
root = Tk()
# set up tkinter canvas with scrollbars
frame = Frame(root, bd=2, relief=SUNKEN)
frame.grid_rowconfigure(0, weight=1)
frame.grid_columnconfigure(0, weight=1)
xscroll = Scrollbar(frame, orient=HORIZONTAL)
xscroll.grid(row=1, column=0, sticky=E+W)
yscroll = Scrollbar(frame)
yscroll.grid(row=0, column=1, sticky=N+S)
canvas = Canvas(frame, width=int(scale*wForm), height=int(scale*hForm), bd=0, xscrollcommand=xscroll.set, yscrollcommand=yscroll.set)
canvas.grid(row=0, column=0, sticky=N+S+E+W)
xscroll.config(command=canvas.xview)
yscroll.config(command=canvas.yview)
frame.pack(fill=BOTH,expand=1)
# add image
#img = PhotoImage(file=formName+'.png')
img = ImageTk.PhotoImage(formImage)
canvas.create_image(0,0,image=img,anchor="nw")
canvas.config(scrollregion=canvas.bbox(ALL))
wForm = img.width()
hForm = img.height()
# finer mouse movements
moveSpeed = 1
def setMoveSpeed(event):
global moveSpeed
try:
int(event.char)
moveSpeed = max(1, int(event.char)*5)
return moveSpeed
except ValueError:
return False
def moveMouseLeft(event):
x, y = pyautogui.position()
pyautogui.moveTo(x-moveSpeed, y)
def moveMouseRight(event):
x, y = pyautogui.position()
pyautogui.moveTo(x+moveSpeed, y)
def moveMouseUp(event):
x, y = pyautogui.position()
pyautogui.moveTo(x, y-moveSpeed)
def moveMouseDown(event):
x, y = pyautogui.position()
pyautogui.moveTo(x, y+moveSpeed)
root.bind('<Key>' , setMoveSpeed)
root.bind('<Left>' , moveMouseLeft)
root.bind('<Right>', moveMouseRight)
root.bind('<Up>' , moveMouseUp)
root.bind('<Down>' , moveMouseDown)
# print coordinates
def printCoordinates(event):
x = event.x # minor adjustments to correct for differences in tkinter vs PIL methods (investigate further)
y = event.y # minor adjustments to correct for differences in tkinter vs PIL methods (investigate further)
canvas.create_text(x, y-5, fill= textColorCanvas, font= fontCanvas, anchor= 'sw',
text= '{'+str(round(x/wForm*100,3))+', '+str(round((1-y/hForm)*100,3))+'}' )
draw.text( (x, y-5), '{'+str(round(x/wForm*100,3))+', '+str(round((1-y/hForm)*100,3))+'}' , fill=textColor, font=font)
print('('+str(x)+', '+str(y)+') >> {'+str(round(x/wForm*100,3))+', '+str(round((1-y/hForm)*100,3))+'}')
root.bind('<Return>', printCoordinates)
root.mainloop()
formImage.save('coordinates - '+formName+'.png')
I cannot run your code, so this is just an educated guess.
Since the canvas doesn't typically have focus, and the binding is on the root window, the values for event.x and event.y are possibly relative to the window as a whole rather than the canvas.
This should be easy to determine. You can print out the coordinates in the binding, and then click in the upper left corner of the canvas, as near to 0,0 as possible. The printed coordinates should also be very near to 0,0 if the coordinates are relative to the canvas. If they are off, they might be off by the distance of the top-left corner of the canvas to the top-left corner of the window.
It's due to Pillow setting the text coordinates on the left-hand corner(i.e. the coordinates selected become the top left-hand corner of whatever textbox pillow adds). But in newer versions of Pillow(I think 8.1), the text has an anchor option. Simply in the parameters for creating text add anchor="mm". The Pillow docs have more info on this(ps. you'll also have to slightly increase the font in some cases due to Pillow font sizes being slightly smaller. I found adding 4 gets pretty close)
EDIT: make sure the cords you're using are also Canvas cords

Filling a Tkinter Canvas Element with an Image

Is there any way to fill a Tkinter element (more specifically an oval) with an image. If not, is there any way to resize an image to fit into an oval? I also would prefer not to use PIL.
canvas.create_oval(val1, val2, val1+sz, val2+sz, fill=clr, outline=outln)
How would you get an image to fit inside a circle like so?
I would also certainly cull the edges around the image if you were wondering.
In plain Tkinter your options are limited. One thing you could do is create a
'mask' image that has a transparent circle in the middle. Then lower your
own image in behind it. (not very efficient though)
from tkinter import *
root = Tk()
canvas = Canvas(root, width=200, height=200, bd=0,
highlightthickness=0)
canvas.pack()
mask = PhotoImage(width=200, height=200)
cx,cy = 100,100 # image & circle center point
r = 100 # circle radius
r_squared = r*r
for y in range(200):
for x in range(200):
# using the formula for a circle:
# (x - h)^2 + (y - k)^2 = r^2
# any pixel outside our circle gets filled
if (x - cx)**2 + (y - cy)**2 > r_squared:
mask.put('blue', (x,y))
canvas.create_image(100,100, image=mask, anchor='c')
myimage = PhotoImage(file='bigpython.gif')
item = canvas.create_image(100,100, image=myimage, anchor='c')
canvas.lower(item)
root.mainloop()
No, there is no way to put an image inside an oval. That is to say, you can't make one fill up an oval. Images can only be rectangular.
Your only choice for resizing images without using PIL is to resize it by factors of two.
I've tried all ways to try and get a photo to fit inside a circle but the only kind-of effective way is to resize the image using the Paint 2D app in Windows and then using the resized image as the image used to fill the circle.

Padding an image for use in wxpython

I'm looking for the most efficient way to 'square' an image for use as an icon. For example, I've got a .png file whose dimensions are 24x20.I don't want to change the 'image' part of the image in any way, I just want to add transparent pixels to the edge of the image so it becomes 24x24. My research suggests that I need to create a transparent canvas 24x24, paste my image on to this, then save the result. I'm working in wxpython and was wondering if anyone could guide me through the process. Better yet, I also have PIL installed, and was wondering if there wasn't a built-in way of doing this. It seems like the kind of operation that would be carried out fairly regularly, but none of the imageops methods quite fit the bill.
Use image.paste to paste the image on a transparent background:
import Image
FNAME = '/tmp/test.png'
top = Image.open(FNAME).convert('RGBA')
new_w = new_h = max(top.size)
background = Image.new('RGBA', size = (new_w,new_h), color = (0, 0, 0, 0))
background.paste(top, (0, 0))
background.save('/tmp/result.png')
You could do it with numpy array pretty easy .. something like this
import matplotlib.pyplot as plt
import numpy as np
im1 = plt.imread('your_im.png')
im0 = np.zeros((24, 24, 4), dtype=im1.dtype)
im0[2:-2,:,:] = im1
plt.imsave('your_new_im.png', im0)
Here's a pure wxPython implementation.
import wx
app = wx.PySimpleApp()
# load input bitmap
bitmap = wx.Bitmap('input.png')
# compute dimensions
width, height = bitmap.GetSize()
size = max(width, height)
dx, dy = (size - width) / 2, (size - height) / 2
# create output bitmap
new_bitmap = wx.EmptyBitmap(size, size)
dc = wx.MemoryDC(new_bitmap)
dc.SetBackground(wx.Brush(wx.Colour(255, 0, 255)))
dc.Clear()
dc.DrawBitmap(bitmap, dx, dy)
del dc
# save output
image = wx.ImageFromBitmap(new_bitmap)
image.SetMaskColour(255, 0, 255)
image.SaveFile('output.png', wx.BITMAP_TYPE_PNG)

Center-/middle-align text with PIL?

How would I center-align (and middle-vertical-align) text when using PIL?
Deprecation Warning: textsize is deprecated and will be removed in Pillow 10 (2023-07-01). Use textbbox or textlength instead.
Code using textbbox instead of textsize.
from PIL import Image, ImageDraw, ImageFont
def create_image(size, bgColor, message, font, fontColor):
W, H = size
image = Image.new('RGB', size, bgColor)
draw = ImageDraw.Draw(image)
_, _, w, h = draw.textbbox((0, 0), message, font=font)
draw.text(((W-w)/2, (H-h)/2), message, font=font, fill=fontColor)
return image
myFont = ImageFont.truetype('Roboto-Regular.ttf', 16)
myMessage = 'Hello World'
myImage = create_image((300, 200), 'yellow', myMessage, myFont, 'black')
myImage.save('hello_world.png', "PNG")
Result
Use Draw.textsize method to calculate text size and re-calculate position accordingly.
Here is an example:
from PIL import Image, ImageDraw
W, H = (300,200)
msg = "hello"
im = Image.new("RGBA",(W,H),"yellow")
draw = ImageDraw.Draw(im)
w, h = draw.textsize(msg)
draw.text(((W-w)/2,(H-h)/2), msg, fill="black")
im.save("hello.png", "PNG")
and the result:
If your fontsize is different, include the font like this:
myFont = ImageFont.truetype("my-font.ttf", 16)
draw.textsize(msg, font=myFont)
Here is some example code which uses textwrap to split a long line into pieces, and then uses the textsize method to compute the positions.
from PIL import Image, ImageDraw, ImageFont
import textwrap
astr = '''The rain in Spain falls mainly on the plains.'''
para = textwrap.wrap(astr, width=15)
MAX_W, MAX_H = 200, 200
im = Image.new('RGB', (MAX_W, MAX_H), (0, 0, 0, 0))
draw = ImageDraw.Draw(im)
font = ImageFont.truetype(
'/usr/share/fonts/truetype/msttcorefonts/Arial.ttf', 18)
current_h, pad = 50, 10
for line in para:
w, h = draw.textsize(line, font=font)
draw.text(((MAX_W - w) / 2, current_h), line, font=font)
current_h += h + pad
im.save('test.png')
One shall note that the Draw.textsize method is inaccurate. I was working with low pixels images, and after some testing, it turned out that textsize considers every character to be 6 pixel wide, whereas an I takes max. 2 pixels and a W takes min. 8 pixels (in my case). And so, depending on my text, it was or wasn't centered at all. Though, I guess "6" was an average, so if you're working with long texts and big images, it should still be ok.
But now, if you want some real accuracy, you better use the getsize method of the font object you're going to use:
arial = ImageFont.truetype("arial.ttf", 9)
w,h = arial.getsize(msg)
draw.text(((W-w)/2,(H-h)/2), msg, font=arial, fill="black")
As used in Edilio's link.
A simple solution if you're using PIL 8.0.0 or above: text anchors
width, height = # image width and height
draw = ImageDraw.draw(my_image)
draw.text((width/2, height/2), "my text", font=my_font, anchor="mm")
mm means to use the middle of the text as anchor, both horizontally and vertically.
See the anchors page for other kinds of anchoring. For example if you only want to center horizontally you may want to use ma.
The PIL docs for ImageDraw.text are a good place to start, but don't answer your question.
Below is an example of how to center the text in an arbitrary bounding box, as opposed to the center of an image. The bounding box is defined as: (x1, y1) = upper left corner and (x2, y2) = lower right corner.
from PIL import Image, ImageDraw, ImageFont
# Create blank rectangle to write on
image = Image.new('RGB', (300, 300), (63, 63, 63, 0))
draw = ImageDraw.Draw(image)
message = 'Stuck in\nthe middle\nwith you'
bounding_box = [20, 30, 110, 160]
x1, y1, x2, y2 = bounding_box # For easy reading
font = ImageFont.truetype('Consolas.ttf', size=12)
# Calculate the width and height of the text to be drawn, given font size
w, h = draw.textsize(message, font=font)
# Calculate the mid points and offset by the upper left corner of the bounding box
x = (x2 - x1 - w)/2 + x1
y = (y2 - y1 - h)/2 + y1
# Write the text to the image, where (x,y) is the top left corner of the text
draw.text((x, y), message, align='center', font=font)
# Draw the bounding box to show that this works
draw.rectangle([x1, y1, x2, y2])
image.show()
image.save('text_center_multiline.png')
The output shows the text centered vertically and horizontally in the bounding box.
Whether you have a single or multiline message no longer matters, as PIL incorporated the align='center' parameter. However, it is for multiline text only. If the message is a single line, it needs to be manually centered. If the message is multiline, align='center' does the work for you on subsequent lines, but you still have to manually center the text block. Both of these cases are solved at once in the code above.
Use the textsize method (see docs) to figure out the dimensions of your text object before actually drawing it. Then draw it starting at the appropriate coordinates.
All the other answers did NOT take text ascender into consideration.
Here's a backport of ImageDraw.text(..., anchor="mm"). Not sure if it's fully compatible with anchor="mm", cause I haven't tested the other kwargs like spacing, stroke_width yet. But I ensure you this offset fix works for me.
from PIL import ImageDraw
from PIL import __version__ as pil_ver
PILLOW_VERSION = tuple([int(_) for _ in pil_ver.split(".")[:3]])
def draw_anchor_mm_text(
im,
xy,
# args shared by ImageDraw.textsize() and .text()
text,
font=None,
spacing=4,
direction=None,
features=None,
language=None,
stroke_width=0,
# ImageDraw.text() exclusive args
**kwargs,
):
"""
Draw center middle-aligned text. Basically a backport of
ImageDraw.text(..., anchor="mm").
:param PIL.Image.Image im:
:param tuple xy: center of text
:param unicode text:
...
"""
draw = ImageDraw.Draw(im)
# Text anchor is firstly implemented in Pillow 8.0.0.
if PILLOW_VERSION >= (8, 0, 0):
kwargs.update(anchor="mm")
else:
kwargs.pop("anchor", None) # let it defaults to "la"
if font is None:
font = draw.getfont()
# anchor="mm" middle-middle coord xy -> "left-ascender" coord x'y'
# offset_y = ascender - top, https://stackoverflow.com/a/46220683/5101148
# WARN: ImageDraw.textsize() return text size with offset considered.
w, h = draw.textsize(
text,
font=font,
spacing=spacing,
direction=direction,
features=features,
language=language,
stroke_width=stroke_width,
)
offset = font.getoffset(text)
w, h = w - offset[0], h - offset[1]
xy = (xy[0] - w / 2 - offset[0], xy[1] - h / 2 - offset[1])
draw.text(
xy,
text,
font=font,
spacing=spacing,
direction=direction,
features=features,
language=language,
stroke_width=stroke_width,
**kwargs,
)
Refs
https://pillow.readthedocs.io/en/stable/handbook/text-anchors.html
https://github.com/python-pillow/Pillow/issues/4789
https://stackoverflow.com/a/46220683/5101148
https://github.com/python-pillow/Pillow/issues/2486
Using a combination of anchor="mm" and align="center" works wonders. Example
draw.text(
xy=(width / 2, height / 2),
text="centered",
fill="#000000",
font=font,
anchor="mm",
align="center"
)
Note: Tested where font is an ImageFont class object constructed as such:
ImageFont.truetype('path/to/font.ttf', 32)
This is a simple example to add a text in the center of the image
from PIL import Image, ImageDraw, ImageFilter
msg = "hello"
img = Image.open('image.jpg')
W, H = img.size
box_image = img.filter(ImageFilter.BoxBlur(4))
draw = ImageDraw.Draw(box_image)
w, h = draw.textsize(msg)
draw.text(((W - w) / 2, (H - h) / 2), msg, fill="black")
box_image.show()
if you are using the default font then you can use this simple calculation
draw.text((newimage.width/2-len(text)*3, 5), text,fill="black", align ="center",anchor="mm")
the main thing is
you have to divide the image width by 2 then get the length of the string you want and multiply it by 3 and subtract it from the division result
newimage.width/2-len(text)*3 #this is X position
**this answer is an estimation for the default font size used if you use a custom font then the multiplier must be changed accordingly. in the default case it is 3

Categories