We Keep Coding

Find minimal number of rectangles in the image

I have binary images where rectangles are placed randomly and I want to get the positions and sizes of those rectangles.
If possible I want the minimal number of rectangles necessary to exactly recreate the image.
On the left is my original image and on the right the image I get after applying scipys.find_objects()
(like suggested for this question).
import scipy
# image = scipy.ndimage.zoom(image, 9, order=0)
labels, n = scipy.ndimage.measurements.label(image, np.ones((3, 3)))
bboxes = scipy.ndimage.measurements.find_objects(labels)
img_new = np.zeros_like(image)
for bb in bboxes:
img_new[bb[0], bb[1]] = 1
This works fine if the rectangles are far apart, but if they overlap and build more complex structures this algorithm just gives me the largest bounding box (upsampling the image made no difference). I have the feeling that there should already exist a scipy or opencv method which does this.
I would be glad to know if somebody has an idea on how to tackle this problem or even better knows of an existing solution.
As result I want a list of rectangles (ie. lower-left-corner : upper-righ-corner) in the image. The condition is that when I redraw those filled rectangles I want to get exactly the same image as before. If possible the number of rectangles should be minimal.
Here is the code for generating sample images (and a more complex example original vs scipy)
import numpy as np
def random_rectangle_image(grid_size, n_obstacles, rectangle_limits):
n_dim = 2
rect_pos = np.random.randint(low=0, high=grid_size-rectangle_limits[0]+1,
size=(n_obstacles, n_dim))
rect_size = np.random.randint(low=rectangle_limits[0],
high=rectangle_limits[1]+1,
size=(n_obstacles, n_dim))
# Crop rectangle size if it goes over the boundaries of the world
diff = rect_pos + rect_size
ex = np.where(diff > grid_size, True, False)
rect_size[ex] -= (diff - grid_size)[ex].astype(int)
img = np.zeros((grid_size,)*n_dim, dtype=bool)
for i in range(n_obstacles):
p_i = np.array(rect_pos[i])
ps_i = p_i + np.array(rect_size[i])
img[tuple(map(slice, p_i, ps_i))] = True
return img
img = random_rectangle_image(grid_size=64, n_obstacles=30,
rectangle_limits=[4, 10])

Here is something to get you started: a naïve algorithm that walks your image and creates rectangles as large as possible. As it is now, it only marks the rectangles but does not report back coordinates or counts. This is to visualize the algorithm alone.
It does not need any external libraries except for PIL, to load and access the left side image when saved as a PNG. I'm assuming a border of 15 pixels all around can be ignored.
from PIL import Image
def fill_rect (pixels,xp,yp,w,h):
for y in range(h):
for x in range(w):
pixels[xp+x,yp+y] = (255,0,0,255)
for y in range(h):
pixels[xp,yp+y] = (255,192,0,255)
pixels[xp+w-1,yp+y] = (255,192,0,255)
for x in range(w):
pixels[xp+x,yp] = (255,192,0,255)
pixels[xp+x,yp+h-1] = (255,192,0,255)
def find_rect (pixels,x,y,maxx,maxy):
# assume we're at the top left
# get max horizontal span
width = 0
height = 1
while x+width < maxx and pixels[x+width,y] == (0,0,0,255):
width += 1
# now walk down, adjusting max width
while y+height < maxy:
for w in range(x,x+width,1):
if pixels[x,y+height] != (0,0,0,255):
break
if pixels[x,y+height] != (0,0,0,255):
break
height += 1
# fill rectangle
fill_rect (pixels,x,y,width,height)
image = Image.open('A.png')
pixels = image.load()
width, height = image.size
print (width,height)
for y in range(16,height-15,1):
for x in range(16,width-15,1):
if pixels[x,y] == (0,0,0,255):
find_rect (pixels,x,y,width,height)
image.show()
From the output
you can observe the detection algorithm can be improved, as, for example, the "obvious" two top left rectangles are split up into 3. Similar, the larger structure in the center also contains one rectangle more than absolutely needed.
Possible improvements are either to adjust the find_rect routine to locate a best fit¹, or store the coordinates and use math (beyond my ken) to find which rectangles may be joined.
¹ A further idea on this. Currently all found rectangles are immediately filled with the "found" color. You could try to detect obviously multiple rectangles, and then, after marking the first, the other rectangle(s) to check may then either be black or red. Off the cuff I'd say you'd need to try different scan orders (top-to-bottom or reverse, left-to-right or reverse) to actually find the minimally needed number of rectangles in any combination.

python split image in overlapping and rotating tiles

I am doing a image classification. I have very imbalanced data. I am trying couple of approaches to overcome the imbalanced data issue. one of them is oversampling the minority class.
The images that i have are already in high resolution(1392x1038), so i am splitting them into 348x256 size 16 tiles. As in oversampling, you simply replicate the minority classes. I was thinking of splitting the image into overlapping tiles with stride 1 or 2, so i would have slighlty different images and it would also help me in oversampling. Following code splits the images into specified number of defined size overlapping tiles
for i in range(0, count):
start_row_idx = random.randint(0, img_height-target_height-1)
start_col_idx = random.randint(0, img_width-target_width-1)
if mode == 'rgb':
patch = img_array[start_row_idx:(start_row_idx+target_height), start_col_idx:(start_col_idx+target_width), :]
else:
patch = img_array[start_row_idx:(start_row_idx+target_height), start_col_idx:(start_col_idx+target_width)]
patches.append(patch)
idxs.append((start_row_idx, start_col_idx))
how can I make it work for rotating overlapping tiles with defined number of tiles and size.
Edited Question:
In following image, the black squares shows the horizontal stride and tile which is I am able to get. I want to get the red color squares in that shape. I think, with red color type cropping i would be able to get more images for oversampling.

As we discussed above, you have tiles that have the potential of being overlapped so this is already being addressed. What is missing are rotating the tiles too. We will need to specify a random angle of rotation so that we can generate a random angle first.
After, this is simply a matter of applying an affine transform that is purely a rotation to the tiles then appending to the list. The problem with rotating images in OpenCV is that when you do rotate the image, it is subject to cropping so you don't get the entire tile contained in the image once you rotate.
I used the following post as inspiration to address this issue so that when you do rotate, the image is fully contained. Take note that the image will expand in dimensions in order to accommodate for the rotation and to keep the entire image contained in the rotated result.
import cv2
import numpy as np
def rotate_about_center(src, angle):
h, w = src.shape[:2]
rangle = np.deg2rad(angle) # angle in radians
# now calculate new image width and height
nw = (abs(np.sin(rangle)*h) + abs(np.cos(rangle)*w))
nh = (abs(np.cos(rangle)*h) + abs(np.sin(rangle)*w))
# ask OpenCV for the rotation matrix
rot_mat = cv2.getRotationMatrix2D((nw*0.5, nh*0.5), angle, 1)
# calculate the move from the old centre to the new centre combined
# with the rotation
rot_move = np.dot(rot_mat, np.array([(nw-w)*0.5, (nh-h)*0.5,0]))
# the move only affects the translation, so update the translation
# part of the transform
rot_mat[0,2] += rot_move[0]
rot_mat[1,2] += rot_move[1]
return cv2.warpAffine(src, rot_mat, (int(math.ceil(nw)), int(math.ceil(nh))), flags=cv2.INTER_LANCZOS4)
You use this function and call this with a random angle then save the patch when you're done. You'll also need to specify a maximum angle of rotation of course.
import random
max_angle = 20 # +/- 20 degrees maximum rotation
patches = []
idxs = []
for i in range(0, count):
start_row_idx = random.randint(0, img_height-target_height-1)
start_col_idx = random.randint(0, img_width-target_width-1)
# Generate an angle between +/- max_angle
angle = (2*max_angle)*random.random() - max_angle
if mode == 'rgb':
patch = img_array[start_row_idx:(start_row_idx+target_height), start_col_idx:(start_col_idx+target_width), :]
else:
patch = img_array[start_row_idx:(start_row_idx+target_height), start_col_idx:(start_col_idx+target_width)]
# Randomly rotate the image
patch_r = rotate_about_center(patch, angle)
# Save it now
patches.append(patch_r)
idxs.append((start_row_idx, start_col_idx))

Bokeh Circle does not fit into square?

I am plotting some geometry using bokeh and came across this. I am plotting a rectangle with equal sides (i.e. a square), and in that square, plotting a circle with diameter = width of the square. The circle should tangent to the square at edges, but it is not.
here is the code:
from bokeh.plotting import output_notebook, figure, show
output_notebook()
p = figure(width=500, height=500)
p.rect(0, 0, 300, 300, line_color='black')
p.circle(x=0, y=0, radius=150, line_color='black',
fill_color='grey', radius_units='data')
p.axis.minor_tick_out = 0
show(p)
Which results in this:
Is there anything I am doing wrong or could change to make the circle fit exactly in the square?
Thanks in advance,
Randall
Here's another case - just drawing a circle:
p = figure(width=500, height=500, x_range=(-150, 150), y_range=(-150, 150))
p.circle(x=0, y=0, radius=150, line_color='black',
fill_color='grey', radius_units='data')
show(p)
radius of the circle is 150 in the x direction, but not the y-direction.

I would like to report that as of Bokeh 0.12.7, this issue can now be fixed in a simpler manner.
As described in other posts, the main issue is not that the circle is not a circle, but that the square is not a square. This is due to the fact that actual area on which Bokeh draws the figure (the canvas) is usually not a square by default or even when the width and height are set to the same value. Bokeh by default will attempt to draw a figure by using up all the space on the canvas. This creates a mismatch between the data distance and the pixel distance of the plot.
As of 0.12.7, figures can now accept a match_aspect property which when set to True will will match the aspect of the data space to the pixel space of the plot.
In your example, simply adding the match_aspect = True in your figure
p = figure(width=500, height=500, match_aspect=True,
title="Circle touches all 4 sides of square")
p.rect(0, 0, 300, 300, line_color='black')
p.circle(x=0, y=0, radius=150, line_color='black',
fill_color='grey', radius_units='data')
will now produce

UPDATE: Please note new answer by #DuCorey below. As of Bokeh 0.12.7, aspect control is now available, for situations like this.
The issue is actually that the square is not square, and that is because the pixel aspect ratio and the "data" aspect ratio do not match. i.e., the distance per pixel is different in the x direction than it is in the y direction.
There are a few options:
You can use various properties to control the dimensions of the central plot area (e.g. plot border width and axis tick label orientation) You can also control you data ranges explicitly. In other words, you can make the aspect ratios match, and then the circle and rect will match
You can use absolute pixel units (e.g. size for a circle, and use a large square marker instead of rect) instead of "data" units.
Alternatively, if you want a circle that "deforms" when the aspects do not match, then your best bet is to use an ellipse with an identical width and height, which will work because in this case bokeh has two dimensions to use to measure (instead of the single radius) and can match each to the scale along each dimension independently.
(This is actually the fundamental difference that explains the behaviour: rect has two dimensions to measure independently. circle does not, it only has one, and has to arbitrarily use the x or y dimension to measure distance per pixel)

ok, based on the suggestions, I tried a few things.
Changed the orientation of the y-axis tick labels - still
had issue.
Changed various stand-offs, even moving in the tick
labels inside the plot (with a negative offset). Did not work either.
Changed the x_range and r_range in figure() to be equal tuples. Did not work either
Changes the plot_height (decreased it), and I could eventually, through rial and error, get the circle to fit in the square with a plot_height that was < plot width.
Lots of great practice controlling attributes of the plot. Time will invested.
However, the last change I tried worked the best. It was one of the first suggestions - change the plot border.
Weirdly, setting p.min_border=40, which on 0.12.6 is the default value, and voila, it appears the chart aspect ratio for a chart where plot_width=plot_height is truly 1 on the screen as well.
p = figure(plot_width=500, plot_height=500)
p.rect(0, 0, 300, 300, line_color=None)
p.circle(x=0, y=0, radius=150, line_color=None,
fill_color='lightgrey', radius_units='data')
p.min_border=40
show(p)
Before and after images showing the effect of adding p.min_border=40. Any value over ~33 appeared to be enough force the plot area to have the same screen x and y dimension - so the square was really a square (and the circle fit inside).

The reason for this is that you're creating a circular marker (or circle glyphs) and placing it at position (0, 0), while it seems like you want to create a circle centered at 0.
I think the rect here "happens" to work because it can scale correctly in both dimensions and remain a "rectangle".
Keyword Args:
radius (UnitsSpecPropertyDescriptor) : The radius values for circle markers (in "data space" units, by default). (default None)
radius_dimension (BasicPropertyDescriptor) : What dimension to measure circle radii along. (default 'x')
radius_units (Enum('screen', 'data')) : (default 'data')
I guess my point is here you've taken a shortcut by trying to use a "glyph" as your plot and specifying the units to be the data units.
If you want to create an actual circle you could do the following:
th = np.linspace(0, 2*np.pi)
r = 150
p = figure(width=500, height=500)
p.rect(0, 0, 300, 300, line_color='black')
p.line(r * np.cos(th), r * np.sin(th), line_color='black')
# p.circle(x=0, y=0, radius=150, line_color='black',
# fill_color='grey', radius_units='data')
p.axis.minor_tick_out = 0
show(p)
Notice the above is harder to fill (I didn't bother) because I'm guessing you need to define some closed polygon function while I only defined a line that happens to be a closed polygon, in this case a circle.

Not sure, but the bleu rectangle is not your rectangle.
Replace:
p.rect(0, 0, 300, 300, line_color='black')
By:
p.rect(-150, -150, 150, 150, line_color='black')

How to crop an image such that if coordinates don't exist, the other side is extended, hence keeping the size of the crop the same?

I'm currently cropping images this way:
self.crop = self.img[self.y_center - self.radius:self.y_center + self.radius,
self.x_center - self.radius:self.x_center + self.radius:, ]
Where y_center and x_center are defined by clicks, and radius is the size of the crop. But I repeatedly run into a problem where the click is on an extreme side of an image. In that case, how do I extend the side on the other side of the image?
Basically I do not want to compromise the size.

The simple & stupid (r) way:
check if the rectangle around the point goes beyond the image
move it the corresponding amount of pixels if it does
repeat for all the 4 sides
Not so s&s is how to handle the case where the rectangle is larger than the entire image, but we'll see...
# X,Y - size of the image - 1 (maximum possible values for x,y)
# x,y - click coordinates relative to the image
t=x+radius; if t>X: x-=t-X
t=y+radius; if t>Y: y-=t-Y
t=x-radius; if t<0: x-=t #t<0 so it's increased actually
t=y-radius; if t<0: y-=t
del t
crop = img[x-radius:min(x+radius,X), y-radius:min(y+radius,Y)]
With min, we handle the case when the rectangle is larger than the image.
We check from the "high" side because we chose the order of adjustments so that the last ones have docked the rectangle to the "low" side.

How can i make all the images on a window scale down or up with the size of the monitor

So i am making a game in python and pygame and i have the indow setup like this
display = pygame.display.set_mode((0,0), pygame.FULLSCREEN)
which makes the size of the window about 1334 X 800 so i based all the sprites and backgrounds on that size of screen but as you know not everyone has the same sized screen as me so my question is how can i make images scale with how big the monitor screen is
(P.S The game is in fullscreen mode)

First, how do you get the resolution and the scaling factor?
This is tricky, because someone's screen may not have the same aspect ratio as your 1334x800. You can letterbox (in various different ways) or stretch the sprites; you need to decide what you want, but I'll show one letterboxing possibility:
NOMINAL_WIDTH, NOMINAL_HEIGHT = 1334., 800.
surface = display.get_surface()
width, height = surface.get_width(), surface.get_height()
xscale = width / NOMINAL_WIDTH
yscale = height / NOMINAL_HEIGHT
if xscale < 1 and yscale < 1:
scale = max(xscale, yscale)
elif xscale > 1 and yscale > 1:
scale = min(xscale, yscale)
else:
scale = 1.0
Now, how do you scale each sprite and background?
Well, first, are you sure you want to? It may be simpler to just transform the whole surface. Whether this is slower or faster is hard to predict without testing (and probably not relevant anyway), but it will definitely look better (because any interpolation, dithering, antialiasing, etc. happens after compositing, instead of before—unless you're going for that 8-bit look, of course, in which case it will destroy the look…). You can do this by compositing everything to an off-screen surface of 1334x800 (or, better, scaling everything up by a constant factor), then transforming that surface for display. (Note that the transform methods include an optional DestSurface argument. You can use this to directly transform from the offscreen surface to the display's surface.)
But let's assume you want to do it the way you asked.
You can do this when loading the sprites. For example:
def rescale(surf, scale):
new_width, new_height = surf.get_width() * scale, surf.get_height() * scale
return pygame.transform.smoothscale(surf, (new_width, new_height))
class ScaledSprite(pygame.sprite.Sprite):
def __init__(self, path, scale):
pygame.sprite.Sprite.__init__(self)
self.image = rescale(pygame.image.load(path), scale)
self.rect = self.image.get_rect()
And the same for the backgrounds.

from this SO question, you can get the size of the monitor with
infoObject = pygame.display.Info()
which gets the height and width of the screen as infoObject.current_w and infoObject.current_h
You can then use these values to scale everything appropriately.