There are several packages available to digitize the line graphs e.g. GetData Graph Digitizer. However, for digitzation of heat maps I could not find any packages or programs.
I want to digitize the heat map (images from png or jpg format) using Python. How to do it?
Do I need to write the entire code from scratch?
Or there are any packages available?
There are multiple ways to do it, many Machine Learning libraries offering custom visualization functions...easier or harder.
You need to split the problem in half.
First, using OpenCV for python or scikit-image you first have to load the images as matrices. You can set some offsets to start right at the beginning of the cells.
import cv2
# 1 - read color image (3 color channels)
image = cv2.imread('test.jpg',1)
Then, you will iterate thru the cells and read the color inside. You can normalise the result if you want. The reason we're introducing some offsets is because the heatmap doesn't start in the top left corner of the original image at (0,0). The offset_x and offset_y will be lists with 2 values each.
offset_x[0]: the offset from the left part of the image up to the beginning of the heatmap(i.e. start_of_heatmap_x)
offset_x[1]: the offset from the right part of the image up to the ending of the heatmap(i.e. image_width - end_of_heatmap_x)
offset_y[0]: the offset from the top part of the image up to the beggining of the heatmap(i.e. start_of_heatmap_y)
offset_y[1]: the offset from the bottom part of the image up to the ending of the heatmap (i.e. image_height - end_of_heatmap_y)
Also, we don't iterate up to the last column. That's because we start from the "0-th" column and we add cell_size/2 on each base local coordinates to obtain the center value of the cell.
def read_as_digital(image, cell_size, offset_x, offset_y):
# grab the image dimensions
h = image.shape[0]
w = image.shape[1]
results = []
# loop over the image, cell by cell
for y in range(offset_y[0], h-offset_y[1]-cell_size, cell_size):
row = []
for x in range(offset_x[0], w-offset_x[0]-cell_size, cell_size):
# append heatmap cell color to row
row.append(image[x+int(cell_size/2),y+int(cell_size/2)])
results.append(row)
# return the thresholded image
return results
Extracting the legend information is not hard because we can derive the values by having the limits (although this applies for linear scales).
So for example, we can derive the step on the legends (from x and y).
def generate_legend(length, offset, cell_size, legend_start, legend_end):
nr_of_cells = (length- offset[0] - offset[1])/cell_size
step_size = (legend_end - legend_start)/nr_of_cells
i=legend_start+step_size/2 # a little offset to center on the cell
values = []
while(i<legend_end):
values.append(i)
i = i+step_size
return values
Then you want to visualize them to see if everything was done right. For example, with seaborn it's very easy [1]. If you want more control, over...anything, you can use scikit learn and matplotlib [2].
Related
I'm trying to develop some code to crop multiple areas within an image. The only info I have is the x/y pixel coordinates of the top left corner of each crop, and the pixel length of each side (this number is uniform as the crops are all perfect squares.
For example, within one image I may have multiple features needing cropping. Here, the first two figures are the x/y coords of the top left corner, and the third is the length of each of the four sides (r) x, y, r
Currently trying to achieve this in skimage, but not getting very far. Sorry if this is unclear, please feel free to ask more Qs.
Ta,
Rhod
Just use Numpy slicing. So if you have:
X = 10
Y = 20
R = 15
your extracted ROI is:
ROI = im[Y:Y+R, X:X+R]
The best code for mosaic I've found you can see at this page:
https://github.com/codebox/mosaic
However, the code doesn't work well on my Windows computer, and also I think the code is too advanced for what it should do. Here are my requirements I've posted on reddit:
1) The main photo already has reduced number of colors (8)
2) I have already every image associated with colour needed to be replaced (e.g. number 1 is supposed to replace black pixels, number 2 replaces green pixels...)
3) I need to enlarge the photo by the small photo's size (9 x 9 small photos will produce 81 times bigger image), which should push the pixels "2n" points away from each other, but instead of producing a n x n same-coloured area around every single one of them (this is how I believe enlarging works in general, correct me if I'm wrong), it will just colour the white spaces with unrecognized colour, which is not associated with any small photo (let's call that colour C)
4) Now all it needs is to run through all non-C coloured pixels and put an image centered on that pixel, which would create the mosaic.
Since I'm pretty new to Python (esp. graphics) and need it just for one use, could someone help me with creating that code? I think that code I got inspired with is too complicated. Two things I don't need:
1) "approximation" - if the enlargement is lesser than needed for 100% quality (e.g. the pictures are 9x9, but every side of the original photo can be only 3 times larger, then the program needs to merge some pixels of different colours together, leading to quality loss)
2) association colour - picture: my palette of pictures is small and of colours as well, I can do it manually
For the ones who didn't get what I mean, here is my idea: https://ibb.co/9GNhqBx
I had a quick go using pyvips:
#!/usr/bin/python3
import sys
import os
import pyvips
if len(sys.argv) != 4:
print("usage: tile-directory input-image output-image")
sys.exit(1)
# the size of each tile ... 16x16 for us
tile_size = 16
# load all the tile images, forcing them to the tile size
print(f"loading tiles from {sys.argv[1]} ...")
for root, dirs, files in os.walk(sys.argv[1]):
tiles = [pyvips.Image.thumbnail(os.path.join(root, name), tile_size,
height=tile_size, size="force")
for name in files]
# drop any alpha
tiles = [image.flatten() if image.hasalpha() else image
for image in tiles]
# copy the tiles to memory, since we'll be using them many times
tiles = [image.copy_memory() for image in tiles]
# calculate the average rgb for an image, eg. image -> [12, 13, 128]
def avg_rgb(image):
m = image.stats()
return [m(4,i)[0] for i in range(1,4)]
# find the avg rgb for each tile
tile_colours = [avg_rgb(image) for image in tiles]
# load the main image ... we can do this in streaming mode, since we only
# make a single pass over the image
main = pyvips.Image.new_from_file(sys.argv[2], access="sequential")
# find the abs of an image, treating each pixel as a vector
def pyth(image):
return sum([band ** 2 for band in image.bandsplit()]) ** 0.5
# calculate a distance map from the main image to each tile colour
distance = [pyth(main - colour) for colour in tile_colours]
# make a distance index -- hide the tile index in the bottom 16 bits of the
# distance measure
index = [(distance[i] << 16) + i for i in range(len(distance))]
# find the minimum distance for each pixel and mask out the bottom 16 bits to
# get the tile index for each pixel
index = index[0].bandrank(index[1:], index=0) & 0xffff
# replicate each tile image to make a set of layers, and zoom the index to
# make an index matching the output size
layers = [tile.replicate(main.width, main.height) for tile in tiles]
index = index.zoom(tile_size, tile_size)
# now for each layer, select pixels matching the index
final = pyvips.Image.black(main.width * tile_size, main.height * tile_size)
for i in range(len(layers)):
final = (index == i).ifthenelse(layers[i], final)
print(f"writing {sys.argv[3]} ...")
final.write_to_file(sys.argv[3])
I hope it's easy to read. I can run it like this:
$ ./mosaic3.py smallpic/ mainpic/Use\ this.jpg x.png
loading tiles from smallpic/ ...
writing x.png ...
$
It takes about 5s on this 2015 laptop and makes this image:
I had to shrink it for upload, but here's a detail (bottom left of the first H):
Here's a google drive link to the mosaic, perhaps it'll work: https://drive.google.com/file/d/1J3ofrLUhkuvALKN1xamWqfW4sUksIKQl/view?usp=sharing
And here's this code on github: https://github.com/jcupitt/mosaic
This question already has an answer here:
Displaying stitched images together without cutoff using warpAffine
(1 answer)
Closed 5 years ago.
In short, my question is how do I put an image on top of another by specifying specific coordinates for the added image? I would need to extend the "canvas" of the base image as needed so that the added image doesn't get cropped.
Here's the extended version:
My project is to take pictures extracted from a drone video and make a rough map with them, by aligning one photo with the last. I know there is software I can use to do this, like Agisoft Photoscan, but my goal is to create a more lightweight, rough solution.
So here's my plan, which I intend to do with each frame:
Use estimateRigidTransform, to generate the transformation matrix to align curr_photo with the last photo, base
Calculate the bounding rectangle needed to enclose the resulting image (using transformations of the four corners)
Modify the transformation matrix so that the top left of the bounding box is at the origin
Apply the transformation to the current photo, using the bounding rectangle's width and height to ensure none of the resulting image gets cropped
Super-impose the current image with the last image (making sure no cropping of either image occurs), by adding curr_image to base at the proper coordinates. This step is what I am asking about.
Here is the code that does steps one to four.
import numpy as np
import cv2
base = cv2.imread("images/frame_03563.jpg")
curr_photo = cv2.imread("images/frame_03564.jpg")
height, width = curr_photo.shape[:2]
# Step 1
# which transformation is required to go from curr_photo to base?
transformation = cv2.estimateRigidTransform(curr_photo, base, True)
# Step 2
# add a line to the affine transformation matrix so it can be used by
# perspectiveTransform
three_by_three = np.array([
transformation[0],
transformation[1],
[0, 0, 1]], dtype="float32")
# get corners of curr_photo (to be transformed)
corners = np.array([
[0, 0],
[width - 1, 0],
[width - 1, height - 1],
[0, height - 1]
])
# where do the corners of the image go
trans_corners = cv2.perspectiveTransform(np.float32([corners]), three_by_three)
# get the bounding rectangle for the four corner points (and thus, the transformed image)
bx, by, bwidth, bheight = cv2.boundingRect(trans_corners)
# Step 3
# modify transformation matrix so that the top left of the bounding box is at the origin
transformation[0][2] = transformation[0][2] - bx
transformation[1][2] = transformation[1][2] - by
# Step 4
# transform the image in a window the size of its bounding rectangle (so no cropping)
mod_curr_photo = cv2.warpAffine(curr_photo, transformation, (bwidth, bheight))
# for viewing
cv2.imshow("base", base)
cv2.imshow("current photo", curr_photo)
cv2.imshow("image2 transformed to image 1", mod_curr_photo)
cv2.waitKey()
I've also attached two sample images. I used the first one as the base, but it works either way.
Edit: I have now turned the answer linked below into a Python module, which you can now grab from GitHub here.
I answered this question a few weeks ago. The answer should contain everything needed to accomplish what you're after; the only thing I don't discuss there is alpha blending or other techniques to blend the borders of the images together as you would with a panorama or similar.
In order to not crop the warped photo you need to calculate the needed padding beforehand because the image warp itself could reference negative indices, in which case it won't draw them...so you need to calculate the warp locations first, pad your image enough to account for those indices outside your image bounds, and then modify your warp matrix to add those translations in so they get warped to positive values.
This allows you to create an image like this:
Image from Oxford's VGG.
I want to create distortion effect like Spiral, stretch, fisheye, Wedge and other effect like underwater and snow like this website using cv2 library in python.
I figured out fisheye distortion.
In OpenCV version 3.0 and above it is possible to perform it using cv2.fisheye.undistortImage(). I have the code in python if you need.
This is what I got for the following input image:
Input Image:
Distorted image:
The function accepts a matrix, which upon modification yield different distortions of the image.
UPDATE
In order to add a snowfall effect you can add some noise like Poisson noise.
Here is a replacement block to map out a fisheye in the middle of the image. Please look elsewhere for details on the math. Use this in place of the 2 for loops in the previous code.
As stated in the first half of my answer (see previous answer), the purpose of this block is to create 2 maps that work together to remap the source image into the destination image.
To create the two maps, this block sweeps through 2 for loops with the dimensions of the image. Values are calculated for the X and y maps (flex_x and flex_y). It starts with assigning each to simply x and y for a 1-to-1 replacement map. Then, if the radius (r) is between 0 and 1, the map for the tangential slide for the fisheye is applied and new flex_x and flex_y values are mapped.
Please see my other answer for more details.
# create simple maps with a modified assignment
# outside the bulge is normal, inside is modified
# this is where the magic is assembled
for y in range(h):
ny = ((2*y-250)/(h-250))-1 #play with the 250's to move the y
ny2 = ny*ny
for x in range(w):
nx = ((2*x-50)/(w-50))-1 #play with the 50's to move the x
nx2 = nx*nx
r = math.sqrt(nx2+ny2)
flex_x[y,x] = x
flex_y[y,x] = y
if r>0 and r<1:
nr1 = 1 - r**2
nr2 = math.sqrt(nr1)
nr = (r + (1.0-nr2)) / 2.0
theta = math.atan2(ny,nx)
nxn = nr*math.cos(theta)
nyn = nr*math.sin(theta)
flex_x[y,x] = (((nxn+1)*w)/2.0)
flex_y[y,x] = (((nyn+1)*h)/2.0)
Here is half of an answer. The cv2.remap function uses maps to choose a pixel from the source for each pixel in the destination. alkasm's answer to this: How do I use OpenCV's remap function?
does a great job of defining the process, but glosses over the usefulness of those maps. If you can get creative in the maps, you can make any effect you want. Here is what I came up with.
The program starts by loading the image and resizing it. This is a convenience for a smaller screen. Then the empty maps are created.
The maps need to be the same dimensions as the image that is being processed, but with a depth of 1. If the resized original is 633 x 400 x 3, the maps both need to be 633 x 400.
When the remapping is done, cv2.remap will used the value at each coordinate in the maps to determine which pixel in the original to use in the destination. For each x,y in the destination, dest[x,y] = src[map1[x,y],map2[x,y]].
The simplest mapping would be if for every (x,y), map1(x,y)=x and map2(x,y)=y. This creates a 1-to-1 map, and the destination would match the source. In this example, a small offset is added to each value. The cosine function in the offset creates both positive and negative shifts, creating waves in the final image.
Note that creating the maps is slow, but the cv2.remap is fast. Once you have created the map, the cv2.remap is fast enough to be applied to frames of video.
import numpy as np #create waves
import cv2
import math
# read in image and resize down to width of 400
# load your image file here
image = cv2.imread("20191114_154534.jpg")
r = 400.0 / image.shape[1]
dim = (400, int(image.shape[0] * r))
# Perform the resizing of the image
resized = cv2.resize(image, dim, interpolation = cv2.INTER_AREA)
# Grab the dimensions of the image and calculate the center
# of the image (center not needed at this time)
(h, w, c) = resized.shape
center = (w // 2, h // 2)
# set up the x and y maps as float32
flex_x = np.zeros((h,w),np.float32)
flex_y = np.zeros((h,w),np.float32)
# create simple maps with a modified assignment
# the math modifier creates ripples. increase the divisor for less waves,
# increase the multiplier for greater movement
# this is where the magic is assembled
for y in range(h):
for x in range(w):
flex_x[y,x] = x + math.cos(x/15) * 15
flex_y[y,x] = y + math.cos(y/30) * 25
# do the remap this is where the magic happens
dst = cv2.remap(resized,flex_x,flex_y,cv2.INTER_LINEAR)
#show the results and wait for a key
cv2.imshow("Resized",resized)
cv2.imshow("Flexed",dst)
cv2.waitKey(0)
cv2.destroyAllWindows()
I am trying to warp an 640x360 image via the OpenCV remap function (in python 2.7). The steps executed are the following
Generate a curve and store its x and y coordinates in two seperate arrays, curve_x and curve_y.I am attaching the generated curve as an image(using pyplot):
Load image via the opencv imread function
original = cv2.imread('C:\\Users\\User\\Desktop\\alaskan-landscaps3.jpg')
Execute a nested for loop so that each pixel is shifted upwards in proportion to the height of the curve at that point.For each pixel I calculate a warping factor by dividing the distance between the curve's y coordinate and the "ceiling" (360) by the height of the image. The factor is then multiplied with the distance between the pixel's y-coordinate and the "ceiling" in order to find the new distance that the pixel must have from the "ceiling" (it will be shorter since we have an upward shift). Finally I subtract this new distance from the "ceiling" to obtain the new y-coordinate for the pixel. I thought of this formula in order to ensure that all entries in the map_y array used in the remap function will be within the area of the original image.
for i in range(0, y_size):
for j in range(0,x_size):
map_y[i][j]= y_size-((y_size - i) * ((y_size - curve_y[j]) / y_size))
map_x[i][j]=j`
Then using the remap function
warped=cv2.remap(original,map_x,map_y,cv2.INTER_LINEAR)
The resulting image appears to be warped somewhat along the curve's path but it is cropped - I am attaching both the original and resulting image
I know I must be missing something but I can't figure out where the mistake is in my code - I don't understand why since all y-coordinates in map_y are between 0-360 the top third of the image has disappeared following the remapping
Any pointers or help will be appreciated. Thanks
[EDIT:] I have edited my function as follows:
#array to store previous y-coordinate, used as a counter during mapping process
floor_y=np.zeros((x_size),np.float32)
#for each row and column of picture
for i in range(0, y_size):
for j in range(0,x_size):
#calculate distance between top of the curve at given x coordinate and top
height_above_curve = (y_size-1) - curve_y_points[j]
#calculated a mapping factor, using total height of picture and distance above curve
mapping_factor = (y_size-1)/height_above_curve
# if there was no curve at given x-coordinate then do not change the pixel coordinate
if(curve_y_points[j]==0):
map_y[i][j]=j
#if this is the first time the column is traversed, save the curve y-coordinate
elif (floor_y[j]==0):
#the pixel is translated upwards according to the height of the curve at that point
floor_y[j]=i+curve_y_points[j]
map_y[i][j]=i+curve_y_points[j] # new coordinate saved
# use a modulo operation to only translate each nth pixel where n is the mapping factor.
# the idea is that in order to fit all pixels from the original picture into a new smaller space
#(because the curve squashes the picture upwards) a number of pixels must be removed
elif ((math.floor(i % mapping_factor))==0):
#increment the "floor" counter so that the next group of pixels from the original image
#are mapped 1 pixel higher up than the previous group in the new picture
floor_y[j]=floor_y[j]+1
map_y[i][j]=floor_y[j]
else:
#for pixels that must be skipped map them all to the last pixel actually translated to the new image
map_y[i][j]=floor_y[j]
#all x-coordinates remain unchanges as we only translate pixels upwards
map_x[i][j] = j
#printout function to test mappings at x=383
for j in range(0, 360):
print('At x=383,y='+str(j)+'for curve_y_points[383]='+str(curve_y_points[383])+' and floor_y[383]='+str(floor_y[383])+' mapping is:'+str(map_y[j][383]))
The bottom line is that now the higher part of the image should not receive mappings from the lowest part so overwriting of pixels should not take place. Yet i am still getting a hugely exaggerated upwards warping effect in the picture which I cannot explain. (see new image below).The top of the curved part is at around y=140 in the original picture yet now is very close to the top i.e y around 300. There is also the question of why I am not getting a blank space at the bottom for the pixels below the curve.
I'm thinking that maybe there is also something going on with the order of rows and columns in the map_y array?
I don't think the image is being cropped. Rather, the values are "crowded" in the top-middle pixels, so that they get overwritten. Consider the following example with a simple function on a checkerboard.
import numpy as np
import cv2
import pickle
y_size=200
x_size=200
x=np.linspace(0,x_size,x_size+1)
y=(-(x-x_size/2)*(x-x_size/2))/x_size+x_size
plt.plot(x,y)
The function looks like this:
Then let's produce an image with a regular pattern.
test=np.zeros((x_size,y_size),dtype=np.float32)
for i in range(0, y_size):
for j in range(0,x_size):
if i%2 and j%2:
test[i][j]=255
cv2.imwrite('checker.png',test)
Now let's apply your shift function to that pattern:
map_y=np.zeros((x_size,y_size),dtype=np.float32)
map_x=np.zeros((x_size,y_size),dtype=np.float32)
for i in range(0, y_size):
for j in range(0,x_size):
map_y[i][j]= y_size-((y_size - i) * ((y_size - y[j]) / y_size))
map_x[i][j]=j
warped=cv2.remap(test,map_x,map_y,cv2.INTER_LINEAR)
cv2.imwrite('warped.png',warped)
If you notice, because of the shift, more than one value corresponds to the top-middle areas, which makes it look like it is cropped. But if you check to the top left and right corners of the image, notice that the values are sparser, thus the "cropping" effect does not occur much. I hope the simple example helps better to understand what is going on.