I am working on a transmission line following algorithm using quadcopters. To do so, I need to calculate the line position on the image I receive from the UAV in order to determine a pitch velocity so the lines can be kept at the center of the image. The problem is that, when I apply a velocity on x-axis to move the UAV to the desired setpoint (left/right moviment), the image plane tilts along with the UAV which increases the positional error incorrectly. The images below exemplify the issue.
I tried something similar to this post since the UAV euler angles is known. This approach reduced the distortion caused by the frame tilting, but I couldn't eliminate it.
Transform a frame to be as if it was taken from above using OpenCV
The code:
f = 692.81 # Focal Length
# Frame Shape
cx = width
cy = height
#Euller Angles
roll_ = 0
pitch_ = pitch
yaw_ = 0
dx = 0
dy = 0
dz = 1
A2 = np.array([[f,0,cx,0],[0,f, cy,0],[0,0,1,0]])
A1 = np.array([[1/f,0,-cx/f],[0,1/f,-cy/f],[0,0,0],[0,0,dz]])
RX = np.array([[1,0,0,0],[0,np.cos(roll_),-(np.sin(roll_)),0],[0, np.sin(roll_), np.cos(roll_),0],[0,0,0,1]])
RY = np.array([[np.cos(pitch_), 0, -np.sin(pitch_),0],[0,1,0,0],[(np.sin(pitch_)), 0, np.cos(pitch_),0],[0,0,0,1]])
RZ = np.array([[np.cos(yaw_), -(np.sin(yaw_)), 0,0],[np.sin(yaw_), np.cos(yaw_), 0,0],[0,0,1,0],[0,0,0,1]])
T = np.array([[1, 0, 0, dx],[0, 1, 0, dy],[0, 0, 1, dz],[0, 0, 0, 1]])
R = np.dot(np.dot(RX, RY), RZ)
H = np.dot(A2, np.dot(T, np.dot(R, A1)))
#The output frame
linha_bw = cv2.warpPerspective(linha_bw, H,(frame.shape[1],frame.shape[0]),None,cv2.INTER_LINEAR)
The results from this transformation can be seen on the graph below. The blue curve is the controller without the image rectification, while the red one is the controller with the code above.
I'm not sure if there is mistakes on my code or there is a better approach to solve my problem through image processing techniques. Any help is highly appreciated !
Related
I have quite complex problem and I have two options to solve it.
For a multiline shapefile (river) I would like to get cross profiles and extract DEM values for the lines.
I was thinking 1: Create ortogonal lines at defined step:
#Define a shp for the output features. Add a new field called 'M100' where the z-value of the line is stored to uniquely identify each profile
layerOut = outShp.CreateLayer('line_utm_neu_perp', layerRef, osgeo.ogr.wkbLineString)
layerDefn = layerOut.GetLayerDefn() # gets parameters of the current shapefile
layerOut.CreateField(ogr.FieldDefn('M100', ogr.OFTReal))
# Calculate the number of profiles/perpendicular lines to generate
n_prof = int(geomIn.Length()/spc)
# Define rotation vectors
rot_anti = np.array([[0, -1], [1, 0]])
rot_clock = np.array([[0, 1], [-1, 0]])
# Start iterating along the line
for prof in range(1, n_prof):
# Get the start, mid and end points for this segment
seg_st = geomIn.GetPoint(prof-1) # (x, y, z)
seg_mid = geomIn.GetPoint(prof)
seg_end = geomIn.GetPoint(prof+1)
# Get a displacement vector for this segment
vec = np.array([[seg_end[0] - seg_st[0],], [seg_end[1] - seg_st[1],]])
# Rotate the vector 90 deg clockwise and 90 deg counter clockwise
vec_anti = np.dot(rot_anti, vec)
vec_clock = np.dot(rot_clock, vec)
# Normalise the perpendicular vectors
len_anti = ((vec_anti**2).sum())**0.5
vec_anti = vec_anti/len_anti
len_clock = ((vec_clock**2).sum())**0.5
vec_clock = vec_clock/len_clock
# Scale them up to the profile length
vec_anti = vec_anti*sect_len
vec_clock = vec_clock*sect_len
# Calculate displacements from midpoint
prof_st = (seg_mid[0] + float(vec_anti[0]), seg_mid[1] + float(vec_anti[1]))
prof_end = (seg_mid[0] + float(vec_clock[0]), seg_mid[1] + float(vec_clock[1]))
# Write to output
geomLine = ogr.Geometry(ogr.wkbLineString)
geomLine.AddPoint(prof_st[0],prof_st[1])
geomLine.AddPoint(prof_end[0],prof_end[1])
featureLine = ogr.Feature(layerDefn)
featureLine.SetGeometry(geomLine)
featureLine.SetFID(prof)
featureLine.SetField('M100',round(seg_mid[2],1))
layerOut.CreateFeature(featureLine)
Problem here is that it works on one line only and not on multiline.
2 option could be creating parallel lines with offset and extract values at the same distance from the start. But I tried it only once and it did not work on my objects.
z = shapefile.offset_curve(10.0,'left')
But here I do not know what object to pass in order to make it work. Also I was thinking about creating buffer and extracting values of raster.
I will be grateful for any suggestions.
This is the code I'm using the generate the image:
pos_vec = [.15, .15, .15]
up_vec = camera_upvec(pos_vec)
viewMat = p.computeViewMatrix(
cameraEyePosition=pos_vec,
cameraTargetPosition=[0, 0, 0],
cameraUpVector=up_vec)
projMat = p.computeProjectionMatrixFOV(
fov=70.25,
aspect=1.0,
nearVal=0,
farVal=3.1)
width, height, rgbImg, depthImg, segImg = p.getCameraImage(
width=1280,
height=720,
viewMatrix=viewMat,
projectionMatrix=projMat)
print(depthImg)
this is my camera_upvec function if that helps:
from numpy import cos, sin, arccos, arcsin
def camera_upvec(pos_vec):
theta = arccos(pos_vec[-1])
sintheta = sin(theta)
phi = arccos(pos_vec[0]/sintheta)
u1 = np.array([cos(theta)*cos(phi), cos(theta)*sin(phi), -sin(theta)])
# u2 = np.array([-sin(phi), cos(phi), 0])
return -u1
And this is what the gui with the images looks like:
GUI with image visuals
Is there something obvious that is going wrong that anyone can tell?
I tried different camera angles. Some other camera nearVal/farVals. I expected the depthImg to not be full of nan values
I figured it out, btw. It was because nearVal needs to be a nonzero value. I'm guessing pybullet divides by nearVal at some point.
I am trying to identify small dashed lines in an image. An example would be identifying copy area in an excel type of application.
I have tried this.
I am finding it difficult to chose the filter sizes. So, I tried a different approach using Fourier Transform to check repeatability.
Given I know the dashed line pixel repetition range I go row by row by using a moving window to check for periodicity by finding dominant frequency in that window.
If dominant frequency is in range of dashed lines period I set the mask in the mask image. I repeat the same for columns. However this is still failing. Any suggestions/other techniques ?
Here is the code:
import cv2
import numpy as np
img = cv2.imread('test.png')
imgGray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
cv2.imshow('imgGray', imgGray)
rows,cols = imgGray.shape
maskImage = np.full((rows, cols), 0, dtype=np.uint8)
kernelL = np.array([[1, 1, 1], [1, -8, 1], [1, 1, 1]], dtype=np.float32)
imgLaplacian = cv2.filter2D(imgGray, cv2.CV_32F, kernelL)
imgResult = imgLaplacian
imgResult = np.clip(imgResult, 0, 255)
imgResult = imgResult.astype('uint8')
imgLaplacian = imgResult
cv2.imshow('imgLaplacian', imgLaplacian)
dashLineSearchInterval = 30
fmaxPixel =9 # minimum interval for dash repetation
fminPixel =7 # maximum interval for dash repetation
stride =2
for y in range(0,rows-dashLineSearchInterval,stride):
for x in range(0,cols-dashLineSearchInterval,stride):
kX = (imgLaplacian[y,x:x+ dashLineSearchInterval]).copy()
kX = kX - np.mean(kX)
N= dashLineSearchInterval
freq = np.fft.fftfreq(N)
ft = np.fft.fft(kX) # fourier transform
power = ft.real**2 + ft.imag**2 # power
maxPowerFreq= np.argmax(power) # dominant frequency
domFreq = freq [maxPowerFreq]
if(domFreq<0):
domFreq = -domFreq
#print(domFreq)
if float(1/fmaxPixel) <= domFreq <= float(1/fminPixel) :
maskImage[y,x:x+dashLineSearchInterval]=255
for x in range(0,cols-dashLineSearchInterval,stride):
for y in range(0,rows-dashLineSearchInterval,stride):
kY = (imgLaplacian[y:y+dashLineSearchInterval,x]).copy()
kY = kY - np.mean(kY)
N= dashLineSearchInterval
freq = np.fft.fftfreq(N)
ft = np.fft.fft(kY) # fourier transform
power = ft.real**2 + ft.imag**2 # power
maxPowerFreq= np.argmax(power) # dominant frequency
domFreq = freq [maxPowerFreq]
if(domFreq<0):
domFreq = -domFreq
#print(domFreq)
if float(1/fmaxPixel) <= domFreq <= float(1/fminPixel) :
maskImage[y:y+dashLineSearchInterval,x]=255
cv2.imshow('maskImage', maskImage)
cv2.waitKey()
I am working on a video processing project and I need to add a 3d effect similar to the flipping of pages in a book using python with multiple photos.
I am using moviepy and have gone through Rotate an image around its x-axis.
However, I am unable to stitch multiple(say 2) photos using the above solution. only one photo is rotating and the other photo comes in between as slide.
Also, its processing takes too long on my mac. Is there any other solution to the problem or any suggestion in my code to fix it?
from moviepy.editor import concatenate, ImageClip, VideoClip
from vapory import *
img_path1 = 'IMG_4936.JPG'
img_path2 = 'IMG_4935.JPG'
list = [img_path2,img_path1]
img_clip = ImageClip(img_path1)
W, H = img_clip.w, img_clip.h
AR = 1.0*W/H
screensize = (W,H)
# Set rotation rate by defining the period (in seconds) for 360 deg. revolution
t_rev = 2.0
total_duration = 9.0
t_half = 1.0 # The time required for a half revolution
t_still = 1.0 # How long (in seconds) to hold the half rotated image still
# Static POV-Ray objects
cam = Camera('location', [ 0, 0, -1],
'look_at', [ 0, 0, 0])
light = LightSource([0, 0, -1]) # Light at camera location
bg = Background('color', [0, 0, 0]) # Black background
slides =[]
f_slides=[]
count=-1
for i in list:
count += 1
print("path",i)
start_time = time.time()
print("before scene: ", count)
def scene(t):
print("inside scene: ",count)
""" Returns the scene at time 't' (in seconds) """
s = Scene(camera = cam, objects = [light, bg])
# Add POV-Ray box with image textured on it
s = s.add_objects([
Box([0, 0, 0],
[W, H, 0],
Texture(Pigment(ImageMap('"{}"'.format(i), 'once')),
Finish('ambient', 1.0)),
'translate', [-0.5, -0.5, 0],
'scale', [AR, 1, 0],
'rotate', [0, (360/t_rev)*t, 0])]) # Can change axis of rotation here
print("format path: ", i)
return s
def make_frame(t):
return scene(t).render(width=W, height=H, antialiasing=0.1)
still_1 = VideoClip(make_frame).to_ImageClip(t=count*2*t_rev).set_duration(t_still)
half_1 = VideoClip(make_frame).subclip(count*2*t_rev, count*2*t_rev+t_half)
still_2 = VideoClip(make_frame).to_ImageClip(t=count*2*t_rev+t_half).set_duration(t_still)
half_2 = VideoClip(make_frame).subclip(count*2*t_rev+t_half, count*2*t_rev+t_rev)
final_clip = concatenate([still_1, half_1, still_2, half_2 ])
f_slides.append(final_clip)
final_clip_f = concatenate_videoclips([j for j in f_slides],method="compose").set_duration(total_duration)
final_clip_f.write_videofile("pic_rot.mp4", fps=12, ffmpeg_params=['-movflags', '+faststart'])
end_time = time.time()
print("Total time for creating video = {}".format(end_time - start_time))
Also, the make_frame function and scene function are being called numerously. I don’t understand this repetition.
I'm am working on a project at the moment where I am trying to create a Hilbert curve using the Python Imaging Library. I have created a function which will generate new coordinates for the curve through each iteration and place them into various lists which then I want to be able to move, rotate and scale. I was wondering if anyone could give me some tips or a way to do this as I am completely clueless. Still working on the a lot of the code.
#! usr/bin/python
import Image, ImageDraw
import math
# Set the starting shape
img = Image.new('RGB', (1000, 1000))
draw = ImageDraw.Draw(img)
curve_X = [0, 0, 1, 1]
curve_Y = [0, 1, 1, 0]
combinedCurve = zip(curve_X, curve_Y)
draw.line((combinedCurve), fill=(220, 255, 250))
iterations = 5
# Start the loop
for i in range(0, iterations):
# Make 4 copies of the curve
copy1_X = list(curve_X)
copy1_Y = list(curve_Y)
copy2_X = list(curve_X)
copy2_Y = list(curve_Y)
copy3_X = list(curve_X)
copy3_Y = list(curve_Y)
copy4_X = list(curve_X)
copy4_Y = list(curve_Y)
# For copy 1, rotate it by 90 degree clockwise
# Then move it to the bottom left
# For copy 2, move it to the top left
# For copy 3, move it to the top right
# For copy 4, rotate it by 90 degrees anticlockwise
# Then move it to the bottom right
# Finally, combine all the copies into a big list
combinedCurve_X = copy1_X + copy2_X + copy3_X + copy4_X
combinedCurve_Y = copy1_Y + copy2_Y + copy3_Y + copy4_Y
# Make the initial curve equal to the combined one
curve_X = combinedCurve_X[:]
curve_Y = combinedCurve_Y[:]
# Repeat the loop
# Scale it to fit the canvas
curve_X = [x * xSize for x in curve_X]
curve_Y = [y * ySize for y in curve_Y]
# Draw it with something that connects the dots
curveCoordinates = zip(curve_X, curve_Y)
draw.line((curveCoordinates), fill=(255, 255, 255))
img2=img.rotate(180)
img2.show()
Here is a solution working on matrices (which makes sense for this type of calculations, and in the end, 2D coordinates are matrices with 1 column!),
Scaling is pretty easy, just have to multiply each element of the matrix by the scale factor:
scaled = copy.deepcopy(original)
for i in range(len(scaled[0])):
scaled[0][i]=scaled[0][i]*scaleFactor
scaled[1][i]=scaled[1][i]*scaleFactor
Moving is pretty easy to, all you have to do is to add the offset to each element of the matrix, here's a method using matrix multiplication:
import numpy as np
# Matrix multiplication
def mult(matrix1,matrix2):
# Matrix multiplication
if len(matrix1[0]) != len(matrix2):
# Check matrix dimensions
print 'Matrices must be m*n and n*p to multiply!'
else:
# Multiply if correct dimensions
new_matrix = np.zeros(len(matrix1),len(matrix2[0]))
for i in range(len(matrix1)):
for j in range(len(matrix2[0])):
for k in range(len(matrix2)):
new_matrix[i][j] += matrix1[i][k]*matrix2[k][j]
return new_matrix
Then create your translation matrix
import numpy as np
TranMatrix = np.zeros((3,3))
TranMatrix[0][0]=1
TranMatrix[0][2]=Tx
TranMatrix[1][1]=1
TranMatrix[1][2]=Ty
TranMatrix[2][2]=1
translated=mult(TranMatrix, original)
And finally, rotation is a tiny bit trickier (do you know your angle of rotation?):
import numpy as np
RotMatrix = np.zeros((3,3))
RotMatrix[0][0]=cos(Theta)
RotMatrix[0][1]=-1*sin(Theta)
RotMatrix[1][0]=sin(Theta)
RotMatrix[1][1]=cos(Theta)
RotMatrix[2][2]=1
rotated=mult(RotMatrix, original)
Some further reading on what I've done:
http://en.wikipedia.org/wiki/Transformation_matrix#Affine_transformations
http://en.wikipedia.org/wiki/Homogeneous_coordinates
http://www.essentialmath.com/tutorial.htm (concerning all the algebra transformations)
So basically, it should work if you insert those operations inside your code, multiplying your vectors by the rotation / translation matrices
EDIT
I just found this Python library that seems to provide all type of transformations: http://toblerity.org/shapely/index.html