I'm currently trying to have a camera mounted on a pan tilt made of two micro servos track a face. My python code has been working and has been successfully identifying a face, but non of my servos have been moving while the Arduino is constantly flashing as if it is receiving input from the Python code. I haven't been able to get the servos to move according to my python code, but I have made simple code on the side to make sure the servos have good connections and they work fine on their own. I'm not sure what is wrong...
Python Code
import numpy as np6
import serial
import time
import sys
import cv2
arduino = serial.Serial('COM3', 9600)
time.sleep(2)
print("Connection to arduino...")
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
cap = cv2.VideoCapture(0)
while 1:
ret, img = cap.read()
cv2.resizeWindow('img', 500,500)
cv2.line(img,(500,250),(0,250),(0,255,0),1)
cv2.line(img,(250,0),(250,500),(0,255,0),1)
cv2.circle(img, (250, 250), 5, (255, 255, 255), -1)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.3)
for (x,y,w,h) in faces:
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),5)
roi_gray = gray[y:y+h, x:x+w]
roi_color = img[y:y+h, x:x+w]
arr = {y:y+h, x:x+w}
print (arr)
print ('X :' +str(x))
print ('Y :'+str(y))
print ('x+w :' +str(x+w))
print ('y+h :' +str(y+h))
xx = int(x+(x+h))/2
yy = int(y+(y+w))/2
print (xx)
print (yy)
center = (xx,yy)
print("Center of Rectangle is :", center)
data =(“X {0: f} Y {1: f} Z” .format (xx, yy))
print ("output = '" +data+ "'")
arduino.write(data.encode())
cv2.imshow('img',img)
k = cv2.waitKey(30) & 0xff
if k == 27:
break
Arduino Code
#include<Servo.h>
Servo servoVer; //Vertical Servo
Servo servoHor; //Horizontal Servo
int x;
int y;
int prevX;
int prevY;
void setup()
{
Serial.begin(9600);
servoVer.attach(5); //Vertical Servo to Pin 5
servoHor.attach(6); //Horizontal Servo to Pin 6
servoVer.write(90);
servoHor.write(90);
}
void Pos()
{
if(prevX != x || prevY != y)
{
int servoX = map(x, 600, 0, 70, 179);
int servoY = map(y, 450, 0, 179, 95);
servoX = min(servoX, 179);
servoX = max(servoX, 70);
servoY = min(servoY, 179);
servoY = max(servoY, 95);
servoHor.write(servoX);
servoVer.write(servoY);
}
}
void loop()
{
if(Serial.available() > 0)
{
if(Serial.read() == 'X')
{
x = Serial.parseInt();
if(Serial.read() == 'Y')
{
y = Serial.parseInt();
Pos();
}
}
while(Serial.available() > 0)
{
Serial.read();
}
}
}
One huge problem is the way you are using Serial.read. That function consumes the character out of the buffer. You don't get to read the same one twice. So let's say you send a 'Y'. The first if statement reads the Y out of the serial buffer and compares to 'X', that's false so it moves on. Then it reads something else from serial, probably a -1 if nothing is left to read. But it doesn't see the 'Y' because that was read in the first if.
What you need to do is to read from serial into a char variable and then use that char variable in your if statements.
char c = Serial.read();
if(c == 'X')...
... if (c == 'Y')...
Related
I am trying to read python script output continuously (In python I am tracking a colored ball and printing it's x, y and radius) from WPF C#.
I tried socket communication
using (var requester = new ZSocket(ZSocketType.REQ))
{
// Connect
requester.Connect("tcp://127.0.0.1:5555");
for (; ; )
{
string requestText = "Hello";
Console.Write("Sending {0}...", requestText);
// Send
requester.Send(new ZFrame(requestText));
// Receive
using (ZFrame reply = requester.ReceiveFrame())
{
Console.WriteLine(" Received: {0} {1}!", requestText, reply.ReadString());
}
}
}
I tried calling python from C# Process
private void Window_Loaded(object sender, RoutedEventArgs e)
{
string python = #"C:\Desktop\test\venv\Scripts\python.exe";
// python app to call
string myPythonApp = #"C:\Users\Desktop\test\main.py";
// Create new process start info
ProcessStartInfo myProcessStartInfo = new ProcessStartInfo(python);
// make sure we can read the output from stdout
myProcessStartInfo.UseShellExecute = false;
myProcessStartInfo.RedirectStandardOutput = true;
myProcessStartInfo.Arguments = myPythonApp;
Process myProcess = new Process();
// assign start information to the process
myProcess.StartInfo = myProcessStartInfo;
// start the process
myProcess.OutputDataReceived += MyProcess_OutputDataReceived;
myProcess.Start();
myProcess.BeginOutputReadLine();
// Read the standard output of the app we called.
// in order to avoid deadlock we will read output first
// and then wait for process terminate:
//StreamReader myStreamReader = myProcess.StandardOutput;
//string myString = myStreamReader.ReadLine();
//await myStreamReader.ReadAsync(result, 0, (int)myStreamReader.BaseStream.Length);
// wait exit signal from the app we called and then close it.
//myProcess.WaitForExit();
//myProcess.Close();
// write the output we got from python app
//Console.WriteLine("Value received from script: " + myString);
}
private void MyProcess_OutputDataReceived(object sender, DataReceivedEventArgs e)
{
Console.WriteLine("++++");
}
Python writes data to txt file, C# reads from that file
Console.WriteLine("hi");
for (; ; )
{
if (File.Exists(textFile))
{
using (StreamReader file = new StreamReader(textFile))
{
int counter = 0;
string ln;
while ((ln = file.ReadLine()) != null)
{
Console.WriteLine(ln);
counter++;
}
file.Close();
}
}
All the cases python is holding the thread and not allowing c# to access it. Now I am working 2 method i.e. getting data from Process, but MyProcess_OutputDataReceived is not triggering.
import sys
from collections import deque
from PIL import Image, ImageOps, ImageDraw
import numpy as np
import argparse
import imutils
import cv2
import time
import pandas as pd
import matplotlib.pyplot as plt
#import zmq
#context = zmq.Context()
#socket = context.socket(zmq.REP)
#socket.bind("tcp://*:5555")
#object tracking def start
ap = argparse.ArgumentParser()
ap.add_argument("-v", "--video",
help="path to the (optional) video file")
ap.add_argument("-b", "--buffer", type=int, default=64,
help="max buffer size")
args = vars(ap.parse_args())
greenLower = (29, 86, 6)
greenUpper = (64, 255, 255)
pts = deque(maxlen=args["buffer"])
if not args.get("video", False):
camera = cv2.VideoCapture(0)
else:
camera = cv2.VideoCapture(args["video"])
Data_Features = ['x', 'y', 'time']
Data_Points = pd.DataFrame(data=None, columns=Data_Features, dtype=float)
start = time.time()
#object tracking def end
#file1 = open("myfile.txt","w")
#L = ["This is Delhi \n","This is Paris \n","This is London \n"]
#file1.write("Hello \n")
#file1.writelines(L)
#file1.close()
while True:
# Wait for next request from client
#message = socket.recv()
#print("Received request: %s" % message)
# Do some 'work'
#time.sleep(1)
# Send reply back to client
#socket.send("World")
#object track vids start
(grabbed, frame) = camera.read()
current_time = time.time() - start
if args.get("video") and not grabbed:
break
frame = imutils.resize(frame, width=1600)
# frame2 = imutils.resize
#blurred = cv2.GaussianBlur(frame, (11, 11), 0)
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, greenLower, greenUpper)
mask = cv2.erode(mask, None, iterations=2)
mask = cv2.dilate(mask, None, iterations=2)
cnts = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)[-2]
center = None
if len(cnts) > 0:
# find the largest contour in the mask, then use
# it to compute the minimum enclosing circle and
# centroid
c = max(cnts, key=cv2.contourArea)
((x, y), radius) = cv2.minEnclosingCircle(c)
M = cv2.moments(c)
center = (int(M["m10"] / M["m00"]), int(M["m01"] / M["m00"]))
# only proceed if the radius meets a minimum size
if (radius < 300) & (radius > 10):
# draw the circle and centroid on the frame,
# then update the list of tracked points
cv2.circle(frame, (int(x), int(y)), int(radius),
(0, 255, 255), 2)
sys.stdout.write(str(x) + '\n')
#print(str(x))
#file1 = open("myfile.txt", "w")
#file1.write(str(x))
#file1.close()
#time.sleep(1)
cv2.circle(frame, center, 5, (0, 0, 255), -1)
# Save The Data Points
Data_Points.loc[Data_Points.size / 3] = [x, y, current_time]
pts.appendleft(center)
for i in range(1, len(pts)):
# if either of the tracked points are None, ignore
# them
if pts[i - 1] is None or pts[i] is None:
continue
# otherwise, compute the thickness of the line and
# draw the connecting lines
thickness = int(np.sqrt(args["buffer"] / float(i + 1)) * 2.5)
cv2.line(frame, pts[i - 1], pts[i], (0, 0, 255), thickness)
cv2.imshow("Frame", frame)
key = cv2.waitKey(1) & 0xFF
if key == ord("q"):
break
#object tracking vids end
h = 0.2
X0 = -3
Y0 = 20
time0 = 0
theta0 = 0.3
# Applying the correction terms to obtain actual experimental data
Data_Points['x'] = Data_Points['x'] - X0
Data_Points['y'] = Data_Points['y'] - Y0
Data_Points['time'] = Data_Points['time'] - time0
# Calulataion of theta value
Data_Points['theta'] = 2 * np.arctan(
Data_Points['y'] * 0.0000762 / h) # the factor correspons to pixel length in real life
Data_Points['theta'] = Data_Points['theta'] - theta0
# Creating the 'Theta' vs 'Time' plot
plt.plot(Data_Points['theta'], Data_Points['time'])
plt.xlabel('Theta')
plt.ylabel('Time')
# Export The Data Points As cvs File and plot
Data_Points.to_csv('Data_Set.csv', sep=",")
plt.savefig('Time_vs_Theta_Graph.svg', transparent=True)
# cleanup the camera and close any open windows
camera.release()
cv2.destroyAllWindows()
Please help me.
Thank you in advance.
Ranjith
I'm using an Arduino Uno and breadboard for the first time and building my first actual robot. I have 2 servos controlling the x and y axis and attached a laser pointer to the end of it. My problem is not setting up the Arduino or anything, it's the getting the servos to move to a certain spot (wherever the mouse is) and then stopping until the mouse moves. I am coding this in Python and will gladly put in the code I am trying to use. (And the Arduino code) If you see anything wrong then please correct me. Thank you. Also the code isn't mine and I just found a cool video that had links to the code but when it get's to the point (wherever the mouse is) it won't stop and instead continue twisting up the wires that are connecting everything and supplying power.
#include<Servo.h>
Servo servoVer; //Vertical Servo
Servo servoHor; //Horizontal Servo
int x;
int y;
int prevX;
int prevY;
void setup()
{
Serial.begin(9600);
servoVer.attach(10); //Attach Vertical Servo to Pin 5
servoHor.attach(11); //Attach Horizontal Servo to Pin 6
servoVer.write(90);
servoHor.write(90);
}
void Pos()
{
if (prevX != x || prevY != y)
{
prevX.update;
prevY.update;
}
}
void loop()
{
if(Serial.available() > 0)
{
if(Serial.read() == 'X')
{
x = Serial.parseInt();
if(Serial.read() == 'Y')
{
y = Serial.parseInt();
Pos();
}
}
while(Serial.available() > 0)
{
Serial.read();
}
}
}
Arduino code ^^
import pygame as game
import time
import serial
import sys
game.init()
servo_comms = serial.Serial('COM5', 9600)
game.mouse.set_visible(True)
display = game.display.set_mode((180,180))
FPS = 60
watch = game.time.Clock()
black = (0,0,0)
def coordinate_graph(x,y):
display.fill((255,225,225))
game.draw.rect(display, (0,225,0), [x + 1, y ,5,2])
game.draw.rect(display, (0,225,0), [x - 2, y ,5,2])
game.draw.rect(display, (0,225,0), [x, y + 1 ,2,5])
game.draw.rect(display, (0,225,0), [x ,y - 2 ,2,5])
watch.tick(FPS)
def starter():
while True:
(x, y) = game.mouse.get_pos()
x = int(x)
y = int(y)
data = ("X{0:.0f}Y{1:.0f}Z".format(x, y))
servo_comms.write(data.encode())
for event in game.event.get():
if event.type == game.quit:
sys.exit(0)
font = game.font.SysFont('Ariel', 24)
img = font.render(data.encode(), True, black)
display.blit(img, (20, 20))
game.display.update()
coords(x, y)
starter()
Python code ^^
If you have any ideas on how to make the motors stop when they get to the coordinates on the 180 x 180 page, please help me. Thank you.
I'm trying to make a face tracking camera w/ python & Arduino using OpenCV. I'm having trouble with the serial getting this error:
'avrdude: ser_open(): can't open device "\.\COM5": Access is denied.'
I'm not sure how I can prevent this. It won't run if the python program is already open, if I open Arduino then python it will run but won't work.
import cv2
import serial
ser = serial.Serial('COM5',baudrate = 52000)
def detectface(camera):
cap = cv2.VideoCapture(camera)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 480)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 640)
faceDetect = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
print(cap.isOpened())
while(cap.isOpened()):
ret, frame = cap.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
face = faceDetect.detectMultiScale(gray, 1.3, 5)
for(x, y, w, h) in face:
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 0, 255), 2)
center = [(x+x+w)/2, (y+y+h)/2]
center[0] = int(center[0])
center[1] = int(center[1])
if(center[0]<235):
ser.write(b'x')
elif(center[0]>245):
ser.write(b'w')
if(center[1]>335):
ser.write(b'h')
elif(center[1]<305):
ser.write(b'y')
cv2.circle(frame,(center[0],center[1]),25,(0,0,2),3,8,0)
cv2.imshow('face', frame)
if(cv2.waitKey(1) & 0xFF == ord('q')):
break
cap.release()
cv2.destroyAllWindows()
detectface(0)
#include <Servo.h>
char tiltChannel=0,panChannel=1;
char serialChar=0;
int center1;
int center2;
char pyInput;
Servo servoTilt, servoPan;
void setup() {
Serial.begin(52000);
servoTilt.attach(9);
servoPan.attach(10);
servoTilt.write(90);
servoPan.write(90);
}
void loop() {
int currentRotationX = 90;
int currentRotationY = 90;
if(Serial.available()>0){
pyInput = Serial.read();
if(pyInput == 'x'){
servoTilt.write(currentRotationX++);
currentRotationX=currentRotationX++;
}
else if(pyInput == 'w'){
servoTilt.write(currentRotationX--);
currentRotationX=currentRotationX--;
}
if(pyInput=='y'){
servoTilt.write(currentRotationY++);
currentRotationY=currentRotationY++;
}
else if(pyInput == 'h'){
servoTilt.write(currentRotationY--);
currentRotationY=currentRotationY--;
}
}
}
first try to open the arduino IDE and see on there if your board was detected in some COM Port. if you see on there that there is a com port open, copy it and paste it in your code.
If you are using anaconda try to install serial with this way:
conda install -c anaconda pyserial
And try i.g:
ser = serial.Serial('COM10', 9600)
Best regards.
I am trying to read an opencv image in a python socket that is sent from c++.
I am able to read the image into another c++ program or VB program and build an image but with python I don't understand what's happening.
The sending code where I send the mat.data:
char *start_s = "<S><size>43434234<cols>64<rows>64<SE>";//plus I send the image size, cols, rows, which varies, not like the static char string shown
char *end_e = "<E>";
cv::Mat image_send = some_mat;
iResult = send( ConnectSocket, start_s, (int)strlen(start_s), 0 );
iResult = send( ConnectSocket, (const char *) image_send.data, i_buffer_size, 0 );
iResult = send( ConnectSocket, end_e, (int)strlen(end_e), 0 );
This is what I have tried with the python, but haven't had any success yet. The image_cols and Image_rows are filtered from the socket, not shown here, and only the image_mat.data from the c++ mat is in the socket that I am trying to put into the image:
data = conn.recv(4757560)
if(i_Read_Image == 2) & (image_cols != 0) & (image_rows != 0):
print ("Entering")
#print(data)
data2 = np.fromstring(data, dtype='uint8')
img_np = cv2.imdecode(data2,cv2.IMREAD_COLOR )
cv2.imshow('image',img_np)
cv2.waitKey(0)
#Also tried this
#img = Image.new('RGB', (image_cols, image_rows))
#img.putdata(data)
#img5 = np.reshape(data2,(image_rows,image_cols))
i_Read_Image = 0
With the help of the comments I was able to get a working answer. The original image is in a single array RGB, this needs to be reshaped and placed into a 'RGB' image, it can be done in one line:
img = Image.fromarray(data2.reshape(image_rows,image_cols,3), 'RGB')
and when reading an opencv data array from a socket: this works:
data = conn.recv(567667)
if(i_Read_Image == 2) & (image_cols != 0) & (image_rows != 0):
data2 = np.fromstring(data, dtype='uint8')
img = Image.fromarray(data2.reshape(image_rows,image_cols,3), 'RGB')
img.show()
I have a weird problem with OpenCV. I was doing template matching with OpenCV on both Python and C++, however, even though Python uses the C++ methods under the hood, I get very different results. Python method gives me really accurate place, C++ is just not even close. What is the reason for this? Is it my C++ code or something else??
I use Python 2.7.11, Apple LLVM version 7.3.0 (clang-703.0.29), and OpenCV3.0.
My Python Code:
def toGray(img):
_, _, channels = img.shape
if channels == 3:
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
else:
gray = img
return gray
def template_match(img, template):
w, h = template.shape[::-1]
res = cv2.matchTemplate(img,template,cv2.TM_CCOEFF_NORMED)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)
top_left = max_loc
bottom_right = (top_left[0] + w, top_left[1] + h)
cv2.rectangle(img,top_left, bottom_right, 255, 2)
plt.subplot(121),plt.imshow(res,cmap = 'gray')
plt.title('Matching Result'), plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(img,cmap = 'gray')
plt.title('Detected Point'), plt.xticks([]), plt.yticks([])
plt.suptitle("TM_CCOEFF_NORMED")
plt.show()
if __name__ == "__main__":
img_name = sys.argv[1]
img_name2 = sys.argv[2]
img_rgb = cv2.imread(img_name)
img_rgb2 = cv2.imread(img_name2)
gimg1 = toGray(img_rgb)
gimg2 = toGray(img_rgb2)
template_match(gimg1, gimg2)
My C++ code (It is exactly the same with OpenCV documentation):
Mat img; Mat templ; Mat result;
char* image_window = "Source Image";
char* result_window = "Result window";
int match_method;
int max_Trackbar = 5;
/// Function Headers
void MatchingMethod( int, void* );
/** #function main */
int main( int argc, char** argv )
{
/// Load image and template
img = imread( argv[1], 1 );
templ = imread( argv[2], 1 );
/// Create windows
namedWindow( image_window, CV_WINDOW_AUTOSIZE );
namedWindow( result_window, CV_WINDOW_AUTOSIZE );
/// Create Trackbar
char* trackbar_label = "Method: \n 0: SQDIFF \n 1: SQDIFF NORMED \n 2: TM CCORR \n 3: TM CCORR NORMED \n 4: TM COEFF \n 5: TM COEFF NORMED";
createTrackbar( trackbar_label, image_window, &match_method, max_Trackbar, MatchingMethod );
MatchingMethod( 0, 0 );
waitKey(0);
return 0;
}
/**
* #function MatchingMethod
* #brief Trackbar callback
*/
void MatchingMethod( int, void* )
{
/// Source image to display
Mat img_display;
img.copyTo( img_display );
/// Create the result matrix
int result_cols = img.cols - templ.cols + 1;
int result_rows = img.rows - templ.rows + 1;
result.create( result_rows, result_cols, CV_32FC1 );
/// Do the Matching and Normalize
matchTemplate( img, templ, result, match_method );
normalize( result, result, 0, 1, NORM_MINMAX, -1, Mat() );
/// Localizing the best match with minMaxLoc
double minVal; double maxVal; Point minLoc; Point maxLoc;
Point matchLoc;
minMaxLoc( result, &minVal, &maxVal, &minLoc, &maxLoc, Mat() );
/// For SQDIFF and SQDIFF_NORMED, the best matches are lower values. For all the other methods, the higher the better
if( match_method == CV_TM_SQDIFF || match_method == CV_TM_SQDIFF_NORMED )
{ matchLoc = minLoc; }
else
{ matchLoc = maxLoc; }
/// Show me what you got
rectangle( img_display, matchLoc, Point( matchLoc.x + templ.cols , matchLoc.y + templ.rows ), Scalar::all(0), 2, 8, 0 );
rectangle( result, matchLoc, Point( matchLoc.x + templ.cols , matchLoc.y + templ.rows ), Scalar::all(0), 2, 8, 0 );
imshow( image_window, img_display );
imshow( result_window, result );
cv::imwrite("rec.jpg", img_display);
return;
}
Original Images:
Python Output:
C++ Output
Looking through the two implementations, the most evident difference between them is the colour format of the images used.
In the Python version, you load the images "as-is". Since your input images are RGB (as the variable names also suggest), you will be doing the template matching on colour images.
img_rgb = cv2.imread(img_name)
img_rgb2 = cv2.imread(img_name2)
However in C++ you load the images as grayscale, since you pass the 1 as second parameter.
img = imread( argv[1], 1 );
templ = imread( argv[2], 1 );
According to cv::matchTemplate documentation:
In case of a color image, template summation in the numerator and each
sum in the denominator is done over all of the channels and separate
mean values are used for each channel. That is, the function can take
a color template and a color image. The result will still be a
single-channel image, which is easier to analyze.
That would suggest that it's quite possible to get different results when applying it on a 3-channel image, than when applying it to a single channel version of the same image.