I've got a program right now that is meant to display an image, and allow for various functions on keypress (e.g., load a new frame, crop a region of interest, etc). It's also set up so that if too much time passes with no action, it'll automatically attempt to load a new image.
def triggeredbytimer():
global f, x1i, y1i, x2i, y2i, sid, keepgoing
keepgoing = False
print "Inactivity has lead to an automatic refresh. Loading newest image."
f = 0; t = 0
incoming = imagerequest(sid, f, t, x1i, y1i, x2i, y2i)
imagecallback(incoming)
def imagecallback(data):
print "----------------------- imagecallback"
global sid, f, x1i, y1i, x2i, y2i, img, refPt, cropping, keepgoing
keepgoing = True
########### \\// change this to change wait for autorefresh
tmr = Timer(10.0, triggeredbytimer)
tmr.start()
##################################
b = data.b; t = data.tr; sid = data.sid; error = int(data.error)
t = float(t); sid = int(sid)
expandedt = time.strftime("%m-%d-%Y %H:%M:%S", time.localtime(t))
print str(t) + " ----> " + expandedt
print "----------------------- image incoming"
timestr = time.strftime("%H:%M:%S", time.gmtime(t))
print "Loaded image from %s."%timestr
imagedata = bridge.imgmsg_to_cv2(b, "bgr8")
npimg = cv2.imencode('.jpg', imagedata)
cv2.imwrite('temp_image_np.jpg', imagedata)
img = cv2.imread('temp_image_np.jpg')
clone = img.copy()
cv2.namedWindow('image', cv2.WINDOW_NORMAL)
cv2.setMouseCallback("image", mouseclickcb)
cv2.imshow("image", img)
while keepgoing == True:
key = cv2.waitKey(0) & 0xFF
## (here there are a lot of keypress functions, for brevity I included only one)
elif key == ord("t"):
print "Looking for newest frame..."
tmr.cancel()
t = 0; f = 0
incoming = imagerequest(sid, f, t, x1i, y1i, x2i, y2i)
imagecallback(incoming)
(NOTE: The images are fetched from a ROS program via services, and that is where the "imagerequest" function comes from. All of that works fine: the problem is specifically as prescribed.)
To summarize here- since there's a bit of irrelevant clippings in this snippet - an image is loaded by imagecallback. When that function is called, a timer starts (time limit is arbitrary, set low here for testing). When the timer expires, triggeredbytime() requests a new image. It acquires the image via a ROS service, then takes the return from that and uses it as an input to imagecallback(data), thus loading the image and allowing all the keypress functions to still be valid.
So the problem is that whenever the program refreshes with a new frame, the while keepgoing == True loop does not stop, leading to stacked loops that go on and get worse with every refresh. this leads to a single keypress executing multiple times, which quickly overwhelms the program it's pulling images from. (Originally, this was just a while True: loop, but I added the "keepgoing" variable to try to remedy the situation - unsuccessfully, as it were.) I'm not sure how to resolve this issue, and attempting to turn the loop "off" long enough to kill the first loop but allowing the next one to execute has not worked out so far. Is there any way to exit this while keepgoing == True loop cleanly to prevent things from stacking as such, or is there a better way in general to address the problem of loading a new image after a certain amount of time?
Thanks!
EDIT: to be clear, the problem is with ending the 'while' loop when the timer hits 0. setting keepgoing = False doesn't seem to work (probably because I can't 'return' it), so I need something similar to that if it's possible...
Related
I have a python program running a clock and thermometer (raspberry pi + fourletterphat), see program below. I change between showing time or temperature by clicking a single button (gpio16).
What I need help with:
I want to pause the program during night, 21:00 - 06:00, and clear the display because the light is annoying. With the current code I get the display to clear at time for night to start but it does not start again.
If, during above period, the button is clicked I want the program to run for 10 seconds and then stop/clear display. I simply have no idea how to do this, Not even where start.
Is there an elegant way to do this, preferably by just adding something to the existing program. See below.
I have tried various ways of either clearing the display during night time and/or pausing the program until button is hit (but only during the night time, i want the program running during day to show temperature or time).
I have found many versions on finding it time.now is within my range for night but they seem not to be compatible with starting the program as described in point 2 above. (e.g. if time.now < night_end or time.now >= night_start:)
in code below function bright() sets the brightness AND turns off the display at night start 20:00.
Function night() is my feeble start on the restart display at night time but I have not gotten further.
#! /usr/bin/env python3
#Tempclock working with fourletterphat from pimoroni. One switch to change between temp and clock.
#sets brightness at startup but not continuously.
import glob
import time
import datetime
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
GPIO.setup(16, GPIO.IN, pull_up_down=GPIO.PUD_DOWN )
input_state = GPIO.input(16)
import fourletterphat as flp
# Find 1s temp sensor
base_dir = '/sys/bus/w1/devices/'
device_folder = glob.glob(base_dir + '28*')[0]
device_file = device_folder + '/w1_slave'
# times for setting brightness of display:
# d2 and d3 time dim and brighten display
d2 = 18
d3 = 7
# d4 and d5 time to turn off display for nigh
d4 = 20
d5 = 6
butt = 1
# Set brightness
def bright():
todays_date = datetime.datetime.now()
hn = (todays_date.hour)
if hn < d5 and hn > d4:
flp.clear()
flp.show()
elif hn < d2 and hn > d3:
flp.set_brightness(12)
else:
flp.set_brightness(0)
# Define nighttime display off
def night():
todays_date = datetime.datetime.now()
hn = (todays_date.hour)
if hn < d5 or hn => d4:
flp.set_brightness(5)
# define temp and time reading
def read_temp_raw():
f = open(device_file, 'r')
lines = f.readlines()
f.close()
return lines
def read_temp():
lines = read_temp_raw()
while lines[0].strip()[-3:] != 'YES':
time.sleep(0.2)
lines = read_temp_raw()
equals_pos = lines[1].find('t=')
if equals_pos != -1:
temp_string = lines[1][equals_pos+2:]
temp_c = float(temp_string) / 1000.0
temp_f = temp_c * 9.0 / 5.0 + 32.0
return temp_c, temp_f
def display_temp():
# temp = read_temp()[1] # F
temp = read_temp()[0] # C
flp.print_float(temp, decimal_digits=1, justify_right=True)
flp.show()
def display_time():
now = datetime.datetime.now()
str_time = time.strftime("%H%M")
flp.print_number_str(str_time)
flp.show()
# Display time or temp button on gpio pin 16 push button counter "butt" and set
# brightness bright() according to time of day. Function night() turns on display for 10sec if at night, then turns it off.
while True:
bright()
if GPIO.input(16) == 0:
butt = butt + 1
night()
if butt > 2:
butt = 1
if butt == 1:
display_time()
flp.show()
elif butt == 2:
display_temp()
flp.show()
time.sleep(0.2)
I'll start by assuming that your code to display time and read temperature is working correctly. Also, since I don't have a RaspberryPi with me, I'm focusing on how to organise your logic on the parts that I think you are having difficulty. My code to specifically turn the display on or off may be wrong, but I guess you figured that part out already. Since your question is about an elegant way to do this, the solution is obviously opinion-based. What I am proposing here is a simple but sound way to do it.
The first thing to notice is that you want two modes of operation. I'll call them day_mode and night_mode. The behaviour you expect is quite different for each mode:
day_mode: display is always on, when button 16 is pressed, the display should change from temperature to time. It should be active from 6:00 until 21:00
night_mode: display is normally off. When button 16 is pressed, the display should turn on and display something (you didn't specify, but I'll assume it is the temperature). It should be active when day_mode is not active.
When thinking about modes, it usually helps to separate three things:
What you need to do when you enter that mode of operation (for example, when you enter day_mode you should turn on the display and show the clock, when you enter night_mode you should turn off the display)
What is the behaviour you want during the mode. In your case, it is what we discussed above.
Any clean-up action needed when you exit the mode. Here none is needed, so we will skip this part.
So, let's begin! From your original code, I'll keep everything but the functions night, bright and the final loop. The first thing to do is create a function that tells in which mode we should be:
import datetime as dt
def which_mode():
current_hour = dt.datetime.now().time().hour
# In your code you seemed to need special behaviour between 6 and 7 and 18 and 21,
# but since it was not in your question, I'm not including it here
if 6 <= current_hour <= 21:
return 'day_mode'
else:
return 'night_mode'
To represent the modes, I'll use functions. If your problem were more complex, classes with specific enter, exit and during methods and a full blown state machine implementation would be needed. Since your problem is small, let's use a small solution. Each mode is a function, and the fact that you are entering the mode is informed by a parameter is_entering. The function will also receive a signal indicating that the button was pressed (a rising edge, but change according to your implementation). Before that, we will create a function that displays time or temperature based on a parameter:
def display_what(what):
if what == 0: # Time
display_time()
else: # Temperature
display_temp()
display_info = 0 # 0 means time, 1 means temperature
def day_mode(is_entering=False, button_pressed=False):
global display_info # Not very elegant, but gets the job done
if is_entering:
# Here the display is turned on. I got the values from your `bright` function
flp.set_brightness(12)
flp.clear()
# This is the `during` logic. Note that it will be run right after `entering` the first time
if button_pressed:
display_info = (display_info + 1) % 2 # Cycles between 0 and 1
display_what(display_info) # Update the display accordingly
For the night_mode function, we will need to keep a global variable to record when the button was last pressed. By convention, when it is smaller than 0, it means we are not counting the time (so we do not keep "clearing the display" all the time).
time_of_last_press = -1.0
def night_mode(is_entering=False, button_pressed=False):
if is_entering:
# Setup the dark display
flp.clear()
flp.set_brightness(0)
if button_pressed:
# Record when the button was pressed and display something
time_of_last_press = time.time()
flp.set_brightness(4) # I understood from your code that at night the display is less bright
display_what(1) # Temperature? You can use a similar logic from day_mode to toogle the displays
elif ((time.time() - time_of_last_press) > 10.0) and (time_of_last_press >= 0):
flp.clear()
flp.set_brightness(0)
time_of_last_press = -1.0
Finally, the main loop becomes relatively simple: check what should be the next mode. If it is different from the current one, set is_entering as True. Then call the function for the current mode.
# Setup the GPIO to detect rising edges.
GPIO.add_event_detect(16, GPIO.RISING)
current_mode = None # In the first run, this forces the `entering` code of the correct mode to run
while True:
button_pressed = GPIO.event_detected(16)
next_mode = which_mode()
changed = next_mode != current_mode
current_mode = next_mode
if current_mode == 'day_mode':
day_mode(changed, button_pressed)
else: # There are only two modes, but could be an elif if there were more modes
night_mode(changed, button_pressed)
time.sleep(0.2)
You may have noticed that there is some repetition: for example, in night_mode, the code to clear the display is the same code used when entering it, so you could actually call night_mode(True) to run that code again. Most importantly, there are two ifs that are similar: the one in which_mode and the one in the main loop. This can be fixed (and the code made more generic) if we change which_mode to return the function representing the mode, instead of a string. This may be a little more difficult to read if you are not used to functions returning functions, but it is a very flexible way of doing this sort of state machine:
# Must be defined after the functions `day_mode` and `night_mode` are defined
def which_mode():
current_hour = dt.datetime.now().time().hour
# In your code you seemed to need special behaviour between 6 and 7 and 18 and 21,
# but since it was not in your question, I'm not including it here
if 6 <= current_hour <= 21:
return day_mode # Note that this is not a string. It is a function!
else:
return night_mode
# Setup the GPIO to detect rising edges.
GPIO.add_event_detect(16, GPIO.RISING)
current_mode = None # In the first run, this forces the `entering` code of the correct mode to run
while True:
button_pressed = GPIO.event_detected(16)
next_mode = which_mode()
changed = next_mode != current_mode
current_mode = next_mode
current_mode(changed, button_pressed) # Now current_mode is a reference to one of the two mode functions, so we just need to call it.
time.sleep(0.2)
Total beginner here.
In this program, I am trying to "monitor" a specific area on my screen. There are several images entering the region (from the right side) with random frequency. (Basically, a "simple" rhythm" game).
I have tried pyautoguy.locateOnScreen, but it was way too slow even though I set the region into a small square. I don't think I can use pixels because the images entering the region are all the same color but have different symbols on them, and each relates to a keyboard letter.
In this last attempt, I tried this:
Create an "endless" while loop, in which I make a screenshot of my desired region/location(bbox), and within that endless loop, I use IF and several ELIF in which I use pyautogui.locate to search a specific image (tiny screenshot of the symbol) on the bigger screenshot made by ImageGrab. I have several images, and each image has a different keyboard response.
I got to about 75-80% accuracy, but when the symbols are close to each other, it always misses. When I checked at which rate the whole loop runs, I get to like 2.9 runs per second, sometimes 4-5. Could anyone give me some pointers on how to double the speed (if that's even possible?)
Thanks!
import time
import pyautogui
import os
import pydirectinput
from PIL import ImageGrab, Image
import time
start_time = time.time()
x = 1 # frames
counter = 0
bbox=(588,786,635,944) #my region I am searching in
def detect(imgDir, haystackImage, confidence=0.90, cache=True):
if (pyautogui.locate(os.path.join('images', imgDir), haystackImage, confidence=confidence)) is not None:
return True
else:
return False
while True:
ScreenGrab = ImageGrab.grab(bbox)
ScreenName = f"screenshot"
Path = f"./images/{ScreenName}.png"
ScreenGrab.save(Path)
hsImage = Image.open('D:/PY/images/screenshot.png')
if detect('D:/PY/images/a.png', hsImage, cache=True):
pydirectinput.press('a')
continue
elif detect('D:/PY/images/b.png', hsImage, cache=True):
pydirectinput.press('b')
continue
elif detect('D:/PY/images/c.png', hsImage, cache=True):
pydirectinput.press('c')
continue
elif detect('D:/PY/images/d.png', hsImage, cache=True):
pydirectinput.press('d')
continue
elif detect('D:/PY/images/e.png', hsImage, cache=True):
pydirectinput.press('e')
continue
else:
pass
counter+=1 #counter for my runs per second
if (time.time() - start_time) > x :
print("FPS: ", counter / (time.time() - start_time))
counter = 0
start_time = time.time()
This code:
Looks for an image a
If it finds a it tries to find a match for any image in the array
image_list
If it finds an image in image_list it looks for e
If it finds e it logs it and moves the mouse to x, y checks for pixel colour and then clicks when when it finds a match, clicks.
This is where my problem arises my avoidLog() function is being called 2-3 times per click. I've added print("click") to check if it's actually clicking and it isn't, it's clicking at the right time but for some reason, my log is triggering more than once.
It's logging correctly, just an odd number of times.
I know python is operating in a single thread in my example so I'm not sure why it's looping back round and not clicking. It would make more sense to me if it was clicking multiple times and logging multiple times.
import pyautogui as py
def avoidLog():
avoidLog = open('Avoided.txt', 'a')
avoidLog.write("Found at: " + str(f) + " at: " + str(skipTime))
avoidLog.write("\n")
avoidLog.close()
image_list = []
while True:
if py.locateOnScreen('a.jpg') != None:
for image in image_list:
found = py.locateCenterOnScreen(image)
if found != None:
skipTrigger = py.locateOnScreen('e.jpg')
if skipTrigger != None:
avoidLog()
py.moveTo(x, y)
r = py.pixelMatchesColor(x,y, (r,g,b))
if r == True:
py.sleep(2)
print("click")
py.click()
break
avoidLog() is called whenever e.jpg is located. However, py.click() is only called if pixelMatchesColor is True
I am having a problem related to performance using OpenCV3.2 with Python. I have just integrated another subsystem to my main program and it slowed down a lot.
This is my initial code without integrating the new subsystem, I am using cv2.getTickCount to measure time, as suggested by OpenCv3.2 Official Website.
# Infinite loop
while True:
# getting tick count
e1 = cv2.getTickCount()
# storing frame
_, img = cap.read()
# define red colours in the screen
findRedColours(img, board)
# getting tick count after the functions
e2 = cv2.getTickCount()
# calculating time
t = (e2 - e1) / cv2.getTickFrequency()
# print time
print(t)
# check if img is none
if img is not None:
# omitted code
k = cv2.waitKey(20) & 0xFF
# start the game, hide info
if (k == ord('s') or k == ord('S')) and start is False:
# create new thread to play game
t = Thread(target=playGame)
t.start()
Basically, I am calling a function in the infinite loop to find red colours and by pressing start I create the Thread and the game starts.
This is the time needed before I press 'S' to create the Thread:
0.019336862
0.016924178
0.022487864
This is the time needed after I press 'S' to create the Thread:
0.091731532
0.125760734
0.098221829
Here everything works fine, there is a light change in the time, but nothing too important. I start to have problems when I add my new subsystem. Here the following code with the integration of the new system, it is the same of the previous one, it is just a function call that changes:
# Infinite loop
while True:
# getting tick count
e1 = cv2.getTickCount()
# storing frame
_, img = cap.read()
# extract grid
gridExtractor.extractGrid(img)
# define red colours in the screen
findRedColours(img, board)
# getting tick count after the functions
e2 = cv2.getTickCount()
# calculating time
t = (e2 - e1) / cv2.getTickFrequency()
# print time
print(t)
# check if img is none
if img is not None:
# omitted code
k = cv2.waitKey(20) & 0xFF
# start the game, hide info
if (k == ord('s') or k == ord('S')) and start is False:
# create new thread to play game
t = Thread(target=playGame)
t.start()
and this is the time before I create the Thread:
0.045629524
0.023788123
0.10517206
It is slightly higher than the one without the integration, but still ok. Here is the time after I create Thread:
1.061517957
0.568310864
0.691701059
There is an enormous difference between this one and the previous one, it reaches even a whole second. It is noticeable even from the camera output, really slow.
My questions are, am I creating my Thread in the wrong way? There is a better and more efficient way to use rather the Threads? Or is there actually a way to optimise performance in this case without having to modify these functions findRedColours(img, board), t = Thread(target=playGame), gridExtractor.extractGrid(img)?
I am new using OpenCV and Python and still having troubles around. Hope someone can address me to the right way. Thanks.
Thanks to user 'deets' who help commenting above it has been possible to optimise performance.
In this case is enough to substitute Thread with Process from multiprocessing module in Python.
from multiprocessing import Process
#omitted code
while True:
# getting tick count
e1 = cv2.getTickCount()
# storing frame
_, img = cap.read()
# extract grid - first subsystem
gridExtractor.extractGrid(img)
# define red colours in the screen - second subsystem
findRedColours(img, board)
# getting tick count after the functions
e2 = cv2.getTickCount()
# calculating time
t = (e2 - e1) / cv2.getTickFrequency()
# print time
print(t)
# check if img is none
if img is not None:
# omitted code
k = cv2.waitKey(20) & 0xFF
# start the game, hide info
if (k == ord('s') or k == ord('S')) and start is False:
# create new thread to play game
p = Process(target=playGame)
p.start()
And the relative time needed is:
0.022570883
0.11354852
0.119643379
Compared to Thread the use of Process is way more efficient in terms of performance.
This is a short script using pyscreenshot (https://github.com/ponty/pyscreenshot) to get the screen image then looks for a smaller loaded image (local file 'refresh.png', about 32x32 px) inside of it.
I need to duplicate tabs in Chrome and middle clicking refresh does this wonderfully without any reloads.
I get error C:\slave\WinInstallerMegaPack\src\opencv\modules\core\src\matrix.cpp:113: error: (-215) s >= 0 when it gets to second part of the watch. Getting the whole screen the checking works, but getting just part of the screen then checking is a problem. I have no idea what that error means, even after looking at some source. Checking the whole screen for 'refresh.png' takes a bit over a second. Afterwards it starts failing on the partial screen screengrab check and I can't figure out why or understand this error. Checking just part of the screen would go much faster.
From the only results on Google I'd have to guess that the array has too many variables, but I don't get how. I added some print statements that show everything before and after the problem is 1 variable of len 4.
import Image, numpy
import cv2
import numpy as np
import pyscreenshot as ImageGrab
import os
from pymouse import PyMouse
from time import clock, sleep
def grabScreen(Bbox = None): #unlike rect in pygame, bbox does top left (x,y), bottom right (x,y). So the second x,y isn't width, height, it's width + x, height + y
tempmousexy = m.position() #get current mousexy
m.move(0,0) #move pointer to the corner for screen shot (change this if you put the refresh button in the very top left, weirdo
if Bbox:
screenshot = ImageGrab.grab(bbox=Bbox) #grab part of screen
else:
screenshot = ImageGrab.grab() #grab whole screen
m.move(tempmousexy[0],tempmousexy[1]) #put mouse pointer back after screenshot
return np.array(screenshot)
def saveScreen(savename = 'img.png'): #not used right now, but a cool line
ImageGrab.grab_to_file(savename)
def findMatch(img, template, closenesslimit = .001, method = cv2.TM_CCOEFF_NORMED):
#convert to gray for faster processing - probably still accurate enough
template = cv2.cvtColor(template, cv2.CV_32FC1)
img = cv2.cvtColor(img, cv2.CV_32FC1)
#check for match
m = cv2.matchTemplate(template, img, cv2.cv.CV_TM_SQDIFF_NORMED)
#we want the minimum squared difference - lower means less difference
mn,_,mnLoc,_ = cv2.minMaxLoc(m)
print 'RAW (best match, worst match, bestmXY, worstmXY):', str(cv2.minMaxLoc(m))
#check if match is closer (less than) closenesslimit
print 'Closest match (lower is better):', str(mn)
if mn > closenesslimit: return False
#else
print '~Match found~ (closeness was better - less - than limit)'
#find rectangle
matchx, matchy = mnLoc
matchw, matchh = template.shape[:2]
print (matchx,matchy,matchx+matchw,matchy+matchh) #checking what is about to be returned
return (matchx,matchy,matchx+matchw,matchy+matchh) #x1, y1, x2, y2
def checkScreen():
print 'Checking whole screen'
return findMatch(grabScreen(),templateimg)
def checkArea(checkrect):
print 'Checking area'
return findMatch(grabScreen(checkrect),templateimg)
global m
m = PyMouse() #load PyMouse for mouse moves and position checks
guessxy = 0 #once a good x,y is detected, detection will start out there
startdelay = 3 #delay while waiting for first event
runningdelay = .2 #delay between repeating check events after first event
if os.name == 'nt':
templateimg = cv2.imread('bensmall.png')
else:
templateimg = cv2.imread('alexsmall.png') #changes based on resolution, would be better if this scaled automatically based on screen res
print 'Global mouse set, small template (refresh button image) loaded'
while True:
while not guessxy: #check the whole screen until a location is found
sleep(startdelay)
guessxy = checkScreen()
if not guessxy: print 'Nothing found yet, sleeping', str(startdelay)
print 'GUESSXY:', str(guessxy)
fails = 0
faillimit = 5
while guessxy: #once location is found, guess near this area again
sleep(runningdelay) #sleep at the beginning avoids awkward pauses later when something is found but it has to sleep again before printing about it
runstart = clock()
temppos = m.position()
print 'GUESSXY (limiting screenshot to this area):', str(guessxy)
found = checkArea(guessxy) #increase guessxy area by some small amount on each side later
print found
if found:
guessxy = found
print guessxy
m.move(guessxy[0],guessxy[1]) #remove this line later
print 'found'
else:
fails+=1
print 'Not found,', str(fails)
if fails>= faillimit: #wasn't near the guess point, go back to checking whole screen
print 'Too much fail, resetting...'
guessxy = 0
print clock()-runstart