We Keep Coding

Reading memory with Pymem (MultiThreading problem?)

Hi I am currently working on Reinforcement-Learning project using Python, the problem is that I need to read a memory address to access game variables for example Speed of Car or Track Progress. Everything is working fine I also tried to create 2 threads that simulateously access the memory address. But When My PPO Agent worker is creating new Enviroment, with the class containing the Speedometer.py class and tries calling return_speed_mph() the program gives an error:
raise pymem.exception.MemoryReadError(address, struct.calcsize('i'), e.error_code)pymem.exception.MemoryReadError: Could not read memory at: 9521188, length: 4 - GetLastError: 6
The Speedometer class looks like this:
from pymem import *
from pymem.process import *
from multiprocessing import Lock, Value
class Speedometer:
mem: Pymem
module: pymem.process
shared_speed: Value
lock: Lock
offsets = [0xC, 0xC, 0x38, 0xC, 0x54]
def __init__(self):
self.module = module_from_name(self.mem.process_handle, "speed.exe").lpBaseOfDll
self.mem = Pymem("speed.exe")
self.lock = Lock()
self.shared_speed = Value('i', 0)
def return_speed_mph(self):
with self.lock:
result = self.mem.read_int(self.get_pointer_address(self.module + 0x00514824, self.offsets))
return result
def get_pointer_address(self, base, offsets):
addr = self.mem.read_int(base)
for offset in offsets:
if offset != offsets[-1]:
addr = self.mem.read_int(addr + offset)
addr = addr + offsets[-1]
return addr
I temporarily fixed the problem by moving the Inicialization of mem and module directly into return_speed_mph() function, but I think that it is memory inefficient:
def return_speed_mph(self):
with self.lock:
self.mem = Pymem("speed.exe")
self.module = module_from_name(self.mem.process_handle, "speed.exe").lpBaseOfDll
result = self.mem.read_int(self.get_pointer_address(self.module + 0x00514824, self.offsets))
return result
Is there any way to fix it somehow without moving the initialization of the module and mem directly into the return_speed_mph() function?
Thanks in advance. :D
(The Library I am using is called Pymem)

How to synchronously sample two systems using python

I need to acquire samples from two systems:
Video capturer (for example, OpenCV)
Electromagnetic tracking system.
The samples must be synchronized, the sampling time must be as stable as possible, and the total capture time is limited.
For this, I have developed two functions (update, update2), and I have tried to execute them using threads, taking as a starting point one of the examples shown in Run certain code every n seconds.
Code:
from threading import Timer, Thread, Event
from threading import Timer, Thread, Event
import polhemus # Package for Tracker system
import cv2, time
import numpy as np
import matplotlib.pyplot as plt
class InfiniteTimer():
"""A Timer class that does not stop, unless you want it to."""
def __init__(self, seconds, target):
self._should_continue = False
self.is_running = False
self.seconds = seconds
self.target = target
self.thread = None
def _handle_target(self):
self.is_running = True
self.target()
self.is_running = False
self._start_timer()
def _start_timer(self):
if self._should_continue: # Code could have been running when cancel was called.
self.thread = Timer(self.seconds, self._handle_target)
self.thread.start()
def start(self):
if not self._should_continue and not self.is_running:
self._should_continue = True
self._start_timer()
else:
print("Timer already started or running, please wait if you're restarting.")
def cancel(self):
if self.thread is not None:
self._should_continue = False # Just in case thread is running and cancel fails.
self.thread.cancel()
else:
print("Timer never started or failed to initialize.")
src=0 # Computer camera
track = polhemus.polhemus() # Tracker system
check =track.Initialize()
capture = cv2.VideoCapture(src) # Video capture system
capture.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
capture.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
capture.read()
frames=[]
pos=[]
def update(): # Function for image data capturing
t11=time.time()
(_, frame) = capture.read() # get one sample from OpenCV
t21=time.time()
frames.append((frame,[t11,t21])) # save sample
def update2(): #Function for pose data capturing
if check:
t1=time.time()
track.Run() # get one sample from Tracker
t2=time.time()
pose=[track.PositionTooltipX1, track.PositionTooltipY1, track.PositionTooltipZ1,
track.AngleX1, track.AngleY1, track.AngleZ1]
pos.append((pose,[t1,t2])) # save sample
else:
print('Tracker not connected')
def mainUpdate(): # main function to sample the two systems at time
Thread(target=update).start()
Thread(target=update2).start()
print('Recording starts at',time.time())
tObject=InfiniteTimer(0.04,mainUpdate) # Execute mainUpdate every 0.04 seconds (25 samples per second)
tObject.start()
time.sleep(5) # Total Capture time
tObject.cancel()
print('Recording ends at', time.time())
Results
I set a sample period of 0.04 seconds and a total capture time of 5 seconds; this should generate approximately 25 samples per second, that is, 125 samples during the entire capture.
However, I only obtained 102 samples with a sampling period that varies between 0.045 and 0.065 seconds.
Is there a way to sample both systems synchronously with a stable sampling period?
Thanks in advance.

How to run a function periodically in python

I have a simple metronome running and for some reason when it is at a lower bpm it is fine, but at higher bpms it is inconsistent and isnt steady.
I don't know what is going on.
I want to try using something to run it periodically. Is there a way to do that?
Here is my code:
class thalam():
def __init__(self,root,e):
self.lag=0.2
self.root=root
self.count=0
self.thread=threading.Thread(target=self.play)
self.thread.daemon=True
self.tempo=60.0/120
self.e=e
self.pause=False
self.tick=open("tick.wav","rb").read()
self.count=0
self.next_call = time.time()
def play(self):
if self.pause:
return
winsound.PlaySound(self.tick,winsound.SND_MEMORY)
self.count+=1
if self.count==990:
self.thread=threading.Thread(target=self.play)
self.thread.daemon=True
self.thread.start()
return
self.next_call+=self.tempo
new=threading.Timer(self.next_call-time.time(),self.play)
new.daemon=True
new.start()
def stop(self):
self.pause=True
winsound.PlaySound(None,winsound.SND_ASYNC)
def start(self):
self.pause=False
def settempo(self,a):
self.tempo=a
class Metronome(Frame):
def __init__(self,root):
Frame.__init__(self,root)
self.first=True
self.root=root
self.e=Entry(self)
self.e.grid(row=0,column=1)
self.e.insert(0,"120")
self.play=Button(self,text="Play",command=self.tick)
self.play.grid(row=1,column=1)
self.l=Button(self,text="<",command=lambda:self.inc("l"))
self.l.grid(row=0,column=0)
self.r=Button(self,text=">",command=lambda:self.inc("r"))
self.r.grid(row=0,column=2)
def tick(self):
self.beat=thalam(root,self.e)
self.beat.thread.start()
self.play.configure(text="Stop",command=self.notick)
def notick(self):
self.play.configure(text="Start",command=self.tick)
self.beat.stop()
def inc(self,a):
if a=="l":
try:
new=str(int(self.e.get())-5)
self.e.delete(0, END)
self.e.insert(0,new)
self.beat.settempo(60.0/(int(self.e.get())))
except:
print "Invalid BPM"
return
elif a=="r":
try:
new=str(int(self.e.get())+5)
self.e.delete(0, END)
self.e.insert(0,new)
self.beat.settempo((60.0/(int(self.e.get()))))
except:
print "Invalid BPM"
return

Playing sound to emulate an ordinary metronome doesn't require "real-time" capabilities.
It looks like you use Tkinter framework to create the GUI. root.after() allows you to call a function with a delay. You could use it to implement ticks:
def tick(interval, function, *args):
root.after(interval - timer() % interval, tick, interval, function, *args)
function(*args) # assume it doesn't block
tick() runs function with given args every interval milliseconds. Duration of individual ticks is affected by root.after() precision but in the long run, the stability depends only on timer() function.
Here's a script that prints some stats, 240 beats per minute:
#!/usr/bin/env python
from __future__ import division, print_function
import sys
from timeit import default_timer
try:
from Tkinter import Tk
except ImportError: # Python 3
from tkinter import Tk
def timer():
return int(default_timer() * 1000 + .5)
def tick(interval, function, *args):
root.after(interval - timer() % interval, tick, interval, function, *args)
function(*args) # assume it doesn't block
def bpm(milliseconds):
"""Beats per minute."""
return 60000 / milliseconds
def print_tempo(last=[timer()], total=[0], count=[0]):
now = timer()
elapsed = now - last[0]
total[0] += elapsed
count[0] += 1
average = total[0] / count[0]
print("{:.1f} BPM, average: {:.0f} BPM, now {}"
.format(bpm(elapsed), bpm(average), now),
end='\r', file=sys.stderr)
last[0] = now
interval = 250 # milliseconds
root = Tk()
root.withdraw() # don't show GUI
root.after(interval - timer() % interval, tick, interval, print_tempo)
root.mainloop()
The tempo osculates only by one beat: 240±1 on my machine.
Here's asyncio analog:
#!/usr/bin/env python3
"""Metronome in asyncio."""
import asyncio
import sys
async def async_main():
"""Entry point for the script."""
timer = asyncio.get_event_loop().time
last = timer()
def print_tempo(now):
nonlocal last
elapsed = now - last
print(f"{60/elapsed:03.1f} BPM", end="\r", file=sys.stderr)
last = now
interval = 0.250 # seconds
while True:
await asyncio.sleep(interval - timer() % interval)
print_tempo(timer())
if __name__ == "__main__":
asyncio.run(async_main())
See Talk: Łukasz Langa - AsyncIO + Music

Doing anything needing time precision is very difficult due to the need for the processor to share itself with other programs. Unfortunately for timing critical programs the operating system is free to switch to another process whenever it chooses. This could mean that it may not return to your program until after a noticeable delay. Using time.sleep after import time is a more consistent way of trying to balance the time between beeps because the processor has less "reason" to switch away. Although sleep on Windows has a default granularity of 15.6ms, but I assume you will not need to play a beat in excess 64Hz. Also it appears that you are using multithreading to try and address your issue, however, the python implementation of threading sometimes forces the threads to run sequentially. This makes matters even worse for switching away from your process.
I feel that the best solution would be to generate sound data containing the metronome beep at the frequency desired. Then you could play the sound data in a way the OS understands well. Since the system knows how to handle sound in a reliable manner your metronome would then work.
Sorry to disappoint but timing critical applications are VERY difficult unless you want to get your hands dirty with the system you are working with.

I would like to tell you that you can't be precise with threads in case of timing because of race conditions (even when you are using locks and semaphores!). Even I have faced the problem.

How do I schedule a one-time script to run x minutes from now? (alternative for 'at')

(Background: I'd like to control a light source with a motion sensor. Light should turn off x minutes after last detected motion. The framework is in place, scheduling is what remains to be done.)
Currently, when motion is detected the light gets turned on and a job to turn it off in 'now + x minutes' is scheduled. Whenever motion is detected during the x minutes the job gets removed from the queue and a new one is set up, extending effectively the time the light stays on.
I tried the "at" command but job handling is quite clunky. Whenever a job is removed from the queue an email gets sent. I looked at the Python crontab module but it would need much additional programming (handling relative time, removing old cronjobs, etc.) and seems to be slower.
What are my alternatives (bash, python, perl)?
-- Edit: My python skills are at beginner level, here's what I put together:
#!/usr/bin/env python2.7
# based on http://raspi.tv/2013/how-to-use-interrupts-with-python-on-the-raspberry-pi-and-rpi-gpio-part-2
# more than 160 seconds without activity are required to re-trigger action
import time
from subprocess import call
import os
import RPi.GPIO as GPIO
PIR = 9 # data pin of PIR sensor (in)
LED = 7 # positive pin of LED (out)
timestamp = '/home/pi/events/motiontime' # file to store last motion detection time (in epoch)
SOUND = '/home/pi/events/sounds/Hello.wav' # reaction sound
# GPIO setup
GPIO.setmode(GPIO.BCM)
GPIO.setup(PIR,GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.setup(LED,GPIO.OUT)
# function which gets called when motion is reported (sensor includes own delay-until-hot again
# and sensibility settings
def my_callback(channel):
now = time.time() # store current epoch time in variable 'now'
f = open(timestamp, "r")
then = float(f.readline()) # read last detection time from file
difference = now - then # calculate time that has passed
call(['/home/pi/bin/kitchenlights.sh', '-1']) # turn light on
call(['/home/pi/bin/lighttimer.sh']) # schedule at job to turn lights off
if difference > 160: # if more than 160 seconds without activity have passed then...
GPIO.output(LED, True) # turn on LED
if not os.path.isfile("/home/pi/events/muted"): # check if system is muted, else
call(['/usr/bin/mplayer', '-really-quiet', '-noconsolecontrols', SOUND]) # play sound
GPIO.output(LED, False) # turn of LED
f = open(timestamp, "w")
f.write(repr(now)) # update timestamp
f.close()
else: # when less than 160 seconds have passed do nothing and
f = open(timestamp, "w")
f.write(repr(now)) # update timestamp (thus increasing the interval of silence)
f.close()
GPIO.add_event_detect(PIR, GPIO.RISING,callback=my_callback,bouncetime=100) # add rising edge detection on a channel
while True:
time.sleep(0.2)
pass
Now that questions come in I think I could put a countdown in the while loop, right? How would that work?

I would approach this with the threading module. To do this, you'd set up the following thread class:
class CounterThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.count = 0
self.start()
def run(self):
while self.count < COUNTLIMIT:
time.sleep(0.1)
self.count += 0.1
#Call function to turn off light here
return
def newSig(self):
self.count = 0
This is a thread which everytime it recieves a new signal (the thread's newSig function is called), the counter restarts. If the COUNTLIMIT is reached (how long you want to wait in seconds), then you call the function to turn off the light.
Here's how you'd incorporate this into your code:
import threading
from subprocess import call
import os
import time
import RPi.GPIO as GPIO
PIR = 9 # data pin of PIR sensor (in)
LED = 7 # positive pin of LED (out)
SOUND = '/home/pi/events/sounds/Hello.wav' # reaction sound
COUNTLIMIT = 160
countThread = None
WATCHTIME = 600 #Run for 10 minutes
# GPIO setup
GPIO.setmode(GPIO.BCM)
GPIO.setup(PIR,GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.setup(LED,GPIO.OUT)
#------------------------------------------------------------
class CounterThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.count = 0
self.start()
def run(self):
call(['/home/pi/bin/kitchenlights.sh', '-1']) # turn light on
while self.count < COUNTLIMIT:
time.sleep(0.1)
self.count += 0.1
call(['/home/pi/bin/kitchenlights.sh', '-0'])
threadKiller()
return
def newSig(self):
self.count = 0
#------------------------------------------------------------
def my_callback(channel):
'''function which gets called when motion is reported (sensor includes own delay-until-hot again and sensibility settings'''
global countThread
try:
countThread.newSig()
except:
countThread = CounterThread()
#------------------------------------------------------------
def threadKiller():
global countThread
countThread = None
#------------------------------------------------------------
def main():
GPIO.add_event_detect(PIR, GPIO.RISING,callback=my_callback,bouncetime=100) # add rising edge detection on a channel
t = 0
while t < WATCHTIME:
t += 0.1
time.sleep(0.1)
#------------------------------------------------------------
if __name__ == "__main__": main()
I don't have any way to test this, so please let me know if there is anything that breaks. Since you said you're new to Python I made a few formatting changes to make your code a bit prettier. These things are generally considered to be good form, but are optional. However, you need to be careful about indents, because as you have them in your question, your code should not run (it will throw an IndentError)
Hope this helps

Python threading for wheel encoders on a Robot

I'm writing the code for a robot which my college is entering into a competition. I'm currently trying to build some wheel encoders using reflectance sensors. I realised a while back that I would probably need to use threading to achieve this, seeing as the robot needs to monitor both the left and right encoders at the same time. The code below is what I have so far:
from __future__ import division
import threading
import time
from sr import *
R = Robot()
class Encoder(threading.Thread):
def __init__(self, motor, pin, div=16):
self.motor = motor
self.pin = pin
self.div = div
self.count = 0
threading.Thread.__init__(self)
def run(self):
while True:
wait_for(R.io[0].input[self.pin].query.d)
self.count += 1
def rotations(self, angle, start_speed=50):
seg = 360/self.div
startcount = self.count
current_dist = angle #Distance away from target
R.motors[self.motor].target = start_speed
while current_dist > 360:
newcount = self.count - startcount
current_dist = angle - newcount*seg
R.motors[self.motor].target = 50
while abs(current_dist) > seg/2:
newcount = self.count - startcount
current_dist = angle - newcount*seg
current_speed = start_speed * current_dist / 360
if current_speed < 5:
R.motors[self.motor].target = 5
else:
R.motors[self.motor].target = current_speed
R.motors[self.motor].target = 0
WheelLeft = Encoder(0,0)
WheelLeft.start()
WheelRight = Encoder(1,3)
WheelRight.start()
WheelRight.rotations(720)
WheelLeft.rotations(720)
The sr module is provided by Southampton University, who are running the competition. It allows us to interact with the robot's hardware.
Now, the threads which get created seem to allow the two reflectance sensors to be monitored separately. This bit of code: R.io[0].input[self.pin].query.d works out whether the value coming from the reflectance sensor has changed. The 'rotations' method turns the wheel through a certain angle by constantly checking how many degrees the wheel has already turned through, and slowing it down as it reaches the end. I would like both wheels to start turning when I run the program, and then slow down and stop when they have gone through 2 rotations. Currently though, when I run the program, one wheel starts turning and slows down and stops, followed by the other wheel. It seems to me like the 'rotations' method is not running in a thread, like the 'run' method is. Is it only the code under the 'run' method that runs in a thread, or is it the whole class?
If it helps, I've been following this tutorial: http://www.devshed.com/c/a/Python/Basic-Threading-in-Python/1/
Also, I would like to know why it is possible to start a thread only with Encoder(0,0).start(). Why do you not have to create an object using the class (e.g. Thread = Encoder(0,0).start() for a new thread to be created?
Sorry if the terminoligy I've used isn't up to scratch, as you can probably tell I'm quite new to threading, and programming in general.

Encoder(0,0).start() is a call to the method to start the thread. In turn, this method calls your run implementation, which doesn't use the rotations method. If you want to do so, then you have to call it in the while loop of run.
With Thread = Encoder(0,0).start() you store the value retrieved from that call (which is None), but to get it you need to start the new thread first anyway.

The run method is the thread of execution.
If you want something else to happen in that thread, you have to call it from Encoder.run().
Oh, and Encoder(0,0).start() does create an object. Just because you didn't bind that object to a local variable doesn't mean it doesn't exist. If it didn't exist you couldn't call its start method.
You have to be very careful about its lifetime though, without a local variable keeping it alive.

You can extend from SR's Poll class so that it can be used in a wait_for:
import poll
class Encoder(poll.Poll):
def __init__(self, motor, pin, div=16):
self.motor = motor
self.pin = pin
self.div = div
self.count = 0
self.target_reached = False
# kick off a thread to count the encoder ticks
self.counter_thread = threading.Thread(target=self._update_count)
self.counter_thread.start()
def _update_count(self):
while True:
wait_for(R.io[0].input[self.pin].query.d)
self.count += 1
def rotations(self, angle, start_speed=50):
if not self.target_reached:
raise Exception("Last motion still in progress!")
self.target_reached = False
# kick off a thread to control the speed
self.angle_thread = threading.Thread(
target=self._update_speeds,
args=(angle, start_speed)
)
self.angle_thread.start()
def _update_speeds(self, angle, start_speed):
# control the motor speed as before
...
# let things know we're done
self.target_reached = True
# implement poll methods
def eval(self):
return (self.target_reached, None)
Which then lets you do:
wheelLeft = Encoder(0,0)
wheelRight = Encoder(1,3)
wheelRight.rotations(720)
wheelLeft.rotations(720)
wait_for(wheelRight & wheelLeft)
Note that an encoder isn't itself a thread - it's a "has a" relationship, not an "is a" relationship