Two simultaneous Python loops with one result - python

I currently have a Python 2.6 piece of code that runs two loops simultaneously. The code uses the gps (gpsd) module and the scapy module. Basically the first function (gpsInfo) contains a continual while loop grabbing GPS data from a GPS device and writing the location to console. The second function (ClientDetect) runs in a continual loop also sniffs the air for wifi data and prints this data when specific packets are found. I've threaded these two loops with the GPS one running as a background thread. What I am looking to do (and have been struggling for 5 days to work out how) is for, when the ClientDetect function finds a match and prints the respective info, I want the respective GPS coordinates of when that hit was made also printed to console. At present my code doesn't seem to work.
observedclients = [] p = "" # Relate to wifi packet session =
gps.gps(mode=gps.WATCH_NEWSTYLE)
def gpsInfo():
while True:
session.poll()
time.sleep(5)
if gps.PACKET_SET:
session.stream
print session.fix.latitude + session.fix.longitude
time.sleep(0.1)
def WifiDetect(p):
if p.haslayer(Dot11):
if p.type == 0 and p.subtype in stamgmtstypes:
if p.addr2 not in observedclients:
print p.addr2
observedclients.append(p.addr2)
def ActivateWifiDetect():
sniff(iface="mon0", prn=WifiDetect)
if __name__ == '__main__':
t = threading.Thread(target=gpsInfo)
t.start()
WifiDetect()
Can anybody look at my code to see how best to grab the data simultaneously for when there is a wifi hit, for the GPS coordinates to be printed too? Somebody mentioned implementing queuing but I have researched this but to no avail with regards to how to implement it.
As said, the aim of this code is to scan for both GPS and specific wifi packets and when detected, print details relating to the packet and the location where is was detected.


A simple way of getting this is store the gps location in a global variable, and have the wifi sniffing thread read that global when it needs to print some data; The gotcha is that since two threads can be accessing the global variable at the same time, you'll want to wrap it with a mutex;
last_location = (None, None)
location_mutex = threading.Lock()
def gpsInfo():
global last_location
while True:
session.poll()
time.sleep(5)
if gps.PACKET_SET:
session.stream
with location_mutex:
# DON'T Print from inside thread!
last_location = session.fix.latitude, session.fix.longitude
time.sleep(0.1)
def WifiDetect(p):
if p.haslayer(Dot11):
if p.type == 0 and p.subtype in stamgmtstypes:
if p.addr2 not in observedclients:
with location_mutex:
print p.addr2, last_location
observedclients.append((p.addr2, last_location))


you need to tell python you are using an external variable when you use gps in a function. The code should look like this:
def gpsInfo():
global gps # new line
while True:
session.poll()
time.sleep(5)
if gps.PACKET_SET:
session.stream
print session.fix.latitude + session.fix.longitude
time.sleep(0.1)
def WifiDetect(p):
global p, observedclients # new line
if p.haslayer(Dot11):
if p.type == 0 and p.subtype in stamgmtstypes:
if p.addr2 not in observedclients:
print p.addr2
observedclients.append(p.addr2)


I think you should be more specific in your goal.
If all you want to do is get GPS coords when a Wifi network is sniffed, just do (psuedo-code):
while True:
if networkSniffed():
async_GetGPSCoords()
If you want a log of all GPS coords and want to match that up with Wifi network data, just print all the data out along with timestamps, and do post-processing do match up Wifi networks with GPS coords via timestamp.

Related

Python-CAN script receiving half of the expected CAN messages

I have written a Python script utilizing the Python-CAN library which records received CAN messages at a 1 second rate for 5 minutes, before logging all the messages into a file and exiting. The computer has a CAN module which is connecting to the CAN bus. (The other device on the bus is an engine) I communicate with it using the SocketCAN interface.
The test engine system that this computer is connected to is sending around 114 messages at what I believe is a 250kb baud rate. I am expecting to see 114 messages recorded in the file for each 1 second period, but instead I'm seeing about half that count. (~65 messages)
Could it be possible that the engine's ECU is set to a 500kb baud rate, and that's why I'm not getting the count I am expecting? I would think there would be no communication if the baud rates do not match, but I do not have physical access to the system because I'm sending the script remotely through an OTA update and not running it myself. (The device is headless, but is setup to run the script on startup) I just see the log files that are generated.
Here is the python code:
(A note, I have code parsing the received messages into the contained signals, but I did not include this code here because it happens at the end, and it is not relevant)
class logging:
def __init__(self):
#Dictionary to hold received CAN messages
self.message_Dict = {}
#List to hold queued dictionaries
self.message_Queue = []
#A "filters" object is also created here, but I did not include it
#I have verified the filters are correct on my test system
def main(self):
#Record the current time
currentTime = datetime.datetime.now()
#Record the overall start time
startTime = datetime.datetime.now()
#Record the iteration start time
lastIterationStartTime = currentTime
#Create the CanBus that will be used to send and receive CAN msgs from the MCU
canbus = can.interfaces.socketcan.SocketcanBus(channel='can0', bitrate=250000)
#These filters are setup correctly, because all the messages come through
#on my test system, but I did not include them here
canbus.set_filters(self.Filters)
# Creating Listener filters and notifier
listener = can.Listener()
#Main loop
while 1:
#create a variable to hold received data
msg2 = canbus.recv()
#Record the current time
currentTime = datetime.datetime.now()
#If a valid message is detected
if(msg2 != None):
if(len(msg2.data) > 0):
try:
#Save the message data into a queue (will be processed later)
self.message_Dict[msg2.arbitration_id] = msg2.data
except:
print("Error in storing CAN message")
#If 1 second has passed since the last iteration,
#add the dictionary to a new spot in the queue
if((currentTime - lastIterationStartTime) >= datetime.timedelta(seconds=1)):
#Add the dictionary with messages into the queue for later processing
messageDict_Copy = self.message_Dict.copy()
self.message_Queue.append(messageDict_Copy)
print("Number of messages in dictionary: " + str(len(self.message_Dict)) + "
Number of reports in queue: " + str(len(self.message_Queue)))
#Clear the dictionary for new messages for every iteration
self.message_Dict.clear()
#Record the reset time
lastIterationStartTime = datetime.datetime.now()
#Once 5 minutes of data has been recorded, write to the file
if((currentTime - startTime) > datetime.timedelta(minutes=5)):
#Here is where I write the data to a file. This is too long to include
#Clear the queue
self.message_Queue = []
#Clear the dictionary for new messages for every iteration
self.message_Dict.clear()
#When the script is run, execute the Main method
if __name__ == '__main__':
mainClass = logging()
mainClass.main()
I appreciate any ideas or input you have. Thank you

In my experience, most of the engine's ECU usually uses 250kb, but the newest ones are using 500kb. I would suggest you too also try the both.
Also, the messages will only come to the bus if they have been sent, it seems silly but for example a truck, if you don't step on the accelerator the messages referring to the accelerator will not appear. So, maybe you need to check if all components are being using as you expect. The lib of can-utils has a 'Can sniffer' that can also help you.
I suggest you to use 'can-utils' to help you in that. It is a powerful tool to can analyses.
Did you try to loop the baudrate? Maybe can also help to find another.

Serial Communication between MATLAB and STM running Micropython

Was hoping I could get some pointers on how to setup communications between MATLAB and an STM32L476 Nucleo-64.
The idea behind this is to essentially navigate some pre generated terrain, using the MATLAB script to create the map of the terrain and show position, while the STM sends in the control action to act as an 'onboard' processor for a robot navigating the path.
The MATLAB script is set up to generate 3 numbers (1 d.p.), corresponding to sensor measurements, which is then sent to the STM via serial communication, the STM will then calculate the control action, which should correspond to two integers returned.
This setup does work using an Arduino as this is what was used previously (so I know there is no issues with MATLAB generating the sensor measurements or terrain), however I need to change the hardware over to STM running micropython, which is where I am running into the trouble.
Am hoping that someone may be able to point me in the right direction, I have used the serial command in MATLAB to write to the board:
s = serial('COM3','BaudRate',115200,'timeout',1);
fopen(s);
.....
fprintf(s,y(1));
fprintf(s,y(2));
fprintf(s,y(3));
Where y is the sensor measurements. I was then was hoping to read the response from the STM back in using fread(s) - this does give numbers for each iteration, however they are quite often repeated and are no where close to what was set on the STM (it also returns a 14x1 array when it should only be a 2x1). I think my mistake maybe in the STM side of things as I tried to read the data in by setting up another serial port.
ser = serial.Serial(
port='COM3',
baudrate=115200)
dat = []
while True:
# Collect sensor data
yLeft = (dat[1])
yRight = (dat[2])
yFront = (dat[3])
Any input would be greatly appreciated - I just seem to be going around in circles at the moment.
Edit - At the moment to try and get it to work I am trying to get it to react to one number and print the result to MATLAB
import serial
import time
import sys
# Declare states
state_list = ['WAIT', 'TEN']
state = 'WAIT' # Default state on init is wait
# Initialise variables
V = 0
ser = serial.Serial(
port='COM3',
baudrate=115200)
while True:
ser.open()
V = ser.read()
ser.close()
# Make decisions according to state machine
if state == 'WAIT':
print('WAITING')
if V > 10:
state = 'TEN'
else:
state = state
elif state == 'TEN':
print('TEN')
if V < 10:
state = 'WAIT'
else:
state = state
else:
print('END OF PROGRAM')
Then the corresponding MATLAB script is:
%% clean up
clear all
clc
%% Set up serial
sobj = 'COM3';
s = serial(sobj,'BaudRate',115200,'timeout',1);
fopen(s);
%% Run Simulation
% Create Data Array to be sent to STM
DatatoSend=15;
% Print data to STM
fprintf(s,DatatoSend)
% Read Response from STM
data = [];
% Receive response data
data = fread(s);
fclose(s)
I am not confident that I am even close to getting this working though.

Problem with MultiPort use in python mido MIDI package

I am working in a program that autosaves MIDI messages as they are played, and everything is working really well when using a single input port.
I want to extend to use 2 or 3 simultaneous ports (the app is aimed at VST piano players, and some use the piano as one MIDI device, and the sustain pedal as a second device in another USB slot). I was able to get it to work using two separate input ports, and then using 2 iter_pending() loops. The basic code is something like this:
ports = mido.get_input_names()
port_A = mido.open_input(ports[0])
port_B = mido.open_input(ports[1])
for msg_A in port_A.iter_pending():
...
for msg_B in port_B.iter_pending():
...
The problem with this is that the loops are sequential. When I test by sending the same midi message through the 2 ports, the event when enters through port_B is processed with a few milliseconds of delay.
The MIDO package includes a different type of port exactly for this: mido.ports.MultiPort()
My problem is that I cannot make it to work.
If I try:
multi = mido.ports.MultiPort([port_A, port_B])
for msg in multi:
print (msg)
as suggested in the MIDO documentation, no msg is printed...
I have done all sorts of syntax combinations for the ports in multi, but nothing seems to work.
What am I doing wrong here?

Have you tried:
open_ports = [portA, portB]
while True:
for port in open_ports:
do_whatever(port.poll())
or to make it asynchronous:
portA = mido.open_input(ports[0], callback=do_whatever)
portB = mido.open_input(ports[1], callback=do_whatever)
while True:
pass
In both cases, do_whatever is a function that takes a single message as an argument and does with that message whatever it is you want to do.

Continuing a Loop and Moving On (Python)

I have a 'while' loop in my Python app (v2.7) which basically communicates with a Bluetooth GPS device and the loop continues for as long as there is data being received. In essence, the while loop looks like this:-
> data = ""
>
> if data == None:
> print "Connection Failed" else:
> print "Connection Established"
>
> while True:
> data = socket.recv(1024)
Now what I want to do is leave this while loop running continually throughout the life of the program until the bluetooth device is switched off which in turn will end the loop as no data will be being received. Nonetheless, as this while loop continues, I then want to move onto another method which will parse the data.
How can I leave the loop running and move on to the next command? Is threading required? I'm new to the concept of threading and so please be patient with me :)
Thanks

Yeah, because you're essentially going to be doing two things at the same time, you'll have to create a new thread (using threading) or perhaps more simply a new process (using multiprocessing or just os.fork)
The easiest thing to do is to put this part in a function, and use multiprocessing or threading to start the function in a different process. Typically you'll have less weirdness with multiple processes than multiple threads, just FYI.

received = ""
while True:
data = socket.recv(1024)
if not data:
break # ends the while loop
received+=data
do_stuff(received)

Python GPS Module: Reading latest GPS Data

I have been trying to work with the standard GPS (gps.py) module in python 2.6. This is supposed to act as a client and read GPS Data from gpsd running in Ubuntu.
According to the documentation from GPSD webpage on client design (GPSD Client Howto), I should be able to use the following code (slightly modified from the example) for getting latest GPS Readings (lat long is what I am mainly interested in)
from gps import *
session = gps() # assuming gpsd running with default options on port 2947
session.stream(WATCH_ENABLE|WATCH_NEWSTYLE)
report = session.next()
print report
If I repeatedly use the next() it gives me buffered values from the bottom of the queue (from when the session was started), and not the LATEST Gps reading. Is there a way to get more recent values using this library? In a Way, seek the Stream to the latest values?
Has anyone got a code example using this library to poll the gps and get the value i am looking for ?
Here is what I am trying to do:
start the session
Wait for user to call the gps_poll() method in my code
Inside this method read the latest TPV (Time Position Velocity) report and return lat long
Go back to waiting for user to call gps_poll()

What you need to do is regularly poll 'session.next()' - the issue here is that you're dealing with a serial interface - you get results in the order they were received. Its up to you to maintain a 'current_value' that has the latest retrieved value.
If you don't poll the session object, eventually your UART FIFO will fill up and you won't get any new values anyway.
Consider using a thread for this, don't wait for the user to call gps_poll(), you should be polling and when the user wants a new value they use 'get_current_value()' which returns current_value.
Off the top of my head it could be something as simple as this:
import threading
import time
from gps import *
class GpsPoller(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.session = gps(mode=WATCH_ENABLE)
self.current_value = None
def get_current_value(self):
return self.current_value
def run(self):
try:
while True:
self.current_value = self.session.next()
time.sleep(0.2) # tune this, you might not get values that quickly
except StopIteration:
pass
if __name__ == '__main__':
gpsp = GpsPoller()
gpsp.start()
# gpsp now polls every .2 seconds for new data, storing it in self.current_value
while 1:
# In the main thread, every 5 seconds print the current value
time.sleep(5)
print gpsp.get_current_value()

The above answers are very inefficient and overly complex for anyone using modern versions of gpsd and needing data at only specific times, instead of streaming.
Most GPSes send their position information at least once per second. Presumably since many GPS-based applications desire real-time updates, the vast majority of gpsd client examples I've seen use the above method of watching a stream from gpsd and receiving realtime updates (more or less as often as the gps sends them).
However, if (as in the OP's case) you don't need streaming information but just need the last-reported position whenever it's requested (i.e. via user interaction or some other event), there's a much more efficient and simpler method: let gpsd cache the latest position information, and query it when needed.
The gpsd JSON protocol has a ?POLL; request, which returns the most recent GPS information that gpsd has seen. Instead of having to iterate over the backlog of gps messages, and continually read new messages to avoid full buffers, you can send a ?WATCH={"enable":true} message at the start of the gpsd session, and then query the latest position information whenever you need it with ?POLL;. The response is a single JSON object containing the most recent information that gpsd has seen from the GPS.
If you're using Python3, the easiest way I've found is to use the gpsd-py3 package available on pypi. To connect to gpsd, get the latest position information, and print the current position:
import gpsd
gpsd.connect()
packet = gpsd.get_current()
print(packet.position())
You can repeat the gpsd.get_current() call whenever you want new position information, and behind the scenes the gpsd package will execute the ?POLL; call to gpsd and return an object representing the response.
Doing this with the built-in gps module isn't terribly straightforward, but there are a number of other Python clients available, and it's also rather trivial to do with anything that can perform socket communication, including this example using telnet:
$ telnet localhost 2947
Trying ::1...
Connected to localhost.
Escape character is '^]'.
{"class":"VERSION","release":"3.16","rev":"3.16","proto_major":3,"proto_minor":11}
?WATCH={"enable":true}
{"class":"DEVICES","devices":[{"class":"DEVICE","path":"/dev/pts/10","driver":"SiRF","activated":"2018-03-02T21:14:52.687Z","flags":1,"native":1,"bps":4800,"parity":"N","stopbits":1,"cycle":1.00}]}
{"class":"WATCH","enable":true,"json":false,"nmea":false,"raw":0,"scaled":false,"timing":false,"split24":false,"pps":false}
?POLL;
{"class":"POLL","time":"2018-03-02T21:14:54.873Z","active":1,"tpv":[{"class":"TPV","device":"/dev/pts/10","mode":3,"time":"2005-06-09T14:34:53.280Z","ept":0.005,"lat":46.498332203,"lon":7.567403907,"alt":1343.165,"epx":24.829,"epy":25.326,"epv":78.615,"track":10.3788,"speed":0.091,"climb":-0.085,"eps":50.65,"epc":157.23}],"gst":[{"class":"GST","device":"/dev/pts/10","time":"1970-01-01T00:00:00.000Z","rms":0.000,"major":0.000,"minor":0.000,"orient":0.000,"lat":0.000,"lon":0.000,"alt":0.000}],"sky":[{"class":"SKY","device":"/dev/pts/10","time":"2005-06-09T14:34:53.280Z","xdop":1.66,"ydop":1.69,"vdop":3.42,"tdop":3.05,"hdop":2.40,"gdop":5.15,"pdop":4.16,"satellites":[{"PRN":23,"el":6,"az":84,"ss":0,"used":false},{"PRN":28,"el":7,"az":160,"ss":0,"used":false},{"PRN":8,"el":66,"az":189,"ss":45,"used":true},{"PRN":29,"el":13,"az":273,"ss":0,"used":false},{"PRN":10,"el":51,"az":304,"ss":29,"used":true},{"PRN":4,"el":15,"az":199,"ss":36,"used":true},{"PRN":2,"el":34,"az":241,"ss":41,"used":true},{"PRN":27,"el":71,"az":76,"ss":42,"used":true}]}]}
?POLL;
{"class":"POLL","time":"2018-03-02T21:14:58.856Z","active":1,"tpv":[{"class":"TPV","device":"/dev/pts/10","mode":3,"time":"2005-06-09T14:34:53.280Z","ept":0.005,"lat":46.498332203,"lon":7.567403907,"alt":1343.165,"epx":24.829,"epy":25.326,"epv":78.615,"track":10.3788,"speed":0.091,"climb":-0.085,"eps":50.65,"epc":157.23}],"gst":[{"class":"GST","device":"/dev/pts/10","time":"1970-01-01T00:00:00.000Z","rms":0.000,"major":0.000,"minor":0.000,"orient":0.000,"lat":0.000,"lon":0.000,"alt":0.000}],"sky":[{"class":"SKY","device":"/dev/pts/10","time":"2005-06-09T14:34:53.280Z","xdop":1.66,"ydop":1.69,"vdop":3.42,"tdop":3.05,"hdop":2.40,"gdop":5.15,"pdop":4.16,"satellites":[{"PRN":23,"el":6,"az":84,"ss":0,"used":false},{"PRN":28,"el":7,"az":160,"ss":0,"used":false},{"PRN":8,"el":66,"az":189,"ss":45,"used":true},{"PRN":29,"el":13,"az":273,"ss":0,"used":false},{"PRN":10,"el":51,"az":304,"ss":29,"used":true},{"PRN":4,"el":15,"az":199,"ss":36,"used":true},{"PRN":2,"el":34,"az":241,"ss":41,"used":true},{"PRN":27,"el":71,"az":76,"ss":42,"used":true}]}]}

Adding my two cents.
For whatever reason my raspberry pi would continue to execute a thread and I'd have to hard reset the pi.
So I've combined sysnthesizerpatel and an answer I found on Dan Mandel's blog here.
My gps_poller class looks like this:
import os
from gps import *
from time import *
import time
import threading
class GpsPoller(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.session = gps(mode=WATCH_ENABLE)
self.current_value = None
self.running = True
def get_current_value(self):
return self.current_value
def run(self):
try:
while self.running:
self.current_value = self.session.next()
except StopIteration:
pass
And the code in use looks like this:
from gps_poll import *
if __name__ == '__main__':
gpsp = GpsPoller()
try:
gpsp.start()
while True:
os.system('clear')
report = gpsp.get_current_value()
# print report
try:
if report.keys()[0] == 'epx':
print report['lat']
print report['lon']
time.sleep(.5)
except(AttributeError, KeyError):
pass
time.sleep(0.5)
except(KeyboardInterrupt, SystemExit):
print "\nKilling Thread.."
gpsp.running = False
gpsp.join()
print "Done.\nExiting."
You can also find the code here: Here and Here

I know its an old thread but just for everyone understanding, you can also use pyembedded python library for this.
pip install pyembedded
from pyembedded.gps_module.gps import GPS
import time
gps = GPS(port='COM3', baud_rate=9600)
while True:
print(gps.get_lat_long())
time.sleep(1)
https://pypi.org/project/pyembedded/

Categories