I am new to socket programing and SW architecture.
My system must be : a GUI in my laptop using python. There are many embedded systems with same sensors ( GPS, Temperature, pressure ...). Each time you select an embedded system, my program needs to establish a connection with it , I need to show its GPS position and the real time feed of its sensors in the GUI ( For now the GUI is not the problem, I can do with Kivy or Tkinter).
This is how it must function :
In the GUI, there is a field to enter the ID of embedded system and a button to try to connect with it.
When the button is clicked, the program establishes connection and shows GPS, Temperature and pressure in real time continuously until connection is lost.
I was thinking of doing it with this architecture :
A thread to deal with the GUI
Each time a button is clicked and an embedded system is found, an object of a class I created is instantiated.
The class has as attributes :
list GPS ( to store GPS feed)
list temperature ( to store Temperature feed)
list pressure
a thread_socket ( the socket is created in a thread to be a client to the embedded system. So each time an object is instantiated of the class, a separate socket is create )
The class has as methods :
Get_Gps() : Each time this method is called the GPS list attribute is updated
Get_Temperature() / Pressure()
Stop() : When this method is called the embedded system needs to shutdown.
In the socket thread, I have methods such as send_message() and receive_message() to send through TCP/IP the request for getting GPS and sensor data or stopping the system.
On each embedded system I will put a server using python that is set up everytime the system starts.
This way the ID of the system is the ip of the server, And my laptop would be a client, searching for the ip adress when I select a system.
My questions are :
Does this architecture seem alright to you ?
Is it correct to receive real time feed in a list ? for example for the gps.
Each time I find a system I instanciate an object to keep things clean, is this a good way to do it?
Do you see any issues or improvements ?
Thank you in advance,
I think your approach in general is fine.
However, you should keep a few things in mind:
When designing your software, you should first identify the different tasks involved and define separate functional units for each task. This is the concept of separation of concerns.
I also suggest to read a bit on the Model-View-Controller (MVC) pattern: In your case, the model would be your class containing the data structure for the measurements and the business logic (e.g. polling data from a source for example every second until the connection is stopped). The view and the controller might both be located in the GUI (which is absolutely fine).
The GUI is not necessarily an explicit thread, but many frameworks rather work with an event-based concept that lets you define the application's behavior for a given user interaction.
Why do you actually need lists for the measurements? Is there a requirement to keep the history of measurements over a certain period of time? Is this a list that will keep growing and growing or rather a rolling list (e.g. for showing the last n seconds/minutes of measurements in the GUI)? There seems a bit of a contradiction to starting a new class instance with every new connection, because you would obviously loose the contents when you stop the connection and terminate the instance.
Hope this gives you some ideas of how to proceed from there.
Related
I am currently trying to implement simple communication between an I/O module as a CanOpen slave and my computer(Python script). The I/O module is connected to my computer with a PEAK USB-CAN adapter.
My goal would be to read or write the inputs/outputs. Is this even possible with the hardware, since I don't have a real "master" from that point of view?
Unfortunately I don't know what else I have to do to be able to communicate correctly with my I/O module.
import canopen
import time
network = canopen.Network()
network.connect(bustype='pcan', channel='PCAN_USBBUS1', bitrate=500000)
#add node and DCF File
IO_module = network.add_node(1, 'path to my DIO.DCF')
network.add_node(IO_module)
IO_module.nmt.state = 'RESET COMMUNICATION' # 000h 82 01
print(IO_module.nmt.state)
time.sleep(5)
IO_module.nmt.state = 'OPERATIONAL'
print(IO_module.nmt.state)
for node_id in network:
print(network[node_id])
IO_module.load_configuration()
i see some kind of communication in my console with timeout errors
INITIALISING
OPERATIONAL
<canopen.node.remote.RemoteNode object at 0x000002A023493A30>
Transfer aborted by client with code 0x05040000
No SDO response received
Transfer aborted by client with code 0x05040000
No SDO response received
Any advices ?
I can't get any further with the documentation alone
https://canopen.readthedocs.io/en/latest/
thank you
The good news is, you probably have all the required hardware. You are doing the "master" part from Python, that's fine. (The CAN bus isn't really master/slave, just broadcast. CANopen can be master/slave sometimes, but it's still all broadcast messages among equals on the same bus.)
You haven't provided information about your device, but I would start checking at a lower level.
Do you even have the CAN-Bus wired up correctly, and if so, what did you do to verify it? (Most common mistake: CAN-Bus not terminated with two 120ohm resistors. Though you usually can get away with just one instead of two.) And have you verified that you are using the correct baud rate?
The library docu example suggests to wait for the heartbeat with node.nmt.wait_for_heartbeat(). Why are you using a sleep instead? If there is no heartbeat, you don't need to continue. (Unless the device docu says that it doesn't implement NMT heartbeat - would be unusual.)
I certainly wouldn't try to go ahead with SDOs if you cannot confirm a NMT heartbeat. Also, some devices don't implement SDOs but only PDOs.
Try sniffing the CAN bus at a lower level (e.g. not PDOs/SDOs but just print the raw messages received - from Python, or with a separate application - e.g. candump on Linux.) Try getting statistics of the CAN "network" interface (on Linux, e.g. ifconfig). If everything is okay, the adapter should be in state "ERROR-ACTIVE", and you should see the frame counter increase for frames you've sent via Python.
I am a newbie to python and I am learning new things day by day. I have a question regarding integrating wxpython and pyserial. I am writing a GUI application to control a microprocessor through pyserial.
I have a wxpython script written - displays good - with buttons and text fields.
I tested communication with my microprocessor using small commands from pyserial - everything is in good place.
Problem:
I will be having a button (say Button A) on my GUI, which after clicked - checks if the serial communication is made (by sending and receiving data ofcourse). Once the communication is good, I have to make sure the communication stays good as long as I am using my GUI. So I decided to write an external function which continuously sends and reads data (probably a for loop). Based on the functions return value I will know if my serial communication is active or not (this might be a bad idea - but thats the best I got)
Now the problem is I have a lot of other features on my GUI, buttons, text fields etc.. So for example when another button (say button B) is pressed I want to send a specific command to the microprocessor. This requires I interrupt the serial communication which was going on in my Step 1, send data from button B click, then re-start the Step 1 communication again (to keep checking my serial communication is active). I dont know how I can interrupt the communication. The Step 1 serial communication (for loop) is bound to the Button A click. Once the Button A is clicked, it goes to a for loop and serial communication is checked continuously.
I have so many buttons and text fields like this - which are going to read and write data to the Microprocessor. Whenever I want to do an event, I have to stop the serial communication in step 1 and restart it again.
On top of all this, I can only check the serial communication (mentioned in Step 1) every 100ms. I cannot just write a for loop. I have to do some modifications - like time.delay(100ms) or something.
I dont know how to frame it, but may be I just require a good algorithm idea or implement this somehow with help of import sched or import thread
I am trying majorly to avoid import thread - because my microprocessor has very minimal RAM. Also using threading with wxPython is pain in the neck (what I read online)
One of my colleagues suggested using "timer service" from my Operating system. I dont think python have a feature like that. I have no clue what he is talking about, at the least. His argument is that, if I can use this, I can run the continuous serial communication check every 100ms very easily.
Any help would be greatly appreciated. I am not looking for any complicated solutions, I appreciate if you attach a piece of code, use very basic programming. I have the wxPython GUI in a single class.
Hope you're well and thanks for reading.
Been revisiting an old project, leveraging plotly to stream data out of mysql with python in-between the two. I've never had great luck w/ plot.ly (which I'm sure relates more to my understanding than their platform), streams/iframes seem to stall over time and I am not apt enough to troubleshoot completely.
My current symptom is this: Plots arbitrarily stall - I'm pushing data, but the iframe isn't updating.
The current solution is: Refresh the browser every X minutes.
The solution works, but it's aggrevating, because I dont understand why the visual is stalling in the first place (is it me, is it them, etc).
As I was reviewing some of the documentation, specifically this link:
https://plot.ly/streaming/
I noticed they call out NOT continually opening and closing streams, and that heartbeats should be placed every so often to keep things alive/fresh.
Here's what I'm currently calling every 10 minutes:
pullData(mysql)
format data
open(plotly.stream1)
write data to plotly.stream1
close(plotly.stream1)
open(plotly.stream2)
write data to plotly.stream2
close(plotly.stream2)
Based on what I am reading, it sounds like I should actually execute the script once on startup, and keep the streams open, but heartbeat() them every 15 or-so seconds between actual write() calls like this:
open(plotly.stream1)
open(plotly.stream2)
every 10 minutes:
pullData(mysql)
format data
write data to plotly.stream1
write data to plotly.stream2
while not pulling and writing:
every 15 seconds:
heartbeat(plotly.stream1)
heartbeat(plotly.stream2)
if error:
close(plotly.stream1)
close(plotly.stream2)
Please excuse the sudo-mess, I'm just trying to convey an idea. Anyone have any advice? I started on my original path of opening, writing, closing based on the streaming example, but that's a one time write. The other example is a constant stream of data. I'm somewhere in between those two.
Furthermore - is this train of thought even related to the iframe not refreshing? Part of me believes the symptom is unrelated to my idea - the data is getting to plot.ly fine - it's my session that's expiring, or the iframe "connection" that's going stale. If the symptom is unrelated, at least I'll have made my source code a bit cleaner and more appropriate.
Any advice is greatly appreciated!
Thanks
-justin
Plotly will close a stream that is inactive for more than 60 seconds. You must send a newline down the streaming channel (a heartbeat) to keep it open. I recommend every 30 seconds.
Your first code example may not work as expected because the client side websocket (that connects the Plot to our system) may close when your first source stream (the stream that connects your script to our system) exits. When you disconnect a source stream a signal is sent to our system that lets it know your stream is now inactive. If a new source stream does not reconnect quickly we close the client connecting websockets.
Now, when your script gets more data and opens a new stream it will successfully stream data to our system but the client-side websocket, now closed, will not pass the data to the Plot. We will cache a certain amount of points for you behind the scenes so that when you refresh the page the websocket reconnects and you get the last n points (where n is set by max-points in the API call).
This is why sending the heartbeat is important. We keep the source stream open and that in turn ensures that all the connected Clients keep their websockets open.
This isn't necessarily the most robust behaviour for a streaming platform to have and we will likely make it better in the future. For now though you will likely see better results by attempting to implement the code in your second example.
Hope that helped!
I apologize in advance for this being a bit vague, but I'm trying to figure out what the best way is to write my program from a high-level perspective. Here's an overview of what I'm trying to accomplish:
RasPi takes input from altitude sensor on serial port at 115000 baud.
Does some hex -> dec math and updates state variables (pitch, roll, heading, etc)
Uses pygame library to do some image manipulation based on the state variables on a simulated heads up display
Outputs the image to a projector at 30 fps.
Note that there's no user input (for now).
The issue I'm running into is the framerate. The framerate MUST be constant. I'd rather skip a data packet than drop a frame.
There's two ways I could see structuring this:
Write one function that, when called, grabs data from the serial bus and spits out the state variables as the output. Then write a pygame loop that calls this function from inside it. My concern with this is that if the serial port starts being read at the end of an attitude message, it'll have to pause and wait for the message to start again (fractions of a second, but could result in a dropped frame)
Write two separate modules, both to be running simultaneously. One continuously reads data from the serial port and updates the state variables as fast as possible. The other just does the image manipulation, and grabs the latest state variables when it needs them. However, I'm not actually sure how to write a multithreaded program like this, and I don't know how well the RasPi will handle such a program.
I don't think that RasPi would work that well running multithreaded programs. Try the first method, though it would be interesting to see the results of a multithreaded program.
I have a long running python process running headless on a raspberrypi (controlling a garden) like so:
from time import sleep
def run_garden():
while 1:
/* do work */
sleep(60)
if __name__ == "__main__":
run_garden()
The 60 second sleep period is plenty of time for any changes happening in my garden (humidity, air temp, turn on pump, turn off fan etc), BUT what if i want to manually override these things?
Currently, in my /* do work */ loop, i first call out to another server where I keep config variables, and I can update those config variables via a web console, but it lacks any sort of real time feel, because it relies on the 60 second loop (e.g. you might update the web console, and then wait 45 seconds for the desired effect to take effect)
The raspberryPi running run_garden() is dedicated to the garden and it is basically the only thing taking up resources. So i know i have room to do something, I just dont know what.
Once the loop picks up the fact that a config var has been updated, the loop could then do exponential backoff to keep checking for interaction, rather than wait 60 seconds, but it just doesnt feel like that is a whole lot better.
Is there a better way to basically jump into this long running process?
Listen on a socket in your main loop. Use a timeout (e.g. of 60 seconds, the time until the next garden update should be performed) on your socket read calls so you get back to your normal functionality at least every minute when there are no commands coming in.
If you need garden-tending updates to happen no faster than every minute you need to check the time since the last update, since read calls will complete significantly faster when there are commands coming in.
Python's select module sounds like it might be helpful.
If you've ever used the unix analog (for example in socket programming maybe?), then it'll be familiar.
If not, here is the select section of a C sockets reference I often recommend. And here is what looks like a nice writeup of the module.
Warning: the first reference is specifically about C, not Python, but the concept of the select system call is the same, so the discussion might be helpful.
Basically, it allows you to tell it what events you're interested in (for example, socket data arrival, keyboard event), and it'll block either forever, or until a timeout you specify elapses.
If you're using sockets, then adding the socket and stdin to the list of events you're interested in is easy. If you're just looking for a way to "conditionally sleep" for 60 seconds unless/until a keypress is detected, this would work just as well.
EDIT:
Another way to solve this would be to have your raspberry-pi "register" with the server running the web console. This could involve a little bit extra work, but it would give you the realtime effect you're looking for.
Basically, the raspberry-pi "registers" itself, by alerting the server about itself, and the server stores the address of the device. If using TCP, you could keep a connection open (which might be important if you have firewalls to deal with). If using UDP you could bind the port on the device before registering, allowing the server to respond to the source address of the "announcement".
Once announced, when config. options change on the server, one of two things usually happen:
A) You send a tiny "ping" (in the general sense, not the ICMP host detection protocol) to the device alerting it that config options have changed. At this point the host would immediately request the full config. set, acquiring the update with it.
B) You send the updated config. option (or maybe the entire config. set) back to the device. This decreases the number of messages between the device and server, but would probably take more work as it seems like more a deviation from your current setup.
Why not use an event based loop instead of sleeping for a certain amount of time.
That way your loop will only run when a change is detected, and it will always run when a change is detected (which is the point of your question?).
You can do such a thing by using:
python event objects
Just wait for one or all of your event objects to be triggered and run the loop. You can also wait for X events to be done, etc, depending if you expect one variable to be updated a lot.
Or even a system like:
broadcasting events