We Keep Coding

Telemetry data through python socket, without stopping execution of the program

I'm building photovoltaic motorized solar trackers. They're controlled by Raspberry Pi's running python script. RPI's are connected to my public openVPN server for remote control and continuous software development. That's working fine. Recently a passionate customer asked me for some sort of telemetry data for his tracker - let's say, it's current orientation, measured wind speed etc.. By being new to python, I'm really struggling with this part.
I've decided to use socket approach from guides like this. Python script listens on a socket, and my openVPN server, which is also web server, connects to it using PHP fsockopen. Python sends telemetry data, PHP makes it user friendly and displays it on the web. Everything so far works, however I don't know how to design my python script around it.
The problem is, that my script has to run continuously, and socket.accept() halts it's execution, waiting for a connection. Didn't find any obvious solution on the web. Would multi-threading work for this? Sounds a bit like overkill.
Is there a way to run socket listening asynchronously? Like, for example, pigpio callback's which I'm using abundantly?
Or alternatively, is there a better way to accomplish my goal?
I tried with remote accessing status file that my script is maintaining, but that proved to be extremely involved with setup and prone to errors when the file was being written.
I also tried running the second script. Problem is, then I have no access to relevant data, or I need to read beforementioned status file, and that leads to the same problems as above.
Relevant bit of code is literally only this:
# Main loop
try:
while True:
# Telemetry
conn, addr = S.accept()
conn.send(data.encode())
conn.close()
Best regards.

For a simple case like this I would probably just wrap the socket code into a separate thread.
With multithreading in python, the Global Interpreter Lock (GIL) means that only one thread executes at a time, so you don't really need to add any further locks to the data if you're just reading the values, and don't care if it's also being updated at the same time.
Your code would essentially read something like:
from threading import Thread
def handle_telemetry_requests():
# Main loop
try:
while True:
# Telemetry
conn, addr = S.accept()
conn.send(data.encode())
conn.close()
except:
# Error handling here (this will cause thread to exit if any error occurs)
pass
socket_thread = Thread(target=handle_telemetry_requests)
socket_thread.daemon = True
socket_thread.start()
Setting the daemon flag means that when the main application ends, the thread will also be terminated.
Python does provide the asyncio module - which may provide the callbacks you're looking for (though I don't have any experience with this).
Other options are to run a flask server in the python apps which will handle the sockets for you and you can just code the endpoints to request the data. Or think about using an MQTT broker - the current data can be written to that - and other apps can subscribe to updates.

How to check if Python script is already running

I have python script on Ubuntu, which sometimes running more than 24 hours. I have set in cron, to run this script every day. However if script is still running, I would like to terminate new instance of this script. I have already found some solution, but they seems to be complicated. I would like to add few lines in the beginning of script, which will be checking if the script is running, if yes return, else continue.
I like this command:
pgrep -a python | grep 'script.py'
it is possible to make some smart solution for this problem?

There is no simple way how to do it. As mentioned in comments you can use creation of some locking file. But i prefer use of sockets. Not sure if it works same on Linux but on Windows i use this:
import socket
class AppMutex:
"""
Class serves as single instance mutex handler (My application can be run only once).
It use OS default property where single UDP port can be bind only once at time.
"""
#staticmethod
def enable():
"""
By calling this you bind UDP connection on specified port.
If binding fails then port is already opened from somewhere else.
"""
try:
AppMutex.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP
AppMutex.sock.bind(("127.0.0.1", 40000))
except OSError:
raise Exception("Application can be run only once.")
Simple call at begining of your script:
AppMutex.enable()

Transferring a mesh from one process to another in PYTHON

I've been cracking my head over this but nothing comes to my mind yet.
I want my script to execute a .py file inside of another already started process. I have a maya process opened, and inside for example modo I want to start file hello.py (print 'hello!') inside that exact Maya.
I already got the PID of that maya process, but don't know how to actually send a command to execute.
is theres some attribute/flag in subprocess or signal modules I could be missing? or is it done even another way?
import os
openedMaya = []
r = os.popen('tasklist /v').read().strip().split('\n')
for i in range(len(r)):
s = r[i]
if 'maya.exe' in s and ': untitled' in s:
openedMaya.append(s)
mayaPID = openedMaya.split('maya.exe')[1].split('Console')[0]
I need a command that could execute hello.py in that maya process.

You could use RPyC to act as a bridge so that you can communicate from one software to another. The idea is that you use RPyC to run an idle server in Maya, where the PYTHONPATH is also pointing to your hello.py script. This server stays active in the session, but the user shouldn't notice it exists.
Then in your other software you use RPyC to broadcast a message using the same port as the server so that it triggers it in Maya. This would then run your command.
It's slightly more overhead, but I have been able to use this successfully for stand-alone tools to trigger events in Maya. As far as using subprocess, you can use it to run a command in a new Maya session, but I don't think there's a way to use it for an existing one.
Hope that nudges you in the right direction.

Maybe an easier way would be to transfer your mesh by using an intermediate file. One process creates the file, another process (running inside the host app) reads it in.

Thanks for the advices, at the end I found a solution by opening the port of maya, by starting a mel command (at the startup):
commandPort -n ":<some_port>";
and connecting from modo to that port through socket:
HOST = '127.0.0.1'
PORT = <some_port>
ADDR=(HOST,PORT)
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(ADDR)
client.send(<message_that_you_want_to_send)
data = client.recv(1024)
client.close()
and i'm able to do whatever I want inside that opened maya, as long as I send mel commands.
Thanks for the help though!

Running infinite loop and getting commands from "outside" (e.g. shell or other scripts)

I am working on my Raspberry Pi, that is handling some WS2812B RGB-LEDs. I can control the light and everything with the neopixel library and Python. So fine right now.
I want this Python script running an infinite loop that only deals with light management. Dimming LEDs, changing color and lots more. But, I want to be able to get commands from other scripts. Let's say I want to type in a shell command that will change the color. In my infinite Python script (LED Handler), I will be able to recognize this command and change the color or the light mode softly to the desired color.
One idea is, to constantly look into a text file, if there is a new command. And my shell script is able to insert command lines into this text file.
But can you tell me, if there is a better solution of doing it?
Many thanks in advance.

One method would be to expose a TCP server, then communicate with the Python process over TCP. A simple example on how to create a TCP server is here, showcasing both the server script (running the LEDs) and the command scripts: example

I suggest opening a port with your python script and make it receive commands from that port (network programming). Although this would make your project more complicated, it is a very robust implementation.

You can use ZeroMQ and host it locally. It provides bindings for Python. Here is an example script (sender and receiver):
from threading import Thread
import zmq
class Sender(Thread):
def run(self):
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.connect('tcp://127.0.0.1:8000')
while True:
socket.send_string(input('Enter command: '))
class Receiver(Thread):
def run(self):
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.bind('tcp://127.0.0.1:8000')
socket.setsockopt(zmq.SUBSCRIBE, b'')
while True:
data = socket.recv().decode('ascii')
print(data) # Do stuff with data.
The receiver would be the instance that controls the lights on the RPi and the sender is the command line script that let's you input the various commands. An advantage is that ZeroMQ supports bindings for various programming languages and you can also send/receive commands over a network.

Another solution is to allow commands from a network connection. The script with the "infinite loop" will read input from a socket and perform the commands.

Window socket programming with Python

I'm trying to implement Window socket using Python.
Mostly, everything has been so far solved using ctypes.windll.ws2_32 and pywin32 lib.
However, I haven't been able to find out how to translate the following C++ codes into Python and I wonder if anyone is kind enough to help:
LRESULT WINAPI AsyncSocketProc(
__in HWND hwnd,
__in UINT uMsg,
__in WPARAM wParam,
__in LPARAM lParam
)
switch(uMsg) {
case WM_CREATE:
//...
case WM_SOCKET: {# this is basically an int constant
switch(WSAGETSELECTEVENT(lParam)){
case FD_ACCEPT:
//accepting new conn
case FD_READ:
//receiving data
}
}
}
In the above code, I couldn't find Python's equivalent for WSAGETSELECTEVENT.
For the FD_ACCEPT, FD_READ, I could find them inside win32file package (of pywin32 lib)
Lastly, the reason why I'm trying to implement this Window socket programming is that the C++ version of the window socket server (above) is non-blocking for an application of mine but Python's built-in select.select is blocking. So I'm trying to see if I can port the C++ version to Python and see if it works.
EDITED:
I would like to clarify that the socket server works as a 'plug in' to an existing C++ program, which doesn't support threading.
The socket server needs to wait (indefinitely) for clients to connect so it needs to continuously listen.
So using a normal Python socket or select.select would entail a while loop (or otherwise how can it acts as a server continuously listening for events? Please correct me I'm wrong), which would block the main program.
Somehow, using the Window Socket server callback above, the main program is not blocked. And this is the main reason while I'm trying to port it to Python.
The socket server is preferably in Python because many related libs the server needs are written in Python.
Thanks a lot.

Have a look at the socket module instead. It already contains all the code you need to work with sockets without using the win32 API.
[EDIT] You can write multi threaded code that can handle several connections. Just accept the connection and then start a new thread, give it the connection and let it read the data in a while 1: data = conn.recv(1024) ... kind of loop.
That said, Python also has a module for just that: SocketServer
[EDIT2] You say
the socket server works as a 'plug in' to an existing program, which doesn't support threading.
It's a bit hard to help with so little information but think about it this way:
You can run the socket server loop in a new thread. This code is isolated from the rest of your app, so it doesn't matter whether the other code uses/supports threads. This solves your "endless loop" problem.
Now this socket server loop will get connections from clients. My guess is that the clients will call methods from the rest of the app and here, things get hairy.
You need a way to synchronize these calls. In other places (like all UI frameworks), there is a single thread which runs any UI calls (drawing something, creating the UI, responding to user input).
But if I understand you correctly, then you can in fact modify the "main loop" of the existing app and let it do more things (like listening to new connections). If you can do this, then there is a way out:
Create a new thread for the socket server as described above. When the server gets a connection, spawn a new thread that talks to the client. When the client sends commands, create "work objects" (see command pattern) and put them into a queue.
In the main loop, you can look into the queue. If something is in there, pop the work objects and call it's run() method.

You don't need or want to port this code. This code is specific to how the WIN32 API notifies native code that a socket operation has completed. It doesn't apply in Python.
The equivalent in python would be, roughly, to paste the "accepting new conn" code in wherever your python code accepts a new connection. And paste the "receiving data" code wherever your python code receives data.
You can also use select, just keep in mind that the semantics are a bit of the reverse of async sockets. With async sockets, you start an operation whenever you want and you get a callback when it completes. With 'select', it tell you when to perform an operation such that it completes immediately.