I'm looking for a python socket server framework - not to handle http, but to handle tcp sockets. I've done it myself, but adding all the features is tedious. This framework would handle thread pooling, socket setup, signal handling, etc.
A big feature is code-reloading. If I use apache/mod_python, or django, or whatever, I don't have to restart the server to get it to use new/changed code. Anybody know how that's done?
thanks!
Colin
Twisted is the usual suspect. Reloading in the case of mod_wsgi is easy since only the WSGI server needs to be restarted, not the whole web server (not that restarting the web server is all that hard, mind you...).
Use Apache, mod_wsgi in daemon mode and follow these guidelines.
Update: I mentioned Apache because you did in your question - I assumed you were talking about a web application that also acted as a socket server.
The Python library has socket servers (see the documentation). AFAIK you can't do hot code reloading in Python without potentially losing packets, for that you would need something specifically designed for hot code reloading, such as Erlang, or else just have a dumb socket receiver which receives and queues packets, and a smarter backend process which does code reloading and packet handling. In that case, your receiver would be acting as a proxy.
Related
I am making my home smart using esp32 and micropython. I have a Django project running on a server I have on my LAN and I want to send commands to my esp32 wirelessly through it. Maybe something like running a uvicorn server and a fastapi app and then sending messages to the uvicorn server endpoints and I have no idea how to do this.
You should probably use either Django Channels or django-websocket
But I'm not sure if websockets package is ported for micropython, so you might need to use plain socket
Or try something completely different. Physically connect the server and esp32 together (I have absolutely no idea about it)
My recommendation is the first option
I have a node.js server running on a Raspberry Pi 3 B+. (I'm using node because I need the capabilities of a bluetooth library that works well).
Once the node server picks up a message from a bluetooth device, I want it to fire off an event/command/call to a different python script running on the same device.
What is the best way to do this? I've looked into spawning child processes and running the script in them, but that seems messy... Additionally, should I set up a socket between them and stream data through it? I imagine this is done often, what is the consensus solution?
Running a child process is how you would run a python script. That's how you do it from nodejs or any other program (besides a python program).
There are dozens of options for communicating between the python script and the nodejs program. The simplest would be stdin/stdout which are automatically set up for you when you create the child process, but you could also give the nodejs app a local http server that the python script could communicate with or vice versa.
Or, set up a regular socket between the two.
If, as you now indicate in a comment, your python script is already running, then you may want to use a local http server in the nodejs app and the python script can just send an http request to that local http server whenever it has some data it wants to pass to the nodejs app. Or, if you primarily want data to flow the opposite direction, you can put the http server in the python app and have the nodejs server send data to the python app.
If you want good bidirectional capabilities, then you could also set up a socket.io connection between the two and then you can easily send messages either way at any time.
I occasionally have a problem with flask socket-io freezing, and I have no clue how to fix it.
My client connects to my socket-io server and performs some chat sessions. It works nicely. But for some reason, sometimes from the client side, there is some call that blocks the whole server (The server is stuck in the process, and all other calls are frozen). What is strange is that the server can be blocked as long as the client side app is not totally shutdown.This is an ios-app / web page, and I must totally close the app or the safari page. Closing the socket itself, and even deallocating it doesn't resolve the problem. When the app is in the background, the sockets are closed and deallocated but the problem persists.
This is a small server, and it deals with both html pages and the socket-server so I have no idea if it is the socket itself or the html that blocks the process. But each time the server was freezing, the log showed some socket calls.
Here is how I configured my server:
socketio = SocketIO(app, ping_timeout=5)
socketio.run(app, host='0.0.0.0', port=5001, debug=True, ssl_context=context)
So my question is:
What can freeze the server (this seems to happen when I leave the app or web-site open for a long time while doing nothing). If I use the services normally the server never freezes. And how I can prevent it from happening (Even if I don't know what causing this, is there a way to blindly stop my server from being stuck at a call?
Thanks you for the answers
According to your comment above, you are using the Flask development web server, without the help of an asynchronous framework such as eventlet or gevent. Besides this option being highly inefficient, you should know that this web server is not battle tested, it is meant for short lived tests during development. I'm not sure it is able to run for very long, specially under the unusual conditions Flask-SocketIO puts it through, which regular Flask apps do not exercise. I think it is quite possible that you are hitting some obscure bug in Werkzeug that causes it to hang.
My recommendation is that you install eventlet and try again. All you need to do is pip install eventlet, and assuming you are not passing an explicit async_mode argument, then just by installing this package Flask-SocketIO should configure itself to use it.
I would also remove the explicit timeout setting. In almost all cases, the defaults are sufficient to maintain a healthy connection.
I have a small server on which I host the wsgi applications I write. This server does not have a lot of ram, and the applications are not frequently used and rarely more than one at once.
Is there a way to configure the server so that the applications are only started when they are needed (when I try to connect on the socket they're served on), somewhat like inetd does ?
depends on the server software you use.
if you use nginx + uwsgi for example, you can configure the uwsgi workers to only be created on requests and get destroyed after a certain amount of inactivity.
http://projects.unbit.it/uwsgi/wiki/Doc
look for "idle" "cheap" "cheaper"
i'm trying to build a web server using apache as the http server, mod_wsgi + python as the logic handler, the server was supposed to handler long request without returning, meaning i want to keep writing stuff into this request.
the problem is, when the link is broken, the socket is in a CLOSE_WAIT status, apache will NOT notify my python program, which means, i have to write something to get an exception, says the link is broken, but those messages were lost and can't be restored.
i tried to get the socket status before writing through /proc/net/tcp, but it could not prevent a quick connect/break connection.
anybody has any ideas, please help, very much thanks in advance!
You cant. It is a limitation of the API defined by the WSGI specification. So, nothing to do with Apache or mod_wsgi really as you will have the same issue with any WSGI server if you follow the WSGI specification.
If you search through the mod_wsgi mailing list on Google Groups you will find a number of discussions about this sort of problem in the past.