I am using redis to save/update/delete data for my web socket server (Implemented using autobahn - twisted based web socket implementation) according to the messages I get from my server clients. For Redis operations I am using redis-py package. When there is more number of concurrent clients connecting to my server, I could see requests served in synchronous manner. I found redis operations blocks server from handling parallel client requests. Why is this happenimg ? How can I solve this issue ? I am doing redis operations from onMessage function of autobahn protocol class.
I found the root cause by googling. Issue was the python package I was using for Redis operation (redis-py) was designed in synchronus manner. So twisted server main thread was in blocking state during data fetch/update from Redis. Now I am trying the twisted based asynchronus package for Redis called txredisapi instead of redis-py in twisted way using defer package.
Related
I am trying to connect to a GRPC server in a celery task. I have the following piece of code
timeout = 1
host = '0.tcp.ngrok.io'
port = '7145'
channel = grpc.insecure_channel('{0}:{1}'.format(host, port))
try:
grpc.channel_ready_future(channel).result(timeout=timeout)
except grpc.FutureTimeoutError:
sys.exit(1)
stub = stub(channel)
When I run this snippet through the Python shell, I am able to establish the connection, and execute the GRPC methods. However, when I run this through the Celery task, I get the grpc.FutureTimeoutError, and the connection does not get established.
The Celery worker lies on the same machine as the grpc server. I tried using the socket library to ping the GRPC server, and that worked (It returned some junk response).
I am using Python 2.7, with grpcio==1.6.0 installed. The Celery version is 4.1.0. Any pointers would be helpful.
I believe Celery uses fork under the hood, and gRPC 1.6 did not support any forking behavior.
Try updating to gRPC 1.7.
My requirement is to communicate socketio with nodejs server to Raspberry Pi running a local Python app. Please help me. I can find ways of communication with web app on google but is there any way to communicate with Python local app with above mentioned requirements.
It's unclear exactly which part you need help with. To make a socket.io connection work, you do the following:
Run a socket.io server on one of your two computers. Make sure it is listening on a known port (it can share a port with a web server if desired).
On the other computer, get a socket.io client library and use that to make a socket.io connection to the other computer.
Register message handlers on both computers for whatever custom messages you intend to send each way and write the code to process those incoming messages.
Write the code to send messages to the other computer at the appropriate time.
Socket.io client and server libraries exist for both node.js and python so you can either type of library for either type of system.
The important things to understand are that you must have a socket.io server up and running. The other endpoint then must connect to that server. Once the connection is up and running, you can then send message from either end to the other end.
For example, you could set up a socket.io server on node.js. Then, use a socket.io client library for python to make a socket.io connection to the node.js server. Then, once the connection is up and running, you are free to send messages from either end to the other and, if you have, message handlers listening for those specific messages, they will be received by the other end.
Quick question: need pure python script of simple http client without using libs (only socket library possible).
Main task of this client is connect to server, receive greetings, sends get requests and read responses. Also it's good if this code will be compatible with Cython compiler.
I would recommend you using requests https://github.com/kennethreitz/requests package.
But your question looks like an assignment which shall teach you how is http working on TCP communication level. In such case I would recommend you
learn using http protocol over telnet or netcat
then learn TCP communication by Python and repeat, what you already know by telnet
I have thoroughly looked at the Chrome packaged app website and sample apps but I couldn't find any example related to websockets implementation in an app. I was wondering if there is any example or sample app that uses Websocket for client/server communication in Chrome app? If not then is there any guide? Is it even possible to use WebSocket? I am using Apache HTTP as my server which is in Python.
I'm assuming you're asking about implementing a WebSocket server, because the browser natively supports the client. (Though if you wanted, you could definitely implement the WebSocket client because you have access to the raw TCP interface.)
The Chrome Apps has published a sample WebSocket chat server that servers HTTP requests to load the chat client and uses WebSockets to send messages between clients.
If you look through the implementation, you see it uses the older chrome.socket API to listen to a TCP socket and respond with the correct WebSocket HTTP headers. It does all the bit manipulation to send and receive frames of data as required by the WebSocket spec.
I'm looking for a python socket server framework - not to handle http, but to handle tcp sockets. I've done it myself, but adding all the features is tedious. This framework would handle thread pooling, socket setup, signal handling, etc.
A big feature is code-reloading. If I use apache/mod_python, or django, or whatever, I don't have to restart the server to get it to use new/changed code. Anybody know how that's done?
thanks!
Colin
Twisted is the usual suspect. Reloading in the case of mod_wsgi is easy since only the WSGI server needs to be restarted, not the whole web server (not that restarting the web server is all that hard, mind you...).
Use Apache, mod_wsgi in daemon mode and follow these guidelines.
Update: I mentioned Apache because you did in your question - I assumed you were talking about a web application that also acted as a socket server.
The Python library has socket servers (see the documentation). AFAIK you can't do hot code reloading in Python without potentially losing packets, for that you would need something specifically designed for hot code reloading, such as Erlang, or else just have a dumb socket receiver which receives and queues packets, and a smarter backend process which does code reloading and packet handling. In that case, your receiver would be acting as a proxy.