I have a python web server (django)
It talks to other services such as elasticsearch
I notice when elasticsearch goes down, the web server soon (after a few minutes) stops responding to client requests.
I use https://github.com/elastic/elasticsearch-py and it implements timeout and don't think it's blocking.
My hunch is that requests piles up during the timeout period and server becomes unavailable but it's a just guess.
What's the reason for the server not being able to handle requests in such a scenario and how do I fix it?
I have nginx - uwsgi - django on unix (amazon ecs) setup if that makes difference
Related
I recently started working with Yaskawa's OPC UA Server provided on its robot's controller.
I'm connecting to the server via Python's OPCUA library. Everything works well, but when my code crashes or when I will close terminal without disconnecting from the server I cannot connect to it once again.
I receive an error from library, saying:
The server has reached its maximum number of sessions.
And the only way to solve this is to restart the controller by turning it off and on again.
Documentation of the server is saying that max number of sessions is 2.
Is there a way to clear the connection to the server without restarting the machine?
The server keeps track of the client session and doesn't know that your client crashed.
But the client can define a short enough SessionTimeout, after which the server can remove the crashed session.
The server may have some custom configuration where you can define the maximum number of sessions that it supports. 2 sessions is very limited, but if the hardware is very limited maybe that is the best you can get. See the product documentation about that.
I have a Python REST service and I want to serve it using HTTP2. My current server setup is nginx -> Gunicorn. In other words, nginx (port 443 and 80 that redirects to port 443) is running as a reverse proxy and forwards requests to Gunicorn (port 8000, no SSL). nginx is running in HTTP2 mode and I can verify that by using chrome and inspecting the 'protocol' column after sending a simple GET to the server. However, Gunicorn reports that the requests it receives are HTTP1.0. Also, I coulnt't find it in this list:
https://github.com/http2/http2-spec/wiki/Implementations
So, my questions are:
Is it possible to serve a Python (Flask) application with HTTP2? If yes, which servers support it?
In my case (one reverse proxy server and one serving the actual API), which server has to support HTTP2?
The reason I want to use HTTP2 is because in some cases I need to perform thousands of requests all together and I was interested to see if the multiplexed requests feature of HTTP2 can speed things up. With HTTP1.0 and Python Requests as the client, each request takes ~80ms which is unacceptable. The other solution would be to just bulk/batch my REST resources and send multiple with a single requests. Yes, this idea sounds just fine, but I am really interested to see if HTTP2 could speed things up.
Finally, I should mention that for the client side I use Python Requests with the Hyper http2 adapter.
Is it possible to serve a Python (Flask) application with HTTP/2?
Yes, by the information you provide, you are doing it just fine.
In my case (one reverse proxy server and one serving the actual API), which server has to support HTTP2?
Now I'm going to tread on thin ice and give opinions.
The way HTTP/2 has been deployed so far is by having an edge server that talks HTTP/2 (like ShimmerCat or NginX). That server terminates TLS and HTTP/2, and from there on uses HTTP/1, HTTP/1.1 or FastCGI to talk to the inner application.
Can, at least theoretically, an edge server talk HTTP/2 to web application? Yes, but HTTP/2 is complex and for inner applications, it doesn't pay off very well.
That's because most web application frameworks are built for handling requests for content, and that's done well enough with HTTP/1 or FastCGI. Although there are exceptions, web applications have little use for the subtleties of HTTP/2: multiplexing, prioritization, all the myriad of security precautions, and so on.
The resulting separation of concerns is in my opinion a good thing.
Your 80 ms response time may have little to do with the HTTP protocol you are using, but if those 80 ms are mostly spent waiting for input/output, then of course running things in parallel is a good thing.
Gunicorn will use a thread or a process to handle each request (unless you have gone the extra-mile to configure the greenlets backend), so consider if letting Gunicorn spawn thousands of tasks is viable in your case.
If the content of your requests allow it, maybe you can create temporary files and serve them with an HTTP/2 edge server.
It is now possible to serve HTTP/2 directly from a Python app, for example using Twisted. You asked specifically about a Flask app though, in which case I'd (with bias) recommend Quart which is the Flask API reimplemented on top of asyncio (with HTTP/2 support).
Your actual issue,
With HTTP1.0 and Python Requests as the client, each request takes ~80ms
suggests to me that the problem you may be experiencing is that each request opens a new connection. This could be alleviated via the use of a connection pool without requiring HTTP/2.
I have an API written with python flask running on Bluemix. Whenever I send it a request and the API takes more than 120 seconds to respond it times out. It does not return anything and it returns the following error: 500 Error: Failed to establish a backside connection.
I need it to be able to process longer requests as well. Is there any way to extend the timeout value or is there a workaround for this issue?
All Bluemix traffic goes through the IBM WebSphere® DataPower® SOA Appliances, which provide reverse proxy, SSL termination, and load balancing functions. For security reasons DataPower closes inactive connections after 2 minutes.
This is not configurable (as it affects all Bluemix users), so the only solution for your scenario is to change your program to make sure the connection is not idle for more than 2 minutes.
In my application I need to "simulate" a HTTP timeout. Simply put, in this scenario:
client -> myapp -> server
client makes a HTTP POST connection to myapp which forwards it to server. However, server does not respond due to network issues or similar problems. I am stuck with an open TCP session from client which I'll need to drop.
My application uses web.py, nginx and uwsgi.
I cannot return a custom HTTP error such as 418 I am a teapot - it has to be a connection timeout to mirror server's behaviour as closely as possible.
One hack-y solution could be (I guess) to just time.wait() until client disconnects but this would use a uwsgi thread and I have a feeling it could lead to resource starvation because a server timeout is likely to happen for other connections. Another approach is pointed out here however this solution implies returning something to client, which is not my case.
So my question is: is there an elegant way to kill a uwsgi worker programmatically from python code?
So far I've found
set_user_harakiri(N) which I could combine with a time.sleep(N+1). However in this scenario uwsgi detects the harakiri and tries re-spawning the worker.
worker_id() but I'm not sure how to handle it - I can't find much documentation on using it
A suggestion to use connection_fd() as explained here
disconnect() which does not seem to do anything, as the code continues and returns to client
suspend() does suspend the instance, but NGINX returns the boilerplate error page
Any other idea?
UPDATE
Turns out it's more complicated than that. If I just close the socket or disconnect from uwsgi the nginx web server detects a 'server error' and returns a 500 boilerplate error page. And, I do not know how to tell nginx to stop being so useful.
The answer is a combination of both.
From the python app, return 444
Configure nginx as explained on this answer i.e. using the uwsgi_intercept_errors directive.
I'm looking to start a web project using Flask and its SocketIO plugin, which depends on gevent (something something greenlets), but I don't understand how gevent relates to the webserver. Does using gevent restrict my server choice at all? How does it relate to the different levels of web servers that we have in python (e.g. Nginx/Apache, Gunicorn)?
Thanks for the insight.
First, lets clarify what we are talking about:
gevent is a library to allow the programming of event loops easily. It is a way to immediately return responses without "blocking" the requester.
socket.io is a javascript library create clients that can maintain permanent connections to servers, which send events. Then, the library can react to these events.
greenlet think of this a thread. A way to launch multiple workers that do some tasks.
A highly simplified overview of the entire process follows:
Imagine you are creating a chat client.
You need a way to notify the user's screens when anyone types a message. For this to happen, you need someway to tell all the users when a new message is there to be displayed. That's what socket.io does. You can think of it like a radio that is tuned to a particular frequency. Whenever someone transmits on this frequency, the code does something. In the case of the chat program, it adds the message to the chat box window.
Of course, if you have a radio tuned to a frequency (your client), then you need a radio station/dj to transmit on this frequency. Here is where your flask code comes in. It will create "rooms" and then transmit messages. The clients listen for these messages.
You can also write the server-side ("radio station") code in socket.io using node, but that is out of scope here.
The problem here is that traditionally - a web server works like this:
A user types an address into a browser, and hits enter (or go).
The browser reads the web address, and then using the DNS system, finds the IP address of the server.
It creates a connection to the server, and then sends a request.
The webserver accepts the request.
It does some work, or launches some process (depending on the type of request).
It prepares (or receives) a response from the process.
It sends the response to the client.
It closes the connection.
Between 3 and 8, the client (the browser) is waiting for a response - it is blocked from doing anything else. So if there is a problem somewhere, like say, some server side script is taking too long to process the request, the browser stays stuck on the white page with the loading icon spinning. It can't do anything until the entire process completes. This is just how the web was designed to work.
This kind of 'blocking' architecture works well for 1-to-1 communication. However, for multiple people to keep updated, this blocking doesn't work.
The event libraries (gevent) help with this because they accept and will not block the client; they immediately send a response and when the process is complete.
Your application, however, still needs to notify the client. However, as the connection is closed - you don't have a way to contact the client back.
In order to notify the client and to make sure the client doesn't need to "refresh", a permanent connection should be open - that's what socket.io does. It opens a permanent connection, and is always listening for messages.
So work request comes in from one end - is accepted.
The work is executed and a response is generated by something else (it could be a the same program or another program).
Then, a notification is sent "hey, I'm done with your request - here is the response".
The person from step 1, listens for this message and then does something.
Underneath is all is WebSocket a new full-duplex protocol that enables all this radio/dj functionality.
Things common between WebSockets and HTTP:
Work on the same port (80)
WebSocket requests start off as HTTP requests for the handshake (an upgrade header), but then shift over to the WebSocket protocol - at which point the connection is handed off to a websocket-compatible server.
All your traditional web server has to do is listen for this handshake request, acknowledge it, and then pass the request on to a websocket-compatible server - just like any other normal proxy request.
For Apache, you can use mod_proxy_wstunnel
For nginx versions 1.3+ have websocket support built-in