We Keep Coding

Telnet to Python server gets stuck

I am preparing a demo, and I wanted to use a simple server for it, instead of nginx or any other popular server.
I got the following one from python official page:
import SimpleHTTPServer
import SocketServer
PORT = 8000
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", PORT), Handler)
print "serving at port", PORT
httpd.serve_forever()
So, I run it, and everything seems to be fine:
python -m SimpleHTTPServer 8000
If I go to the browser, or curl the IP address, it works just fine:
xx.xx.xx.xx - - [31/May/2019 09:36:33] "GET / HTTP/1.1" 200 -
xx.xx.xx.xx - - [31/May/2019 09:37:43] "GET / HTTP/1.1" 200 -
xx.xx.xx.xx - - [31/May/2019 09:37:46] "HEAD / HTTP/1.1" 200 -
xx.xx.xx.xx - - [31/May/2019 09:39:03] "HEAD / HTTP/1.1" 200 -
xx.xx.xx.xx - - [31/May/2019 09:39:06] "GET / HTTP/1.1" 200 -
xx.xx.xx.xx - - [31/May/2019 09:39:12] "GET / HTTP/1.1" 200 -
xx.xx.xx.xx - - [31/May/2019 09:39:13] "GET / HTTP/1.1" 200 -
Now, when I connect to the server through telnet, and I do GET, the server get stuck:
$ telnet xx.xx.xx.xx 8000
Trying xx.xx.xx.xx...
Connected to xx.xx.xx.xx.
Escape character is '^]'.
GET / HTTP/1.1
...It just hangs here foever...
When I hit on ctrl+c on the server, it shows me an exception that happened, that I don't know how to interpret in this context:
^C----------------------------------------
Exception happened during processing of request from ('xx.xx.xx.xx', 47346)
Traceback (most recent call last):
File "/usr/lib/python3.5/socketserver.py", line 313, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib/python3.5/socketserver.py", line 341, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python3.5/socketserver.py", line 354, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python3.5/socketserver.py", line 681, in __init__
self.handle()
File "/usr/lib/python3.5/http/server.py", line 422, in handle
self.handle_one_request()
File "/usr/lib/python3.5/http/server.py", line 400, in handle_one_request
if not self.parse_request():
File "/usr/lib/python3.5/http/server.py", line 334, in parse_request
_class=self.MessageClass)
File "/usr/lib/python3.5/http/client.py", line 206, in parse_headers
line = fp.readline(_MAXLINE + 1)
File "/usr/lib/python3.5/socket.py", line 576, in readinto
return self._sock.recv_into(b)
KeyboardInterrupt
----------------------------------------
I also SSH-ed into the same machine where the server is running and did the request connecting to the localhost. Hanged too.
I would like to know:
How is telnet GET different to other GETs.
At python server level, what does the exception mean, and how to solve it.

How is telnet GET different to other GETs.
telnet is not different here. Your server is hanging because it is waiting for you to finish typing your GET request. For this, you just need to press Enter twice, after typing your GET request.
This is because:
One CR-LF character is required to end the message header section of the HTTP request (as specified here)
another one is the actual CR-LF that tells telnet you finished entering input
At python server level, what does the exception mean, and how to solve it.
This is just a consequence of you pressing Ctrl + C, thus interrupting the Python process running your SimpleHTTPServer.

flask-socket.io: frequent time-outs

I have a flask-socket.io application that is pretty standard:
server: eventlet
I start the app using: socketio.run(app, host='0.0.0.0')
Frequently but not always I have some kind of timeout:
Traceback (most recent call last):
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/wsgi.py", line 507, in handle_one_response
result = self.application(self.environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/flask/app.py", line 1997, in __call__
return self.wsgi_app(environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/flask_socketio/__init__.py", line 42, in __call__
start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/middleware.py", line 47, in __call__
return self.engineio_app.handle_request(environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/socketio/server.py", line 360, in handle_request
return self.eio.handle_request(environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/server.py", line 267, in handle_request
environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/socket.py", line 89, in handle_get_request
start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/socket.py", line 130, in _upgrade_websocket
return ws(environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/async_eventlet.py", line 19, in __call__
return super(WebSocketWSGI, self).__call__(environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/websocket.py", line 127, in __call__
self.handler(ws)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/socket.py", line 155, in _websocket_handler
pkt = ws.wait()
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/websocket.py", line 633, in wait
for i in self.iterator:
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/websocket.py", line 503, in _iter_frames
message = self._recv_frame(message=fragmented_message)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/websocket.py", line 526, in _recv_frame
header = recv(2)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/websocket.py", line 442, in _get_bytes
d = self.socket.recv(numbytes - len(data))
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/greenio/base.py", line 360, in recv
return self._recv_loop(self.fd.recv, b'', bufsize, flags)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/greenio/base.py", line 354, in _recv_loop
self._read_trampoline()
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/greenio/base.py", line 325, in _read_trampoline
timeout_exc=socket_timeout('timed out'))
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/greenio/base.py", line 207, in _trampoline
mark_as_closed=self._mark_as_closed)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/hubs/__init__.py", line 163, in trampoline
return hub.switch()
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/hubs/hub.py", line 295, in switch
return self.greenlet.switch()
socket.timeout: timed out
I am not able to interpret this traceback. Can somebody with some experience in flask-socket.io help?
I am not posting any code because I would not know where to start. All files in the traceback are from the installed modules.
EDIT:
I got some more info on the socket.io requests. After the above Exception the following requests are logged:
127.0.0.1 - - [04/Jan/2018 10:10:51] "GET /socket.io/?EIO=3&transport=websocket&sid=f93955151a3a4576b2e96427cc27121e HTTP/1.1" 500 0 60.061493
127.0.0.1 - - [04/Jan/2018 10:10:51] "GET /socket.io/?EIO=3&transport=polling&t=1515056991349-3&sid=f93955151a3a4576b2e96427cc27121e HTTP/1.1" 400 218 60.001593
127.0.0.1 - - [04/Jan/2018 10:10:52] "GET /socket.io/?EIO=3&transport=polling&t=1515057052758-4 HTTP/1.1" 200 381 0.000875
(12472) accepted ('127.0.0.1', 39520)
127.0.0.1 - - [04/Jan/2018 10:10:52] "POST /socket.io/?EIO=3&transport=polling&t=1515057052767-5&sid=10663b1e21e6492b81b5455ebc805408 HTTP/1.1" 200 219 0.001145

You may use socketio.run(app, host='0.0.0.0', port=5000, debug=True) if you want to switch in debug mode.
Then you may take a look on the socket.io official documentation website, most especialy on client-api and server-api.
Beause in the httpsserver-options part of the server-api (for javascript), you can see that there is some options when running the server, like:
pingInterval: 10000,
pingTimeout: 5000,
I expect that those arguments could be re-used as kwargs when you run you server with socketio.run(app, host='0.0.0.0', port=5000, debug=True,pingInterval = 10000, pingTimeout= 5000)
And in the flask-socketio documentation, in the "Error Handling" part,there is nice tips to to deal with exceptions
You may add something like
#socketio.on_error_default # handles all namespaces without an explicit error handler
def default_error_handler(e):
pass
Another tips could be to adapt this previous error_handler for timeout issue and re-run the server when this event will be triggered.
You may also note that:
The message and data arguments of the current request can also be inspected with the request.event variable, which is useful for error logging and debugging outside the event handler
You may also use this following code to handle the final socket.timeout: timed out error :
try:
socketio.run(app,...
except socket.error as socketerror:
print("Error: ", socketerror)

To complement a-stefani's answer. Proper way to change pingInterval and pingTimeout in flask-socketio is with:
from flask_socketio import SocketIO
socketio = SocketIO(app,ping_timeout=5,ping_interval=10)
This doesn't work: socketio.run(app, host='0.0.0.0', port=5000, debug=True,pingInterval = 10000, pingTimeout= 5000)

Django server is hanging in request handler

I do a poll once a second, and after some idle time (10min - 1h) the server stops responding. If I press Ctrl-C handler is terminated and the server is back alive. Error after pressing Ctrl-C (while trying to connect with a new client):
...same messages repeat...
[22/Sep/2016 21:54:27] "GET /ajax/?id=STATUS_POLL&_=1474566714286 HTTP/1.1" 200 0
[2016-09-22 21:54:28,217: DEBUG/MainProcess] NONE
[22/Sep/2016 21:54:28] "GET /ajax/?id=STATUS_POLL&_=1474566714287 HTTP/1.1" 200 0
[2016-09-23 09:33:14,657: INFO/MainProcess] Starting view
[23/Sep/2016 09:36:34] "GET / HTTP/1.1" 200 18970
Traceback (most recent call last):
File "c:\python27\lib\wsgiref\handlers.py", line 86, in run
self.finish_response()
File "c:\python27\lib\wsgiref\handlers.py", line 128, in finish_response
self.write(data)
File "c:\python27\lib\wsgiref\handlers.py", line 212, in write
self.send_headers()
File "c:\python27\lib\wsgiref\handlers.py", line 270, in send_headers
self.send_preamble()
File "c:\python27\lib\wsgiref\handlers.py", line 194, in send_preamble
'Date: %s\r\n' % format_date_time(time.time())
File "c:\python27\lib\socket.py", line 328, in write
self.flush()
File "c:\python27\lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine
[23/Sep/2016 09:36:34] "GET / HTTP/1.1" 500 59
Request handler:
class CTrackerView(TemplateView):
__process = ProcessWrapper()
#staticmethod
def ajax_handler(request):
if request.GET.get('id', "") == "STATUS_POLL":
if CTrackerView.__process.is_alive(no_log=False):
return HttpResponse("UPDATING")
else:
return HttpResponse("")
process_wrapper.py:
class ProcessWrapper:
def is_alive(self, no_log=False):
if (not hasattr(self, "_process")) or not self._process:
if not no_log:
log("NONE", DEBUG)
return False
As you can see from the last code fragment, it just checks the variable is not defined (process is deleted) and return False during all the idle time. (Message "NONE" in log). The problem is difficult to reproduce.

Django - Leaving socket open while receiving data

When I try to load in a largish (50mb) video, the server throws this error:
[14/Mar/2016 02:16:13] "GET /media/media/uploads/SampleVideo_1280x720_50mb.mp4 HTTP/1.1" 200 52464391
[14/Mar/2016 02:16:13] "GET /media/media/uploads/SampleVideo_1280x720_50mb.mp4 HTTP/1.1" 200 286720
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 86, in run
self.finish_response()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 128, in finish_response
self.write(data)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 217, in write
self._write(data)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 328, in write
self.flush()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 41] Protocol wrong type for socket
[14/Mar/2016 02:16:13] "GET /media/media/uploads/SampleVideo_1280x720_50mb.mp4 HTTP/1.1" 500 59
- Broken pipe from ('127.0.0.1', 52070)
As you can see the video is requested twice before the error is thrown. It seems this is caused when the socket closes before the whole video is loaded in. This is also supported by the fact small videos don't throw errors (on Chrome - everything throws an error on Safari)
I'm using django 1.9 in the development server and html5 to display the videos onto the page.
How would I be able to keep the socket open until all the packets have been received? And why would this even be the default behaviour? I can't think of any useful application of having the socket close before all the required data is sent.

Using AWS S3 to store and handle the media this problem was fixed on all browsers. It must have something to do with the development server.

how can fix this error googleApp engine error 10054

i am using google app engine on my server (listen --address=0.0.0.0 all network) my application a short time correctyl(i get some users input, some process,show the data and insert mysql db) after i take this error message and web browser didn't show anything, my system: google app engine, python 2.7, mysql(via rdbms) i change the port(8080,8091,8090vs...) or restart app engine, my applicayion again work but after then again same error message
Traceback (most recent call last):
File "C:\Python27\lib\SocketServer.py", line 284, in _handle_request_noblock
self.process_request(request, client_address)
File "C:\Python27\lib\SocketServer.py", line 310, in process_request
self.finish_request(request, client_address)
File "C:\Python27\lib\SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver.py", line 2734, in __init__
BaseHTTPServer.BaseHTTPRequestHandler.__init__(self, *args, **kwargs)
File "C:\Python27\lib\SocketServer.py", line 639, in __init__
self.handle()
File "C:\Python27\lib\BaseHTTPServer.py", line 343, in handle
self.handle_one_request()
File "C:\Python27\lib\BaseHTTPServer.py", line 313, in handle_one_request
self.raw_requestline = self.rfile.readline(65537)
File "C:\Python27\lib\socket.py", line 476, in readline
data = self._sock.recv(self._rbufsize)
error: [Errno 10054]

Try switching to a more serious networking framework, not based on a toy HTTP server (BaseHTTPServer is a toy HTTP server).
10054 is the Windows error ECONNRESET. This indicates that the connection your HTTP server is trying to read from has been closed. It's not exactly an error condition - connections get closed, it's a normal part of their lifecycle - but the Google AppEngine (development!) server you're using seems to be treating it as an error. Perhaps in doing so it ends up responding incorrectly to other requests as well.
A correct HTTP server will not have a problem dealing with this situation.

The AppEngine development server is indeed pretty limited (single-threaded, HTTP 1.0 only, not robust to connection resets), but it is the only option for emulating in a local dev. environment the behavior of the production server, including stubs to AppEngine services and sandboxing of some modules (including os and socket).
Until Google provide us with a more robust alternative, I added exception handlers for the expected socket errors in the handle_one_request method of BaseHTTPServer.py (located in the Lib subdir of the Python installation directory)
[end of BaseHTTPServer.py::handle_one_request() - Python 2.7]
except socket.error as socket_error:
import errno
if socket_error.errno in (errno.ECONNABORTED, errno.WSAECONNABORTED, errno.ECONNRESET, errno.WSAECONNRESET):
pass
else:
raise
ECONNABORTED/WSAECONNABORTED is happening on remote disconnects - see Python issue 14574 for details.