I do a poll once a second, and after some idle time (10min - 1h) the server stops responding. If I press Ctrl-C handler is terminated and the server is back alive. Error after pressing Ctrl-C (while trying to connect with a new client):
...same messages repeat...
[22/Sep/2016 21:54:27] "GET /ajax/?id=STATUS_POLL&_=1474566714286 HTTP/1.1" 200 0
[2016-09-22 21:54:28,217: DEBUG/MainProcess] NONE
[22/Sep/2016 21:54:28] "GET /ajax/?id=STATUS_POLL&_=1474566714287 HTTP/1.1" 200 0
[2016-09-23 09:33:14,657: INFO/MainProcess] Starting view
[23/Sep/2016 09:36:34] "GET / HTTP/1.1" 200 18970
Traceback (most recent call last):
File "c:\python27\lib\wsgiref\handlers.py", line 86, in run
self.finish_response()
File "c:\python27\lib\wsgiref\handlers.py", line 128, in finish_response
self.write(data)
File "c:\python27\lib\wsgiref\handlers.py", line 212, in write
self.send_headers()
File "c:\python27\lib\wsgiref\handlers.py", line 270, in send_headers
self.send_preamble()
File "c:\python27\lib\wsgiref\handlers.py", line 194, in send_preamble
'Date: %s\r\n' % format_date_time(time.time())
File "c:\python27\lib\socket.py", line 328, in write
self.flush()
File "c:\python27\lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine
[23/Sep/2016 09:36:34] "GET / HTTP/1.1" 500 59
Request handler:
class CTrackerView(TemplateView):
__process = ProcessWrapper()
#staticmethod
def ajax_handler(request):
if request.GET.get('id', "") == "STATUS_POLL":
if CTrackerView.__process.is_alive(no_log=False):
return HttpResponse("UPDATING")
else:
return HttpResponse("")
process_wrapper.py:
class ProcessWrapper:
def is_alive(self, no_log=False):
if (not hasattr(self, "_process")) or not self._process:
if not no_log:
log("NONE", DEBUG)
return False
As you can see from the last code fragment, it just checks the variable is not defined (process is deleted) and return False during all the idle time. (Message "NONE" in log). The problem is difficult to reproduce.
Related
Need help solving an issue with Superset (version 1.0.1) running on docker.
Sendings reports (dashboards) by email is configured. Small dashboards are sent fine.
But sending several report ends with this error:
Report Schedule execution failed when generating a screenshot.
[05/Oct/2021:16:00:06 +0300] "GET /static/assets/5d82d1b53c008164c101.chunk.js HTTP/1.1" 200 308964 "http://192.168.90.132:8088/superset/dashboard/49/" "Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0"
Message: Failed to convert data to an object
[2021-10-05 16:00:08,431: ERROR/ForkPoolWorker-7] Message: Failed to convert data to an object
[2021-10-05 16:00:13,752: WARNING/ForkPoolWorker-7] Message: Failed to decode response from marionette
, retrying in 0 seconds...
[2021-10-05 16:00:13,754: WARNING/ForkPoolWorker-7] Message: Tried to run command without establishing a connection
, retrying in 0 seconds...
[2021-10-05 16:00:13,755: WARNING/ForkPoolWorker-7] Message: Tried to run command without establishing a connection
, retrying in 0 seconds...
[2021-10-05 16:00:13,756: WARNING/ForkPoolWorker-7] Message: Tried to run command without establishing a connection
, retrying in 0 seconds...
Failed at generating thumbnail 'WebDriver' object has no attribute 'screenshot'
[2021-10-05 16:00:13,758: ERROR/ForkPoolWorker-7] Failed at generating thumbnail 'WebDriver' object has no attribute 'screenshot'
Report Schedule execution failed when generating a screenshot.
Traceback (most recent call last):
File "/app/superset/utils/celery.py", line 50, in session_scope
yield session
File "/app/superset/reports/commands/execute.py", line 374, in run
raise ex
File "/app/superset/reports/commands/execute.py", line 371, in run
session, self._model, self._scheduled_dttm
File "/app/superset/reports/commands/execute.py", line 344, in run
self._session, self._report_schedule, self._scheduled_dttm
File "/app/superset/reports/commands/execute.py", line 261, in next
raise ex
File "/app/superset/reports/commands/execute.py", line 257, in next
self.send()
File "/app/superset/reports/commands/execute.py", line 195, in send
notification_content = self._get_notification_content()
File "/app/superset/reports/commands/execute.py", line 175, in _get_notification_content
screenshot_data = self._get_screenshot()
File "/app/superset/reports/commands/execute.py", line 167, in _get_screenshot
raise ReportScheduleScreenshotFailedError()
superset.reports.commands.exceptions.ReportScheduleScreenshotFailedError: Report Schedule execution failed when generating a screenshot.
[2021-10-05 16:00:13,775: ERROR/ForkPoolWorker-7] Report Schedule execution failed when generating a screenshot.
Traceback (most recent call last):
File "/app/superset/utils/celery.py", line 50, in session_scope
yield session
File "/app/superset/reports/commands/execute.py", line 374, in run
raise ex
File "/app/superset/reports/commands/execute.py", line 371, in run
session, self._model, self._scheduled_dttm
File "/app/superset/reports/commands/execute.py", line 344, in run
self._session, self._report_schedule, self._scheduled_dttm
File "/app/superset/reports/commands/execute.py", line 261, in next
raise ex
File "/app/superset/reports/commands/execute.py", line 257, in next
self.send()
File "/app/superset/reports/commands/execute.py", line 195, in send
notification_content = self._get_notification_content()
File "/app/superset/reports/commands/execute.py", line 175, in _get_notification_content
screenshot_data = self._get_screenshot()
File "/app/superset/reports/commands/execute.py", line 167, in _get_screenshot
raise ReportScheduleScreenshotFailedError()
superset.reports.commands.exceptions.ReportScheduleScreenshotFailedError: Report Schedule execution failed when generating a screenshot.
Report state: Report Schedule execution failed when generating a screenshot.
[2021-10-05 16:00:13,776: INFO/ForkPoolWorker-7] Report state: Report Schedule execution failed when generating a screenshot.
Whenever this view is executed my server showed up with some error which I don't know why! the view which I was executing is
#api_view(['POST'])
def Signin(request):
if request.method == 'POST':
username = request.POST.get('username')
passw = request.POST.get('password')
email = request.POST.get('email')
fullname = request.POST.get('fullname')
college = request.POST.get('college_name')
city = request.POST.get('city')
country = request.POST.get('country')
User.objects.create_user(
username=username, password=passw, email=email)
Userinfo.objects.create(
user_name=username, full_name=fullname, user_email=email, college_name=college, city=city, country=country , varified_user=False)
return Response({'status': 'created'})
And what i know after checking my database is my code is executing till user_name=username, full_name=fullname, user_email=email, college_name=college, city=city, country=country , varified_user=False) because I can see the data which I was inserting, now i don't know what error can cause by return Response({'status': 'created'}). The message in my terminal is
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
September 29, 2020 - 21:19:18
Django version 3.1, using settings 'backend.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
[29/Sep/2020 21:19:31] "POST /api/usercheck/ HTTP/1.1" 200 18
[29/Sep/2020 21:19:34] "POST /api/usercheck/ HTTP/1.1" 200 18
[29/Sep/2020 21:19:35] "POST /api/usercheck/ HTTP/1.1" 200 21
[29/Sep/2020 21:19:58] "POST /signin/ HTTP/1.1" 200 3318
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 65332)
Traceback (most recent call last):
File "C:\Python38\lib\socketserver.py", line 650, in process_request_thread
self.finish_request(request, client_address)
File "C:\Python38\lib\socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Python38\lib\socketserver.py", line 720, in __init__
self.handle()
File "E:\Project_Bookbhandar\env\lib\site-packages\django\core\servers\basehttp.py", line 174, in handle
self.handle_one_request()
File "E:\Project_Bookbhandar\env\lib\site-packages\django\core\servers\basehttp.py", line 182, in handle_one_request
self.raw_requestline = self.rfile.readline(65537)
File "C:\Python38\lib\socket.py", line 669, in readinto
return self._sock.recv_into(b)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
----------------------------------------
Can anyone help with this??
You will want to be using HttpResponse or JsonResponse.
I have a flask-socket.io application that is pretty standard:
server: eventlet
I start the app using: socketio.run(app, host='0.0.0.0')
Frequently but not always I have some kind of timeout:
Traceback (most recent call last):
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/wsgi.py", line 507, in handle_one_response
result = self.application(self.environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/flask/app.py", line 1997, in __call__
return self.wsgi_app(environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/flask_socketio/__init__.py", line 42, in __call__
start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/middleware.py", line 47, in __call__
return self.engineio_app.handle_request(environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/socketio/server.py", line 360, in handle_request
return self.eio.handle_request(environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/server.py", line 267, in handle_request
environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/socket.py", line 89, in handle_get_request
start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/socket.py", line 130, in _upgrade_websocket
return ws(environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/async_eventlet.py", line 19, in __call__
return super(WebSocketWSGI, self).__call__(environ, start_response)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/websocket.py", line 127, in __call__
self.handler(ws)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/engineio/socket.py", line 155, in _websocket_handler
pkt = ws.wait()
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/websocket.py", line 633, in wait
for i in self.iterator:
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/websocket.py", line 503, in _iter_frames
message = self._recv_frame(message=fragmented_message)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/websocket.py", line 526, in _recv_frame
header = recv(2)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/websocket.py", line 442, in _get_bytes
d = self.socket.recv(numbytes - len(data))
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/greenio/base.py", line 360, in recv
return self._recv_loop(self.fd.recv, b'', bufsize, flags)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/greenio/base.py", line 354, in _recv_loop
self._read_trampoline()
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/greenio/base.py", line 325, in _read_trampoline
timeout_exc=socket_timeout('timed out'))
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/greenio/base.py", line 207, in _trampoline
mark_as_closed=self._mark_as_closed)
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/hubs/__init__.py", line 163, in trampoline
return hub.switch()
File "/projects/ici_chat_prototype01/env/lib/python3.5/site-packages/eventlet/hubs/hub.py", line 295, in switch
return self.greenlet.switch()
socket.timeout: timed out
I am not able to interpret this traceback. Can somebody with some experience in flask-socket.io help?
I am not posting any code because I would not know where to start. All files in the traceback are from the installed modules.
EDIT:
I got some more info on the socket.io requests. After the above Exception the following requests are logged:
127.0.0.1 - - [04/Jan/2018 10:10:51] "GET /socket.io/?EIO=3&transport=websocket&sid=f93955151a3a4576b2e96427cc27121e HTTP/1.1" 500 0 60.061493
127.0.0.1 - - [04/Jan/2018 10:10:51] "GET /socket.io/?EIO=3&transport=polling&t=1515056991349-3&sid=f93955151a3a4576b2e96427cc27121e HTTP/1.1" 400 218 60.001593
127.0.0.1 - - [04/Jan/2018 10:10:52] "GET /socket.io/?EIO=3&transport=polling&t=1515057052758-4 HTTP/1.1" 200 381 0.000875
(12472) accepted ('127.0.0.1', 39520)
127.0.0.1 - - [04/Jan/2018 10:10:52] "POST /socket.io/?EIO=3&transport=polling&t=1515057052767-5&sid=10663b1e21e6492b81b5455ebc805408 HTTP/1.1" 200 219 0.001145
You may use socketio.run(app, host='0.0.0.0', port=5000, debug=True) if you want to switch in debug mode.
Then you may take a look on the socket.io official documentation website, most especialy on client-api and server-api.
Beause in the httpsserver-options part of the server-api (for javascript), you can see that there is some options when running the server, like:
pingInterval: 10000,
pingTimeout: 5000,
I expect that those arguments could be re-used as kwargs when you run you server with socketio.run(app, host='0.0.0.0', port=5000, debug=True,pingInterval = 10000, pingTimeout= 5000)
And in the flask-socketio documentation, in the "Error Handling" part,there is nice tips to to deal with exceptions
You may add something like
#socketio.on_error_default # handles all namespaces without an explicit error handler
def default_error_handler(e):
pass
Another tips could be to adapt this previous error_handler for timeout issue and re-run the server when this event will be triggered.
You may also note that:
The message and data arguments of the current request can also be inspected with the request.event variable, which is useful for error logging and debugging outside the event handler
You may also use this following code to handle the final socket.timeout: timed out error :
try:
socketio.run(app,...
except socket.error as socketerror:
print("Error: ", socketerror)
To complement a-stefani's answer. Proper way to change pingInterval and pingTimeout in flask-socketio is with:
from flask_socketio import SocketIO
socketio = SocketIO(app,ping_timeout=5,ping_interval=10)
This doesn't work: socketio.run(app, host='0.0.0.0', port=5000, debug=True,pingInterval = 10000, pingTimeout= 5000)
I use a flask server to serve static files from a server but once in a while the server becomes completely unresponsive, downloading a file keeps loading but never downloads. When I opened up the terminal, I found some weird requests, I hit CTRL + C and the server immediately becomes responsive again and downloads continue. This happens every so often and I have no idea what's causing this and how to prevent it from freezing my flask server, is this someone trying to hack?
user#server:~/worker# python server.py
* Running on http://0.0.0.0:80/
93.134.13.318 - - [03/Sep/2014 02:07:18] code 400, message Bad request syntax ('\x00')
93.134.13.318 - - [03/Sep/2014 02:07:18] "" 400 -
93.134.13.318 - - [03/Sep/2014 02:07:19] "GET http://httpheader.net HTTP/1.1" 404 -
93.134.13.318 - - [03/Sep/2014 02:07:40] code 400, message Bad request syntax ('\x04\x01\x00P\xc6\xce\x0eu0\x00')
93.174.93.218 - - [03/Sep/2014 02:07:40] "P��u0" 400 -
^C----------------------------------------
Exception happened during processing of request from ('93.174.93.218 ', 45082)
Traceback (most recent call last):
File "/usr/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 310, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 638, in __init__
self.handle()
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 200, in handle
rv = BaseHTTPRequestHandler.handle(self)
File "/usr/lib/python2.7/BaseHTTPServer.py", line 340, in handle
self.handle_one_request()
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 231, in handle_one_request
self.raw_requestline = self.rfile.readline()
File "/usr/lib/python2.7/socket.py", line 447, in readline
data = self._sock.recv(self._rbufsize)
KeyboardInterrupt
----------------------------------------
42.36.63.90 - - [03/Sep/2014 03:21:20] "GET / HTTP/1.1" 404 -
63.63.193.195 - - [03/Sep/2014 03:21:20] "GET / HTTP/1.1" 404 -
I had the same issue. It's common in django/flask apps to hang when running as python server.py, because this in not an optimal environment for them. Only use it for testing purpose. When you're done with it and want to release, you should put it behind a wsgi/uwsgi + apache/nginx and your problem will go away.
http://flask.pocoo.org/docs/0.10/deploying/uwsgi/
http://flask.pocoo.org/docs/1.0/deploying/uwsgi/
I'm using Django==1.1 (i have to)
and file based session. I get following traceback (below)
I know that during initialization of django.contrib.sessions.backends.file.SessionStore the constructor is getting very long session_key.
But where this class instance is created - I don't know. I'm trying to find out. And why the heck so long session_key is created?
Traceback (most recent call last):
File "/Users/john/.virtualenvs/app/lib/python2.7/site-packages/django/core/servers/basehttp.py", line 279, in run
self.result = application(self.environ, self.start_response)
File "/Users/john/.virtualenvs/app/lib/python2.7/site-packages/django/core/servers/basehttp.py", line 651, in __call__
return self.application(environ, start_response)
File "/Users/john/.virtualenvs/app/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 245, in __call__
response = middleware_method(request, response)
File "/Users/john/.virtualenvs/app/lib/python2.7/site-packages/django/contrib/sessions/middleware.py", line 36, in process_response
request.session.save()
File "/Users/john/.virtualenvs/app/lib/python2.7/site-packages/django/contrib/sessions/backends/file.py", line 88, in save
fd = os.open(session_file_name, flags)
OSError: [Errno 63] File name too long: '/tmp/sessionid.eJydVktv4zYQ9m63Fx7aQ3-EfXJFvYWeit1FkYuLrpNuDwsQlMTQSmRJJqU4KVCgP70zQ8mPxMmhB8kWh5z55psX_33_z-7d4uaDaWv11_ez2UyW26rZvb_5wQpTVo224raUvdx9d_3Tu9lsUXacrefNsM2VEfmcdT5b2fXc9tL0AjaqvtoqWA5YMX2xw58uZIv1_Nujl3x7LAv4jeDh8HiXnjkruqf-byY61kV48PNg2k79_FUaK_dgI2ZXie957CqI4b2ef_y8voblhPVfuhRfGbMHrBIkHPb6foCrqilP4XLO9AVwHJ_kl5fINOnnPrPLmx-PTFV1r8zuw_VvbDYrirZ7EkZpBq-ibWxvhqJvDRhji6K8k41ul2W-3Lalqu3S7urlblDmif1BbyC2ECIfQGXVCMHa_E4VPRK7AsuAtQRS1nOpNZgAR4RVNWwQW2nviRsMi3ghLmSxUUQSytVjb5CYlOkTUENfASCMusM8GGXZujW9Kj9ViCE7h1bSIpC7KJEUjtiAmvX8Xj39bkplkOCALeqOh8zmYPeuhYNb2aEgou0xW6z-RJ4clbAOCFd9x1O20PAf_kE04ahRNbhTTv7cVqouLeahhwZ8zia3RC_zWpHIZ4sek5SO4AKB8UPcuxtaYGeixSc0fszI0oM4AeonJEKqEnhSQjNYgCLrSlpn6RkzVmHEABmYCzidlEBR4KP-FqkR-RNWCwEKCFBZWThMOIOIXUFm4-LQKSEf2qqUTUH1FSOaICGhupVD3QtSCOyhOIWTxIU27dCNVjJXqyNzrRFDhzUAotCbLI3SkWYU8QMIdauMAYfrVpbOTgjUaucgZmVAVvsuDE9UHUIURuhlSOQSZyOzITEbOmYXjl6M_mqFhkGbi_xWPopSdf0GjkSAN2Ku8TSlNOWp8xGfnHeJcEJ0NKbCRPIRXEQhiAj4IXMiQhzFLnL2pNxImlDZ6Ax-xuSPUipMXCF9secS3gF5VqL8cjhE0-5lhfGP_Yl6caZgytY4IA11uwedhnSGcOLIr1G3RTs0pCs6B9vFzqsraIcAPKYYxOkBeDb6nL-suaKt0f2EKi7hpyk4ifwpAlNYTookIQMJMb2RDy5mSXTWgs774n6jjGJf8b2CVdgN2F0jTAh3kkK3gR7bKOywqC6DhV9Xn7C3ER8NRQ3zOT3kc7Gp6tKoBhd9xJTSaLBD3hulRCdB1iPiNCRp5GIJeVMPJdBRNYCoQmIcVtxJZZkmhywaUzylvEgdqaAr80Z6YR95J4oaPmBnBm0ioRIhpbgCk2BfGdXLrhtZYV-mTgliAk1KcHPownxkKKOWlsE3kKKRDT1RoJ3bWfLiDMHNstfPcM97foh7rvN7_lvHAqzwtwP9kYalrBqaKuFRvxsVXszG_KaR7rqF1UCTdlxQTYtGusnuYW64-8k07D26GECu0uRHR3Alh2aJzfDSxYXjBQFf__P-wmEeXrjBcCi2i3cYDpOSZmnkfmIcgJQw2iUjxwGUy-chgCFJw9d7IwQ-dyHQp9zijMSDAZLxnFLtSDy9SeHw1I47_JuPFNt9hf0Zi59uWoAaRuoRNU7Wl6jdaOU4W19HnV1AHVAn4zRaL6N2odVnUa4xyvr1kL5-65viAQMcPIOpffQMh3d-8hmdf8aud2wqvZl6NQ_cJczdGkVlBcyk4v6JRKnzPl8Oy_8ACepvHw:1SWRxQ:eJ6zYlFnV8NFqaM2mjeYJUjvBlM'
[22/May/2012 15:45:43] "GET /login/ HTTP/1.1" 500 2736
The request.session gets constructed in SessionMiddleware.process_request, you could check in the file django/contrib/sessions/middleware.py. In early versions,(just checked in recent Django its still vulnerable), there are few checking upon the incoming session key in request.COOKIES, before touching the backend, thus you got a over-long one that fails session.save().