I'm using gevent and flask to create a streaming app. I've embedded an img element inside the html page and set its src=/video_feed
The corresponding flask code is
def gen():
global vc
"""Video streaming generator function."""
if vc is None:
vc = cv2.VideoCapture(0)
_, frame = vc.read()
while True:
rval, frame = vc.read()
r, frame = cv2.imencode('.jpg', frame)
yield (b'--frame\r\n'
b'Content-Type: image/jpeg\r\n\r\n' + frame.tobytes() + b'\r\n')
#app.route('/video_feed')
def video_feed():
"""Video streaming route. Put this in the src attribute of an img tag."""
return Response(gen(),
mimetype='multipart/x-mixed-replace; boundary=frame')
The problem: The streaming works fine but the problem is Flask blocks and refuses to accept new requests until the client disconnects!
This is the code I use to start it with gevent
def start_gevent(app_port):
http_server = WSGIServer(('', app_port), app)
http_server.serve_forever()
Here is my app log.
::1 - - [2018-04-27 18:02:56] "GET / HTTP/1.1" 200 1827 0.008464
::1 - - [2018-04-27 18:02:59] "GET /video.html HTTP/1.1" 200 1850 <--streaming starts here
0.001877 [wxpython.py] OnClose called <-- client disconnect
::1 - - [2018-04-27 18:03:01] "GET /video_feed HTTP/1.1" 200 2951956 2.790426 <-- flask log of the request comes late
::1 - - [2018-04-27 18:03:01] "GET /database.html HTTP/1.1" 200 118 0.003394 <-- client made these requests concurrently, but flask didnt respond, it logs them now
::1 - - [2018-04-27 18:03:01] "GET /database.html HTTP/1.1" 200 118 0.000549
Can anyone advice?
Related
The following Python2 webserver will log every single request including the one from localhost (127.0.0.1).
webserver.py
import SimpleHTTPServer, SocketServer, sys
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
port = 80
httpd = SocketServer.TCPServer(("", port), Handler)
sys.stderr = open('/home/user/log.txt', 'w', 1)
httpd.serve_forever()
As example; curl localhost (from the same machine) will produce the following log.
10.0.0.1 - - [10/Jan/2019 00:00:00] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [10/Jan/2019 00:00:01] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [10/Jan/2019 00:01:01] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [10/Jan/2019 00:02:02] "GET / HTTP/1.1" 200 -
My question: Would it be possible to make an exception for local request? I don't want to log any request from localhost/127.0.0.1.
I'm thinking something like this but not really sure how to implement it in Python2 yet.
webserver_v2_do_not_log_localhost.py
import webserver.py # webserver code above, or simply paste everything in here.
if SourceIPAddress == 127.0.0.1:
print('DO NOT log request from localhost/127.0.0.1')
# Script here
else:
print('Log everything')
# Script here
Any idea on the scripts would be highly appreciated. Thanks
Desired Output when performing tail -F log.txt (external IP only, not localhost)
10.0.0.1 - - [10/Jan/2019 00:00:00] "GET / HTTP/1.1" 200 -
You can use logging.Filterclass.
When you declare your logger, do something like that:
import logging
logging.basicConfig(filename='myapp.log', level=logging.INFO)
class Global:
SourceIPAddress = ''
class IpFilter(logging.Filter):
def filter(self, rec):#the rec is part of the function signature.
return not Global.SourceIPAddress == '127.0.0.1'
def main():
log = logging.getLogger('myLogger')
log.addFilter(IpFilter())
log.info("log")
Global.SourceIPAddress = '127.0.0.1'
log.info("Don't log")
if __name__ == '__main__':
main()
Of course I implemented it in a very simple way and you should save the IP in a better place(:
I would also check this links for more info:
https://docs.python.org/3/howto/logging-cookbook.html
https://www.programcreek.com/python/example/3364/logging.Filter
I have an app that's outputting a JSON through a socket which was working fine in the past, however recently the socket doesn't seem to be establishing and it continuously POST & GET's with the lines:
<user_ip>,<client_ip> - - [17/Jul/2018 12:48:17]"GET /socket.io/?EIO=3&transport=polling......HTTP/1.1" 200 221 0.000000
<user_ip>,<client_ip> - - [17/Jul/2018 12:48:17]"POST /socket.io/?EIO=3&transport=polling......HTTP/1.1" 200 243 0.517600
Additionally, when the WebSocket first tries, I can see from Chromes console the message:
WebSocket connection to ..... failed: Establishing a tunnel via proxy server failed.
I've also had a look through what I think is a similar issue based on socketIO-client but wasn't able to resolve my problem.
Can anyone help with overcoming this connection issue?
Included a full log on running the app below:
Server initialized for eventlet.
* Debugger is active!
* Debugger PIN: 129-744-633
(7616) wsgi starting up on http://0.0.0.0:6328
(7616) accepted (<client_ip>, 50548)
<user_ip>,<client_ip> - - [17/Jul/2018 13:18:23] "GET /<app_url> HTTP/1.1" 200 1664 0.015000
3602c46fa50247eb9d397fda82f3eae8: Sending packet OPEN data {'sid': '3602c46fa50247eb9d397fda82f3eae8', 'upgrades': ['websocket'], 'pingTimeout': 60000, 'pingInterval': 25000}
3602c46fa50247eb9d397fda82f3eae8: Sending packet MESSAGE data 0
<user_ip>,<client_ip> - - [17/Jul/2018 13:18:23] "GET /socket.io/?EIO=3&transport=polling&t=1531829903367-0 HTTP/1.1" 200 381 0.000000
(7616) accepted (<client_ip>, 50560)
3602c46fa50247eb9d397fda82f3eae8: Received packet MESSAGE data 0/testnamespace
3602c46fa50247eb9d397fda82f3eae8: Sending packet MESSAGE data 0/testnamespace
<user_ip>,<client_ip> - - [17/Jul/2018 13:18:23] "POST /socket.io/?EIO=3&transport=polling&t=1531829903377-1&sid=3602c46fa50247eb9d397fda82f3eae8 HTTP/1.1" 200 221 0.000000
<user_ip>,<client_ip> - - [17/Jul/2018 13:18:23] "GET /socket.io/?EIO=3&transport=polling&t=1531829903379-2&sid=3602c46fa50247eb9d397fda82f3eae8 HTTP/1.1" 200 226 0.000000
(7616) accepted (<client_ip>, 50562)
3602c46fa50247eb9d397fda82f3eae8: Received packet MESSAGE data 2/testnamespace,["event_1",{"data":"Web app connection successful."}]
3602c46fa50247eb9d397fda82f3eae8: Sending packet MESSAGE data 2/testnamespace,["client_event",{"json":[{"Date&Time":"........}]
When I run this sample code :
from flask import Flask
app = Flask(__name__)
def main () :
return "Welcome to Flask "
if __name__ == "__main__":
app.run(debug=True, host="0.0.0.0", port=80)
Entering following on the terminal: python3 app.py, the output is:
* Serving Flask app "app" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production
environment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://0.0.0.0:80/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 144-032-769
127.0.0.1 - - [13/Jun/2018 00:11:59] "GET / HTTP/1.1" 404 -
127.0.0.1 - - [13/Jun/2018 00:11:59] "GET /favicon.ico HTTP/1.1" 404
-
127.0.0.1 - - [13/Jun/2018 00:12:15] "GET / HTTP/1.1" 404 -
127.0.0.1 - - [13/Jun/2018 00:12:16] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [13/Jun/2018 00:12:22] "GET / HTTP/1.1" 404 -
127.0.0.1 - - [13/Jun/2018 00:12:22] "GET /favicon.ico HTTP/1.1" 404 -
It does not stop
And when I open 127.0.0.1 in my browser it says that Not Found
you have not given main() a route decorator. your function should look like this:
#app.route('/')
def main():
return "Welcome to Flask "
After setting the .py script to have app.run(host='0.0.0.0') all my put/get/etc request end up in 404.
E:\location>c:Python27\python.exe coordinatorSim.py
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
192.168.0.101 - - [01/Nov/2015 09:19:18] "PUT /patient/start HTTP/1.1" 404 -
I send the request to 192.168.0.103:5000/patient/start from another machine in the wifi network, which is the ip of the machine on which the py script runs on.
If I remove the app.run(host='0.0.0.0'), than the requests work, on the default localhost address, 127.0.0.1:5000 (given that I send the request to 127.0.0.103:5000/patient/start)
What is it that I am missing?
the put request is:
app = Flask(__name__)
app.debug = True
app.run(host='0.0.0.0')
sSocket = None
......
# Creates client for socket communication. The http client supplies the IP address and port no.
#app.route('/patient/start', methods = ['PUT'])
##requires_auth
def patient_start():
global sThreadStarted
global sSocket
global sSocketThread
# If the client is already started, return an error code !?
#print "JSON:", request.json
if len(request.json) > 0:
l_address = request.json["address"]
l_port = request.json["port"]
# Start client
print l_address, l_port
try:
sSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, e:
return "Error creating socket: " + e
try:
sSocket.connect((l_address, int(l_port)))
except socket.gaierror, e:
return "Address error: " + e
except socket.error, e:
return "Connection error: " + e
try:
sThreadStarted = True
sSocketThread.start()
except threading.ThreadError, e:
return "Threading error: " + e
message = {
'status': 200,
'message': 'Socket created'
}
resp = jsonify(message)
resp.status_code = 200
return resp
else:
return bad_request()
I was facing this issue while hosting my Flask React App on Amazon EC2. Took me a couple of hours to resolve this issue.
The solution is to set the SERVER_NAME = None in the settings.py file for the Flask App.
All credits to Ronhanson for his comment on Github.
Just to reiterate :
Open port 5000 in the EC2 security group inbound rules.
In manage.py add the line app.run(host='0.0.0.0')
In settings.py update SERVER_NAME = None
Follow the above steps and your app would be accessible from external networks.
From doc
By default, a route only answers to GET requests, but that can be changed by providing the methods argument to the route() decorator
So you have to set
#app.route('/url', methods=['GET', 'PUT'])
When I had this problem in a small Raspberry Pi script, I found that having the "app.run(host='0.0.0.0')" at the top of the script was a problem.
I now use a construct like this:
from flask import Flask, jsonify, abort, make_response
...
app = Flask(__name__)
#app.route('/')
def index():
return "Yadda yadda yadda!"
#app.route('/properties', methods=['GET'])
def get_properties():
return jsonify({'site': site})
...
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=True)
The "debug=True" works fine in this context, as do GETs and PUTs from remote clients. The documentation discourages the use of debug=True when listening on all addresses, but does not prohibit it ("If you have debug disabled or trust the users on your network...", emphasis mine).
I also had the problem that i couldnt run my app in local network. The error message in my web browser was
Not Found The requested URL was not found on the server. If you
entered the URL manually please check your spelling and try again.
and in the log of my terminal it said
"GET / HTTP/1.1" 404 -
so -> it couldnt find anything, why:
The problem is that you run your app with app.run its not a var so:
First you have to make your imports, build and configurate your app. Then you can start your web server which is running your app at your local machine like this in your
__init__.py
app.run(host= '0.0.0.0', debug=True)
or like this directly at the end of your web app
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=True)
In the HTTP protocol you can send many requests in one socket using keep-alive and then receive the response from server at once, so that will significantly speed up whole process. Is there any way to do this in python requests lib? Or are there any other ways to speed this up that well using requests lib?
Yes, there is. Use requests.Session and it will do keep-alive by default.
I guess I should include a quick example:
import logging
import requests
logging.basicConfig(level=logging.DEBUG)
s = requests.Session()
s.get('http://httpbin.org/cookies/set/sessioncookie/123456789')
s.get('http://httpbin.org/cookies/set/anothercookie/123456789')
r = s.get("http://httpbin.org/cookies")
print(r.text)
You will note that these log message occur
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): httpbin.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies/set/sessioncookie/123456789 HTTP/1.1" 302 223
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies HTTP/1.1" 200 55
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies/set/anothercookie/123456789 HTTP/1.1" 302 223
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies HTTP/1.1" 200 90
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies HTTP/1.1" 200 90
If you wait a little while, and repeat the last get call
INFO:requests.packages.urllib3.connectionpool:Resetting dropped connection: httpbin.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /cookies HTTP/1.1" 200 90
Note that it resets the dropped connection, i.e. reestablishing the connection to the server to make the new request.