I am trying to make a search engine, running on the web. So I used mod_wsgi to get the query from the webpage by python. Then, I used zeromq to send the query to C++ searching program.
But the problem is, it seems zeromq and mod_wsgi does not work together.
It is definitely true that python can import zmq (I tested it) but when it runs on the web, it shows error message that (actually an error log from apache)
File "D:/wsgi_app/wsgi_app.py", line 2, in <module>, referer: http://localhost/
import zmq, referer: http://localhost/
File "D:\\util\\Python27\\lib\\site-packages\\zmq\\__init__.py", line 35, in <module>, referer: http://localhost/
from zmq.utils import initthreads # initialize threads, referer: http://localhost/
ImportError: DLL load failed: \xc1\xf6\xc1\xa4\xb5\xc8 \xb8\xf0\xb5\xe2\xc0\xbb \xc3\xa3\xc0\xbb \xbc\xf6 \xbe\xf8\xbd\xc0\xb4\xcf\xb4\xd9., referer: http://localhost/
I have no idea why wsgi cannot import zmq. By the way the source below is complete python code
from cgi import parse_qs, escape
import zmq
def application( # It accepts two arguments:
# environ points to a dictionary containing CGI like environment variables
# which is filled by the server for each received request from the client
environ,
# start_response is a callback function supplied by the server
# which will be used to send the HTTP status and headers to the server
start_response):
# get a query from the webpage :)
data = parse_qs(environ['QUERY_STRING'])
query = data.get('query', [''])[0]
query = escape(query) #prevent script injection
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect ("tcp://localhost:5555") #connect to C++ search server database
socket.send (query)
# build the response body possibly using the environ dictionary
response_body = 'The request method was %s' % environ['REQUEST_METHOD']
# HTTP response code and message
status = '200 OK'
# These are HTTP headers expected by the client.
# They must be wrapped as a list of tupled pairs:
# [(Header name, Header value)].
response_headers = [('Content-Type', 'text/plain'),
('Content-Length', str(len(response_body)))]
# Send them to the server using the supplied function
start_response(status, response_headers)
# Return the response body.
# Notice it is wrapped in a list although it could be any iterable.
return [response_body]
In your wsgi script, explicitly add the location of your dependencies:
import site
site.addsitedir(path_to_zeromq)
Related
From the uwsgi documentation:
def application(env, start_response):
start_response('200 OK', [('Content-Type','text/html')])
return [b"Hello World"]
Is it possible to respond to http request(close http connection) and continue execution flow(without any usage of threads/queues/external services etc)?
like this:
def application(env, start_response):
start_response('200 OK', [('Content-Type','text/html')])
end_response(b"Hello World")
#HTTP connection is closed
#continue execution..
TL;DR - if you're using Django, you can skip to the end of the answer.
Yes, there is a way to do this. You can hook on to a .close() method of the object returned by your application callable (or alternatively, wsgi.file_wrapper returned through the env.
Check PEP333, specifically the server side spec part: https://peps.python.org/pep-0333/#the-server-gateway-side:
result = application(environ, start_response)
try:
for data in result:
if data: # don't send headers until body appears
write(data)
if not headers_sent:
write('') # send headers now if body was empty
finally:
if hasattr(result, 'close'):
result.close()
As you can see, result.close() will be called at the very end, by which point the data is already sent.
This is, of course, just some reference code, and it does not deal with upstream and terminating the connection before continuing execution. But well-behaved wsgi servers, such as uwsgi and (presumably) gunicorn, do.
They will signal to the upstream that the request has finished sending by closing the socket (or whatever the upstream protocol requires), and then call .close().
If you are using Django, you are already set, because it has request_finished signal. Here's how it works, it hooks to .close() as described above:
https://github.com/django/django/blob/57c7220280db19dc9dda0910b90cf1ceac50c66f/django/http/response.py#L323
Unfortunately, there is no way to continue the code execution after you have returned the response. It would be much easier if you use multithreading but if not you can workaround it in Flask by adding an AJAX call to your HTML response which will send a POST request to one of the server extra route whose handler function will be the execution code you want after returning the response. Here's one of the possible approach using Flask:
myflaskapp.py
from flask import Flask, render_template_string
import time
app = Flask(__name__)
#app.route('/run', methods=['POST'])
def run():
# this is where you put your "continue execution..." code
# below code is used to test if it runs after HTTP connection close
time.sleep(8)
print('Do something')
return ''
#app.route('/')
def index():
return render_template_string('''
Hello World!
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
<script>
$(function() {
$.ajax({
type: "POST",
url: "{{ url_for('run') }}"
});
})
</script>
''')
if __name__ == "__main__":
app.run(host='0.0.0.0')
You can run the server on port 9091 with the command:
uwsgi --http 127.0.0.1:9091 --wsgi-file myflaskapp.py --callable app
To test if it is working or not, you can go to the address localhost:9091. If everything works well, you should see that the page is loaded immediately while the terminal will only print out Do something after 8 seconds have passed, indicating the function run executes after the HTTP connection is closed.
I'm following Falcon tutorial for Python.
Everything worked fine until this part:
The response I'm getting when trying this command http localhost:8000/images is:
HTTP/1.1 500 Internal Server Error
Content-Length: 110
Content-Type: text/plain
Date: Sat, 01 Dec 2018 15:50:26 GMT
Server: waitress
Internal Server Error
The server encountered an unexpected internal server error
(generated by waitress)
I read it's a problem in the code but I can't find it, it's exactly as in the tutorial, app.py file:
import falcon
from images import Resource
api = application = falcon.API()
images = Resource()
api.add_route('/images', images)`
images.py:
import json
import falcon
class Resource(object):
def on_get(self, req, resp):
doc = {
'images': [
{
'href': '/images/1eaf6ef1-7f2d-4ecc-a8d5-6e8adba7cc0e.png'
}
]
}
# Create a JSON representation of the resource
resp.body = json.dumps(doc, ensure_ascii=False)
# The following line can be omitted because 200 is the default
# status returned by the framework, but it is included here to
# illustrate how this may be overridden as needed.
resp.status = falcon.HTTP_200
Also, I have an empty file named __init__.py and all the files are in the same folder, C:\look\look.
P.S.
I tried to add an HTTP requests scratch file (using PyCharm IDE) but there is no option to add that kind of a file (after I press Ctrl + Shift + Alt + Insert). I couldn't find how to fix this anywhere.
I see that the question is pretty old, but I found a solution. Just run the server with command:
waitress-serve --port=8000 --call look.app:get_app
since we get the app from calling the function get_app()
I've made a simple flask application:
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.1
host:google.be
HTTP/1.0 404 NOT FOUND
Content-Type: text/html
Content-Length: 233
Server: Werkzeug/0.9.6 Python/2.7.6
Date: Mon, 08 Dec 2014 19:15:43 GMT
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>
Connection closed by foreign host.
One of the things I would like the change is the server header which at the moment is set as Werkzeug/0.9.6 Python/2.7.6 to something my own chosing. But I can't seem to find anything in the documentation on how to do this.
You can use Flask's make_response method to add or modify headers.
from flask import make_response
#app.route('/index')
def index():
resp = make_response("Hello, World!")
resp.headers['server'] = 'ASD'
return resp
#bcarroll's answer works but it will bypass other processes defined in original process_response method such as set session cookie.
To avoid the above:
class localFlask(Flask):
def process_response(self, response):
#Every response will be processed here first
response.headers['server'] = SERVER_NAME
super(localFlask, self).process_response(response)
return(response)
You can change the Server header for every response by overriding the Flask.process_response() method.
from flask import Flask
from flask import Response
SERVER_NAME = 'Custom Flask Web Server v0.1.0'
class localFlask(Flask):
def process_response(self, response):
#Every response will be processed here first
response.headers['server'] = SERVER_NAME
return(response)
app = localFlask(__name__)
#app.route('/')
def index():
return('<h2>INDEX</h2>')
#app.route('/test')
def test():
return('<h2>This is a test</h2>')
http://flask.pocoo.org/docs/0.12/api/#flask.Flask.process_response
Overriding Server header in code does not work if You use production server like gunicorn. The better way is to use proxy server behind gunicorn and there change Server header.
TL;DR - overwrite /python3.8/http/server.py send_response method. Comment the server header addition line.
Why?
Adding/Manipulating headers in flask (in any way that mentioned above) will fire the response with the configured headers from flask to the web server but the WSGI logic (which happens independently, after & before flask logic) will be the last one to modify those values if any.
In your case(Werkzeug) some headers are hard-coded in python http module which werkzeug depending on. The server header is one of them.
Easy way:
#app.after_request
def changeserver(response):
response.headers['server'] = SERVER_NAME
return response
I'm trying to log an error in a decorator function using app.logger.error(''), but it just doesn't work. In addition I cant debug this well and I can only see the response from the http client:
(I'm using nginx+uwsgi+flask)
HTTP/1.1 502 Bad Gateway
Server: nginx
Date: Sun, 12 Aug 2012 15:45:09 GMT
Content-Type: text/html
Content-Length: 14
Connection: keep-alive
Everything works great with out the line: app.logger.error('panic !!!')
def mydecorator():
def decorator(f):
def wrapped_function(*args, **kwargs):
try:
ip = Mytable.query.filter_by(ip=request.remote_addr).first()
except:
app.logger.error('panic !!!')
else:
dootherthing()
resp = make_response(f(*args, **kwargs))
h = resp.headers
h['add-this-header'] = ":)"
return resp
return update_wrapper(wrapped_function, f)
return decorator
It seems that it is out of context or something.
in fact, the decorator wasnt able to detect the app instance out of context, i solve this using current_app:
1st. Import the method: from flask import current_app
2nd. append any app class to current_app: current_app.logger.error('panic !!!')
info # http://flask.pocoo.org/docs/api/#flask.current_app
"Points to the application handling the request. This is useful for
extensions that want to support multiple applications running side by
side. This is powered by the application context and not by the
request context, so you can change the value of this proxy by using
the app_context() method."
Is app defined anywhere in the script that you've posted?
Also, to help with debugging, when testing you should consider using the run() method with debug mode.
app.run(debug=True)
I'm currently writing a basic dispatch model server based on the Python Eventlet library (http://eventlet.net/doc/). Having looked at the WSGI docs on Eventlet (http://eventlet.net/doc/modules/wsgi.html), I can see that the eventlet.wsgi.server function logs the x-forwarded-for header in addition to the client IP address.
However, the way to obtain this is to attach a file-like object (the default which is sys.stderr) and then have the server pipe that to that object.
I would like to be able to obtain the client IP from within the application itself (i.e. the function that has start_response and environ as parameters). Indeed, an environ key would be perfect for this. Is there a way to obtain the IP address simply (i.e. through the environ dictionary or similar), without having to resort to redirecting the log object somehow?
What you want is in the wsgi environ, specifically environ['REMOTE_ADDR'].
However, if there is a proxy involved, then REMOTE_ADDR will be the address of the proxy, and the client address will be included (most likely) in HTTP_X_FORWARDED_FOR.
Here's a function that should do what you want, for most cases (all credit to Sævar):
def get_client_address(environ):
try:
return environ['HTTP_X_FORWARDED_FOR'].split(',')[-1].strip()
except KeyError:
return environ['REMOTE_ADDR']
You can easily see what is included in the wsgi environ by writing a simple wsgi app and pointing a browser at it, for example:
from eventlet import wsgi
import eventlet
from pprint import pformat
def show_env(env, start_response):
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['%s\r\n' % pformat(env)]
wsgi.server(eventlet.listen(('', 8090)), show_env)
And combining the two ...
from eventlet import wsgi
import eventlet
from pprint import pformat
def get_client_address(environ):
try:
return environ['HTTP_X_FORWARDED_FOR'].split(',')[-1].strip()
except KeyError:
return environ['REMOTE_ADDR']
def show_env(env, start_response):
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['%s\r\n\r\nClient Address: %s\r\n' % (pformat(env), get_client_address(env))]
wsgi.server(eventlet.listen(('', 8090)), show_env)