I am developing a Flask application based web application ( https://github.com/opensourcehacker/sevabot ) which has HTTP based API services.
Many developers are using and extending the API and I'd like to add a feature which prints Flask's HTTP request to Python logging output, so you can see raw HTTP payloads, source IP and headers you get.
What hooks Flask offers where this kind of HTTP request dumping would be the easiest to implement
Are there any existing solutions and best practices to learn from?
Flask makes a standard logger available at at current_app.logger, there's an example configuration in this gist, though you can centralise the logging calls in a before_request handler if you want to log every request:
from flask import request, current_app
#app.before_request
def log_request():
if current_app.config.get('LOG_REQUESTS'):
current_app.logger.debug('whatever')
# Or if you dont want to use a logger, implement
# whatever system you prefer here
# print request.headers
# open(current_app.config['REQUEST_LOG_FILE'], 'w').write('...')
Related
I am trying to setup Azure AD authentication for a web application using FastAPI. I am using the fastapi_msal python package to do this. The problem I am having is that when I go to the web app, I am able to login, but once i am authenticated, it says the redirect URI that the application is using begins with HTTP. However, Azure requires the redirect uri begin with HTTPS unless running the app locally. Does anyone know how I can change the redirect uri to begin with https instead?
The code for my project pretty much exactly resembles the code from this example project here. However, I have found a similar project using Flask instead of FastAPI. And there is a specific portion of the code that addresses this redirect uri problem:
# This section is needed for url_for("foo", _external=True) to automatically
# generate http scheme when this sample is running on localhost,
# and to generate https scheme when it is deployed behind reversed proxy.
# See also https://flask.palletsprojects.com/en/1.0.x/deploying/wsgi-standalone/#proxy-setups
from werkzeug.middleware.proxy_fix import ProxyFix
app.wsgi_app = ProxyFix(app.wsgi_app, x_proto=1, x_host=1)
Does anyone know how I can do something like this for a web app using FastAPI instead?
The full source code for the Flask app can be found here
I'm using an OpenAPI 3.0 specification (swagger.yml) and use Swagger Codegen to create the corresponding Python Flask application stubs. This is how I run the application to expose my Swagger API:
app = connexion.App(__name__, specification_dir='./swagger/')
app.app.json_encoder = encoder.JSONEncoder
app.add_api('swagger.yaml', arguments={'title': 'My Test API'})
# add CORS support to send Access-Control-Allow-Origin header
CORS(app.app)
So far so good. The application logic is handled within the generated Python stubs which are linked by the x-openapi-router-controller: swagger_server.controllers.user_controller.
I now however need to access HTTP Request specific information within the application itself to for example react differently based on the HTTP_CLIENT_IP address
How can I obtain that information within my controller endpoint?
Use Flask's request context.
For example, to get the HTTP_CLIENT_IP, use:
from flask import request
http_client_ip = request.remote_addr
You can read more about request here.
Attached two related links addressing the same issue on request header parameters and how connexion does not forward them to custom controllers. I ended up manually accessing them via
access_token = connexion.request.headers['access_token']
I have an application in Google App Engine that consists in 2 modules (A and B). A handles user requests and it's available without authentication. B is a microservice that perform certain tasks when A requires it. So we have A making requests to B using urlfetch:
from google.appengine.api import urlfetch
from google.appengine.api import app_identity
rpc = urlfetch.create_rpc()
urlfetch.make_fetch_call(
rpc,
"https://b-dot-my-project.appspot.com/some/url",
method='GET',
follow_redirects=False,
headers = {
'X-Appengine-Inbound-Appid': 'my-project',
},
)
response = rpc.get_result()
B's app.yaml looks something like:
runtime: python27
api_version: 1
threadsafe: yes
service: b
handlers:
- url: /.*
script: my_module.app
login: admin
auth_fail_action: unauthorized
In the docs, they suggest:
When issuing a request to another App Engine app, your App Engine app
must assert its identity by adding the header
X-Appengine-Inbound-Appid to the request. If you instruct the URL
Fetch service to not follow redirects, App Engine will add this header
to requests automatically.
No matter what I do, I keep getting a 401 when making this request. Both A and B are deployed in the same project. Tried setting follow_redirects=False and adding the headers X-Appengine-Inbound-Appid manually (though I didn't expect it to work for the reasons described here), still not sure if the header is being set, as the logs for B don't include request headers and the failure condition happens before my handler module gets executed.
I would rather if possible to rely on A authenticating to B rather than just dropping the option login: admin and rely only on the header, as it is nicer to be able to call B from a project admin account (for debugging purposes for example).
Instead of specifying login: admin in your config, use the python library instead: https://cloud.google.com/appengine/docs/standard/python/refdocs/google.appengine.api.users This way you can check for the app engine header first, and fallback to the admin google user.
Instead of login:admin, you could check the header in module B request for 'HTTP_USER_AGENT': 'AppEngine-Google; (+http://code.google.com/appengine; appid: s~my-project)'. That tells you it came from urlfetch, taskqueue, or cron job.
Citing Google App Engine inter module communication authorization the problem I have is that in the Docs (communication between modules) says:
You can configure any manual or basic scaling module to accept
requests from other modules in your app by restricting its handler to
only allow administrator accounts, specifying login: admin for the
appropriate handler in the module's configuration file. With this
restriction in place, any URLFetch from any other module in the app
will be automatically authenticated by App Engine, and any request
that is not from the application will be rejected.
And this is exactly the configuration I have for my module called "api1". In my app.yaml file I have:
# can accept requests from other modules.
# with login: admin and they are authenticated automatically.
- url: /.*
script: _go_app
login: admin
I'm trying now, from a different module in the same app, to make a service call as suggested in the doc using urfetch.fetch() method, and my implementation is:
from google.appengine.api import urlfetch, modules, app_identity
from rest_framework.response import Response, status
#api_view(['POST'])
def validate_email(request):
url = "http://%s/" % modules.get_hostname(module="api1")
payload = json.dumps({"SOME_KEY":"SOME_VALUE"})
appid = app_identity.get_application_id()
result = urlfetch.fetch(url + "emails/validate/document",
follow_redirects=False,
method=urlfetch.POST,
payload=payload,
headers={"Content-Type":"application/json")
return Response({
'status_code': result.status_code,
'content': result.content
}, status=status.HTTP_200_OK)
According to the documentation, having specified the follow_redirects=False, fetch() will automatically insert an header in my call (I've even tried to add it explicitly) with the "X-Appengine-Inbound-Appid" : MY-APP-ID.
Unfortunately I get as result of the fetch call a 302 redirect, if I follow it, it's a redirect to the authentication form. This occurs in Development server as well as in Production.
Can you please let me know how can I call my api1 service inside my validate_email method (belonging to a different module in the same app)?
Is there another way to authenticate the call since it seems the way suggested inside the documentation is not working?
Thank you
As written here this is a tracked issue now on google appengine public issue tracker. So everyone can go there to check for updates.
In the meanwhile I solved the issue removing the login: admin from the app.yaml and in the handler of my service I've checked manually for the existence of the header X-Appengine-Inbound-Appid and its value.
I want to proxy requests made to my Flask app to another web service running locally on the machine. I'd rather use Flask for this than our higher-level nginx instance so that we can reuse our existing authentication system built into our app. The more we can keep this "single sign on" the better.
Is there an existing module or other code to do this? Trying to bridge the Flask app through to something like httplib or urllib is proving to be a pain.
I spent a good deal of time working on this same thing and eventually found a solution using the requests library that seems to work well. It even handles setting multiple cookies in one response, which took a bit of investigation to figure out. Here's the flask view function:
from dotenv import load_dotenv # pip package python-dotenv
import os
#
from flask import request, Response
import requests # pip package requests
load_dotenv()
API_HOST = os.environ.get('API_HOST'); assert API_HOST, 'Envvar API_HOST is required'
#api.route('/', defaults={'path': ''}) # ref. https://medium.com/#zwork101/making-a-flask-proxy-server-online-in-10-lines-of-code-44b8721bca6
#api.route('/<path>')
def redirect_to_API_HOST(path): #NOTE var :path will be unused as all path we need will be read from :request ie from flask import request
res = requests.request( # ref. https://stackoverflow.com/a/36601467/248616
method = request.method,
url = request.url.replace(request.host_url, f'{API_HOST}/'),
headers = {k:v for k,v in request.headers if k.lower() == 'host'},
data = request.get_data(),
cookies = request.cookies,
allow_redirects = False,
)
#region exlcude some keys in :res response
excluded_headers = ['content-encoding', 'content-length', 'transfer-encoding', 'connection'] #NOTE we here exclude all "hop-by-hop headers" defined by RFC 2616 section 13.5.1 ref. https://www.rfc-editor.org/rfc/rfc2616#section-13.5.1
headers = [
(k,v) for k,v in res.raw.headers.items()
if k.lower() not in excluded_headers
]
#endregion exlcude some keys in :res response
response = Response(res.content, res.status_code, headers)
return response
Update April 2021: excluded_headers should probably include all "hop-by-hop headers" defined by RFC 2616 section 13.5.1.
I have an implementation of a proxy using httplib in a Werkzeug-based app (as in your case, I needed to use the webapp's authentication and authorization).
Although the Flask docs don't state how to access the HTTP headers, you can use request.headers (see Werkzeug documentation). If you don't need to modify the response, and the headers used by the proxied app are predictable, proxying is staightforward.
Note that if you don't need to modify the response, you should use the werkzeug.wsgi.wrap_file to wrap httplib's response stream. That allows passing of the open OS-level file descriptor to the HTTP server for optimal performance.
My original plan was for the public-facing URL to be something like http://www.example.com/admin/myapp proxying to http://myapp.internal.example.com/. Down that path leads madness.
Most webapps, particularly self-hosted ones, assume that they're going to be running at the root of a HTTP server and do things like reference other files by absolute path. To work around this, you have to rewrite URLs all over the place: Location headers and HTML, JavaScript, and CSS files.
I did write a Flask proxy blueprint which did this, and while it worked well enough for the one webapp I really wanted to proxy, it was not sustainable. It was a big mess of regular expressions.
In the end, I set up a new virtual host in nginx and used its own proxying. Since both were at the root of the host, URL rewriting was mostly unnecessary. (And what little was necessary, nginx's proxy module handled.) The webapp being proxied to does its own authentication which is good enough for now.