We Keep Coding

Flask redirect url_for generates Cloud Run service URL instead of domain

Background:
I've built and deployed an app with Google Cloud Firebase. At a high level, I have a Python Flask server running on Cloud Run, and I serve static JS files with Firebase hosting.
Issue:
Sometimes, I want to redirect the user, but I'm sending them to the Cloud Run service URL rather than my app domain.
EDIT: I'm NOT experiencing this in the JS on the browser, but ONLY in the Python on the server.
Python
If a user navigates to a page without being signed in, e.g. following a link, they are redirected to my login page. For example, if someone who is not signed in tries to look at someone else's profile, the following code redirects them to the authentication blueprint's login endpoint:
if not session.get('user'):
return redirect(url_for('authentication.login'))
I would expect them to be redirected to my-app-name.web.app/auth/login but instead they're routed to my-cloudrun-service-name-XXXXX-region.run.app/auth/login. While the pages loaded look the same, they're really not. The second one redirects to the Cloud Run service URL, which doesn't have my static files served by Firebase Hosting.
I'm not really sure, but I believe this happens because Flask is generating the URL using the request context. So the request from the browser hits Cloud Run Load Balancer, which directs the request to my Cloud Run instance, but that means the Flask app in my Cloud Run instance only sees the internal Google Cloud redirect, and it doesn't know my domain.
I've tried solving this by setting app.config['SEVER_NAME'] = my-app-name.web.app, but I just get the "Not Found" page on each request. Is SEVER_NAME the solution but I'm not implementing it correctly, or is there another way to fix the Flask url_for generation?

I've found what I deem to be an effective solution / workaround.
I set the app config to store my BASE_URL when the app is created:
app.config['BASE_URL'] = 'https://my-url.domain'
Then I can access this as application context during requests, even from blueprints:
#blueprint.route('my_route/')
def my_route():
if not session.get('user'):
return redirect(current_app.config['BASE_URL'] + url_for('authentication.login', 302)
This has two key benefits for me:
There's only one place, the app config, to update if I change the domain
url_for is still used, so there's no hardcoding in case blueprints or routes change

Get the URL of a server

I am using flask and flask email for sending an email. When I work on I used the localhost as a base url. I deployed it on the server and email sent is still showing with localhost address.
For example
base_url = "http://localhost:0.0.0.6465"
url=base_url+'/login'
I sent an email(i am using flask-mail) with the url and I can login with the url.
With the same script when I deployed on the server I am getting with same localhost address.I need the server URL should be the base url.
To debug this I tried
url=request.base_url+'/login'
I am getting 404 error in the browser if I use this. I dont want to change the initializtion of base_url because I have to use both in the local as well as in the server.
How can I do that?

You can get the URL to the currently running app through request.host_url. But what you really want to to do to get an external URL to a specific part of your application, is to use url_for as you'd do when referencing your regular endpoints, but with the parameter _external=True:
Given that you have:
#app.route('/login')
def login():
....
You can generate an external URL by using:
from flask import (
url_for,
)
...
url = url_for('login', _external=True)
This will also take into account any proxies in front of your application if you need that, as long as you've used the ProxyFix middleware when setting up your app object.
Since this uses the same mechanism as Flask uses when generating URLs between different pages, it should behave just as you want - i.e. it'll work both on localhost and on the remote host.

Using https as standard with django project

I am learning django and trying to complete my first webapp.
I am using shopify api & boilder plate (starter code) and am having an issue with the final step of auth.
Specifically, the redirect URL -- it's using HTTP:// when it should NOT and I don't know how to change it..
#in my view
def authenticate(request):
shop = request.GET.get('shop')
print('shop:', shop)
if shop:
scope = settings.SHOPIFY_API_SCOPE
redirect_uri = request.build_absolute_uri(reverse('shopify_app_finalize')) #try this with new store url?
print('redirect url', redirect_uri) # this equals http://myherokuapp.com/login/finalize/
permission_url = shopify.Session(shop.strip()).create_permission_url(scope, redirect_uri)
return redirect(permission_url)
return redirect(_return_address(request))
Which is a problem because my app uses the Embedded Shopify SDK which causes this error to occur at the point of this request
Refused to frame 'http://my.herokuapp.com/' because it violates the following Content Security Policy directive: "child-src 'self' https://* shopify-pos://*". Note that 'frame-src' was not explicitly set, so 'child-src' is used as a fallback.
How do i change the URL to use HTTPS?
Thank you so much in advance. Please let me know if I can share any other details but my code is practically identical to that starter code

This is what the Django doc says about build_absolute_uri:
Mixing HTTP and HTTPS on the same site is discouraged, therefore
build_absolute_uri() will always generate an absolute URI with the
same scheme the current request has. If you need to redirect users to
HTTPS, it’s best to let your Web server redirect all HTTP traffic to
HTTPS.
So you can do two things:
Make sure your site runs entirely on HTTPS (preferred option): Setup your web server to use HTTPS, see the Heroku documentation on how to do this. Django will automatically use HTTPS for request.build_absolute_uri if the incoming request is on HTTPS.
I'm not sure what gets passed in the shop parameter but if it contains personal data I'd suggest to use HTTPS anyway.
Create the URL yourself:
url = "https://{host}{path}".format(
host = request.get_host(),
path = reverse('shopify_app_finalize'))
But you will still need to configure your server to accept incoming HTTPS requests.

appcfg.py shows You must be logged in as an administrator

When I try to upload a sample csv data to my GAE app through appcfg.py, it shows the below 401 error.
2015-11-04 10:44:41,820 INFO client.py:571 Refreshing due to a 401 (attempt 2/2)
2015-11-04 10:44:41,821 INFO client.py:797 Refreshing access_token
Error 401: --- begin server output ---
You must be logged in as an administrator to access this.
--- end server output ---
Here is the command I tried,
appcfg.py upload_data --application=dev~app --url=http://localhost:8080/_ah/remote_api --filename=data/sample.csv

This is how we do it in order to use custom authentication.
Custom handler in app.yaml
- url: /remoteapi.*
script: remote_api.app
Custom wsgi app in remote_api.py to override CheckIsAdmin
from google.appengine.ext.remote_api import handler
from google.appengine.ext import webapp
import re
MY_SECRET_KEY = 'MAKE UP PASSWORD HERE' # make one up, use the same one in the shell command
cookie_re = re.compile('^"?([^:]+):.*"?$')
class ApiCallHandler(handler.ApiCallHandler):
def CheckIsAdmin(self):
"""Determine if admin access should be granted based on the
auth cookie passed with the request."""
login_cookie = self.request.cookies.get('dev_appserver_login', '')
match = cookie_re.search(login_cookie)
if (match and match.group(1) == MY_SECRET_KEY
and 'X-appcfg-api-version' in self.request.headers):
return True
else:
self.redirect('/_ah/login')
return False
app = webapp.WSGIApplication([('.*', ApiCallHandler)])
From here we script the uploading of data that was exported from our live app. Use the same password that you made up in the python script above.
echo "MAKE UP PASSWORD HERE" | appcfg.py upload_data --email=some#example.org --passin --url=http://localhost:8080/remoteapi --num_threads=4 --kind=WebHook --filename=webhook.data --db_filename=bulkloader-progress-webhook.sql3
WebHook and webhook.data are specific to the Kind that we exported from production.

I had a similar issue, where appcfg.py was not giving me any credentials dialog, so I could not authenticate. I downgraded from GAELauncher 1.27 to 1.26, and the authentication started working again.
Temporary solution: go to https://console.developers.google.com/storage/browser/appengine-sdks/featured/ to get version 1.9.26
Submitted bug report: https://code.google.com/p/google-cloud-sdk/issues/detail?id=340

You cannot use the appcfg.py upload_data command with the development server [edit: as is; see Josh J's answer]. It only works with the remote_api endpoint running on App Engine and authenticated with OAuth2.
An easy way to load data into the dev server's datastore is to create an endpoint that reads a CSV file and creates the appropriate datastore entities, then hit it with the browser. (Be sure to remove the endpoint before deploying the app, or restrict access to the URL with login: admin.)

You must have an oauth token for a google account that is not an admin of that project. Try passing the --no_cookies flag so that it prompts for authentication again.

Maybe this has something to do with it? From the docs
Connecting your app to the local development server
To use the local development server for your app running locally, you
need to do the following:
Set environment variables. Add or modify your app's Datastore
connection code. Setting environment variables
Create an environment variable DATASTORE_HOST and set it to the host
and port on which the local development server is listening. The
default host and port is http://localhost:8080. (Note: If you use the
port and/or host command line arguments to change these defaults, be
sure to adjust DATASTORE_HOST accordingly.) The following bash shell
example shows how to set this variable:
export DATASTORE_HOST=http://localhost:8080 Create an environment
variable named DATASTORE_DATASET and set it to your dataset ID, as
shown in the following bash shell example:
export DATASTORE_DATASET= Note: Both the Python and Java
client libraries look for the environment variables DATASTORE_HOST and
DATASTORE_DATASET.
Link to Docs
https://cloud.google.com/datastore/docs/tools/devserver

Allow hop-by-hop headers in Django proxy middleware

I need to implement a HTTP proxy in Django and my Google safari led me to a project called django-webproxy.
Although no longer maintained, it's quite simple. Most of the logic relies on a simple proxy Middleware class that intercepts all requests to the Django WSGI server and handles it.
If the Middleware returns any data, the WSGI server simply passes it back to the client but if it returns nothing, Django simply handles the request by passing to the other Middleware.
Everything works fine, pretty much, but I need to implement proxy authentication which mean i have to send a 407 status code to the client with a Proxy-Authenticate header. This sin't allowed by Django as it is a hop-by-hop header and Django throws an exception. How can i hack/force/kludge Django into allowing me to send hop-by-hop headers?
FYI, ihe code for the middleware class can be found here.

from django.core.servers import basehttp
del basehttp._hop_headers['proxy-authenticate']
del basehttp._hop_headers['proxy-authorization']
This worked for me.

django.core.servers.basehttp._hop_headers is no longer with us present in the basehttp module (since Django 1.10).
I know 2 ways to change it:
Start your server like so:
$ python -O ./manage.py runserver --noreload
Change wsgiref.util._hoppish:
import wsgiref.util
wsgiref.util._hoppish = {
'connection': 1, 'keep-alive':1,
'te':1, 'trailers':1, 'transfer-encoding':1,
'upgrade':1
}.__contains__