I would like to upgrade my app engine python version from Python 2 to Python 3. But in second generation app engine we cannot use login field in the handler in app.yaml to make certain pages in app engine only accessible to admin.
As per the guidelines Google suggests as follows: The login field is not supported. Use Cloud Identity and Access Management for user management.
I am not able to figure how can I use Identity and Access Management to control login access?
Are you trying to have admin only endpoints that can actually be used called by an admin user? Or are you trying to have admin only endpoints that are only meant to run cron jobs and/or enqueue tasks?
If it is the former (i.e. have pages/handlers that will actually be viewed by admin personnel), then the dcumentation here may be what you're looking for. Unfortunately, as I have noticed with app engine documentation, you may have to read pages upon pages of "theory" and never see sample code you can try to actually use. My guess however is that you will probably end up writing a decorator to check user authorization and authentication, found below.
If you are only trying to limit access to endpoints to secure running cron jobs and queueing tasks, then you are probably looking for this and this solution. Basically, you write a decorator to verify if the endpoint/handler is being called by a cron job or a task queue. Here's working code which should be good to run:
# main.py
from flask import Flask, request, redirect, render_template
app = Flask(__name__)
# Define the decorator to protect your end points
def validate_cron_header(protected_function):
def cron_header_validator_wrapper(*args, **kwargs):
# https://cloud.google.com/appengine/docs/standard/python3/scheduling-jobs-with-cron-yaml#validating_cron_requests
header = request.headers.get('X-Appengine-Cron')
# If you are validating a TASK request from a TASK QUEUE instead of a CRON request, then use 'X-Appengine-TaskName' instead of 'X-Appengine-Cron'
# example:
# header = request.headers.get('X-Appengine-TaskName')
# Other possible headers to check can be found here: https://cloud.google.com/tasks/docs/creating-appengine-handlers#reading_app_engine_task_request_headers
# If the header does not exist, then don't run the protected function
if not header:
# here you can raise an error, redirect to a page, etc.
return redirect("/")
# Run and return the protected function
return protected_function(*args, **kwargs)
# The line below is necessary to allow the use of the wrapper on multiple endpoints
# https://stackoverflow.com/a/42254713
cron_header_validator_wrapper.__name__ = protected_function.__name__
return cron_header_validator_wrapper
#app.route("/example/protected/handler")
#validate_cron_header
def a_protected_handler():
# Run your code here
your_response_or_error_etc = "text"
return your_response_or_error_etc
#app.route("/yet/another/example/protected/handler/<myvar>")
#validate_cron_header
def another_protected_handler(some_var=None):
# Run your code here
return render_template("my_sample_template", some_var=some_var)
Now you have to use Cloud IAM Client Librares in your code in order to provide access. You can find an example of how to use it in Python 3 here.
User authentication in most of Google Cloud Platform is very different from the Python 2 App Engine approach. You could fully implement user login in a variety of ways in your app to make this restriction, but you can't just set a property in a .yaml file to do this.
Or, try putting your admin functions all in a separate App Engine service, then use IAP (Identity-Aware Proxy) to restrict access to that service to desired users. This page should help. And here's a codelab I wrote that has detailed steps to protect an entire App Engine app, not just one service, with IAP.
Things have changed since this question was asked so providing an updated answer
As of March, 2022
You can still use login:required in app.yaml file and it will force a visitor to login with their gmail account.
Python 3 now supports users API (see announcement) which means that if a page is protected by login: required, you can now call is_current_user_admin() on the route handler to confirm it is your administrator
Even if you don't wish to use the users API, you can still get the details of the logged in user (for pages protected by login:required) by checking for any of the following headersX-Appengine-User-Email, X-Appengine-User-Id. You can refer to my response to this other SO question
Related
Hi i have a flask application that Build as a docker image to serve as an api
this image is deployed to multiple environments (DEV/QA/PROD)
i want to use an applicationInsight for each environment
using a single application Insight works fine
here is a code snippet
app.config['APPINSIGHTS_INSTRUMENTATIONKEY'] = APPINSIGHTS_INSTRUMENTATIONKEY
appinsights = AppInsights(app)
#app.after_request
def after_request(response):
appinsights.flush()
return response
but to have multiple application i need to configure app.config with the key of the app insight
i thought of this solution which thourghs errors
here a snippet :
app = Flask(__name__)
def monitor(key):
app.config['APPINSIGHTS_INSTRUMENTATIONKEY'] = key
appinsights = AppInsights(app)
#app.after_request
def after_request(response):
appinsights.flush()
return response
#app.route("/")
def hello():
hostname = urlparse(request.base_url).hostname
print(hostname)
if hostname == "dev url":
print('Dev')
monitor('3ed57a90-********')
if hostname == "prod url":
print('prod')
monitor('941caeca-********-******')
return "hello"
this example contains the function monitor which reads the url and decide which app key to give so it can send metrics to the right place but apparently i can't do those processes after the request is sent (is there a way a config variable can be changed based on the url condition ?)
error Message :
AssertionError: The setup method 'errorhandler' can no longer be called on the application. It has already handled its first request, any changes will not be applied consistently. Make sure all imports, decorators, functions, etc. needed to set up the application are done before running it.
i hope someone can guide me to a better solution
thanks in advance
AFAIK, Normally Application Insights SDK collect the telemetry data, and it has sent to Azure by batch. So, you have to keep a single application insights resource for an application. Use staging for use different application insights for same application.
When the request started for the service till to complete his response the Application insights has taking care of the specific service life cycle. The application while start to end it will track the information. So that we can't use more than one Application Insights in single application.
When Application starts the AI start collecting telemetry data when Application stops then the AI stops gathering telemetry information. We were using Flush to even though in between application stops to send information to AI.
I have tried what you have used. It confirms the same in the log
I have tried with single application insights I can be able to collect all telemetry information.
I have a function which calls several API's and updates the database upon being called. I want to schedule the function to run daily at specific time.
Already tried flask_apscheduler and APScheduler which gives this error:
This typically means that you attempted to use functionality that needed an active HTTP request. Consult the documentation on testing for information about how to avoid this problem.
Any leads on this will be helpful.
You should:
Post the code where you define your flask application.
Specify how you try to access the app.
How you're calling the APIs.
Whether those APIs are 3rd party or part of your blueprint.
However, this is probably a context issue. I have come across a similar one with SQLAlchemy before.
You will need to somehow get access to your app, either by using app_context or by importing current_app from Flask and accessing the config.
Assuming you imported the app where your function is used, try this:
with app.app_context():
# call your function here
Refer to this document for more information: Flask Documentation
Another approach you can try, is passing your app configurations through a config class object.
You can define the jobs you want to schedule and pass a reference to your function inside.
Check this example from flask-apscheduler repository on GitHub.
I'm building a Flask app and started using Flask-Login for authentication. What troubles me is that Flask-Login calls the load_user callback for every request that flask handles. Here is the example from https://flask-login.readthedocs.org/en/latest/#how-it-works:
#login_manager.user_loader
def load_user(userid):
return User.get(userid)
To retrieve the user, I need to pass a session token a remote web service across a VPN, and the remote web service does a db query -- this results in noticeable latency on every web request. My load_user looks something like this:
#login_manager.user_loader
def load_user(userid):
# notice that I don't even use the userid arg
try:
# have to get session_token from session; why not
# just get the entire user object from session???
token = session.get('session_token')
user_profile = RestClient().get_user_profile(token)
return User(user_profile['LDAP_ID'])
except:
return None
Seems like maybe I'm subverting the framework. Could just store/retrieve user from session, so why bother to get it from the web service? This option also seems to subvert Flask-Login, but eliminates latency and makes good use of session.
The best way to handle this is to cache session information using something like memcached or redis (look into Flask-Cache for help).
You should have a key-value cache store that structures the cache like so:
key: sessionID
value: user object
This is what most frameworks do -- Flask-Login is a generic tool -- so you have to implement this yourself.
Incidentally, if you're looking for a way to abstract away that nasty LDAP stuff on the backend, you might want to check out https://stormpath.com -- they sync with LDAP servers, and provide a REST API on top of it. There's also a Flask library for interacting with it: Flask-Stormpath.
I'm writing a Django app that uses a REST api I created. The purpose is to prove api use cases using the web app. In my view, I therefore call the api using the python-requests library like so:
def my_view_method(request):
if request.method == 'POST':
form = MyForm(request.POST)
if form.is_valid():
data = form.cleaned_data
data_to_post = {
'fieldA': data.get('fieldA_in_form'),
'fieldB': data.get('fieldB_in_form'),
}
post_url = "http://%s/%s/" % (request.get_host(), 'entries')
logger.info("request api url: "+ post_url)
r = requests.post(post_url, data=data_to_post)
return HttpResponseRedirect('/')
else:
form = MyForm()
return render(request, 'myview.html', { 'form': form })
I have verified using Unit Tests that POSTing to /entries/ with valid data results in the correct database updates.
url = '/entries/'
#verify initial db state
data = { 'fieldA': value1, 'fieldB': value2 }
response = self.client.post(url, data, format='json')
# verify db was updated
In my Functional Tests, I use LiveServerTestCase and interact with the Form. When the test submits the form, the browser tab shows "Connecting..." in the title and the test case hangs. It wasn't so when I was directly interacting with the database instead of calling the api using requests, so that must be the source of the delay.
Is there something about how LiveServerTestCase works that I'm not understanding here?
Could it be that the LiveServerTestCase server can only handle a single request at a time? So it hangs because it can't deal with a request-from-within-a-request?
The source says it will "handle one request at a time", but then again it says "without blocking", so that's ambiguous...
I think you're going to be best off dropping LiveServerTestCase and just rolling your own test runner. You can use setUp to spin up runserver in a separate process, and tearDown to reset the database (manage.py flush?). If you want to use a test database, you could use a different settings.py, or just move the "real" database out of the way for the duration of the test...
The request is hanging after the test is complete because you're making the request via the requests library and not the official test client described here.
You don't even need to do this though, it makes more sense to test the API directly and not spin up a web server through Django.
This is a decription of LiveServerTestCase from the documentation here
LiveServerTestCase launches a live Django server in the background on
setup, and shuts it down on teardown. This allows the use of automated
test clients such as, for example, the Selenium client, to execute a
series of functional tests inside a browser and simulate a real user’s
actions.
There's no need to even initalise the Django application if you want to test the API itself.
One method for testing I like to follow for problems like this is the component integration methodology, sort of described on Wikipedia here.
In this example we would test the Django application (the front end) separately from the service layer (the API). You may still use the API's when testing the front end, but the separation exists to define how you write the tests.
Since the API is a Flask application, I would use Flasks built in testing tools described here.
The API has nothing to do with Django, yes it is consumed by a Django application, and that application has a hard dependency on that API, but you're wanting to specifically test the API itself.
To aid in UI testing, you could use the API as part of your test fixture/setUp routine to save yourself from using the UI to add any test data needed for execution. However, if you're wanting to test the API itself then doing it through the Django client is not going to work due to the issue mentioned above.
this is my first question on stackoverflow and I'm new to programming:
What is the right way to load data into the GAE datastore when deploying my app? This should only happen once at deployment.
In other words: How can I call methods in my code, such that these methods are only called when I deploy my app?
The GAE documentation for python2.7 says, that one shouldn't call a main function, so I can't do this:
if __name__ == '__main__':
initialize_datastore()
main()
Create a handler that is restricted to admins only. When that handler is invoked with a simple GET request you could have it check to see if the seed data exists and if it doesn't, insert it.
Configuring a handler to require login or administrator status.
Another option is to write a Python script that utilizes the Remote API. This would allow you to access local data sources such as a CSV file or a locally hosted database and wouldn't require you to create a potentially unwieldy handler.
Read about the Remote API in the docs.
Using the Remote API Shell - Google App Engine