I wanted to test redirection in my Bottle application. Unfortunately I did not find the way to test the redirection location. So far I was just able to test that the redirection was held, by testing that BottleException was raised.
def test_authorize_without_token(mocked_database_utils):
with pytest.raises(BottleException) as resp:
auth_utils.authorize()
Is there a way to obtain HTTP response status code or/and redirection location?
Thanks for help.
WebTest is a full-featured and easy way to test WSGI applications. Here's an example that checks for a redirect:
from bottle import Bottle, redirect
from webtest import TestApp
# the real webapp
app = Bottle()
#app.route('/mypage')
def mypage():
'''Redirect'''
redirect('https://some/other/url')
def test_redirect():
'''Test that GET /mypage redirects'''
# wrap the real app in a TestApp object
test_app = TestApp(app)
# simulate a call (HTTP GET)
resp = test_app.get('/mypage', status=[302])
# validate the response
assert resp.headers['Location'] == 'https://some/other/url'
# run the test
test_redirect()
Related
First of all I am aware of flask-testing library with LiveServerTestCase class but it hasn't updated since 2017 and GitHub full of issues of it not working neither on Windows or MacOs and I haven't found any other solutions.
I am trying to write some tests for flask app using selenium to validate FlaskForms inside this app.
Simple test like this:
def test_start(app):
driver.get("http://127.0.0.1:5000/endpoint")
authenticate(driver)
falls on selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_CONNECTION_REFUSED error. (As far as I understood in my case app creates in #pytest.fixtures and immediately shuts down and I need to find a way to keep it running for the whole test duration)
My question is: Is it possible to to create some live server in each test that will remain working so I could call API endpoints via selenium?
Simple fixtures if it helps:
#pytest.fixture
def app():
app = create_app()
...
with app.context():
# creating db
...
yield app
also:
#pytest.fixture
def client(app):
"""Test client"""
return app.test_client()
Finally got it all working. My conftest.py
import multiprocessing
import pytest
from app import create_app
#pytest.fixture(scope="session")
def app():
app = create_app()
multiprocessing.set_start_method("fork")
return app
#pytest.fixture
def client(app):
return app.test_client()
Important note that using python <3.8 line multiprocessing.set_start_method("fork") is not necessary (as far as I understood in v.3.8 they refactored multiprocessing module so further upon without this line you would get pickle Error on windows and Mac).
And one simple test looks like
def test_add_endpoint_to_live_server(live_server):
#live_server.app.route('/tests-endpoint')
def test_endpoint():
return 'got it', 200
live_server.start()
res = urlopen(url_for('.te', _external=True))# ".te is a method path I am calling"
assert url_for('.te', _external=True) == "some url"
assert res.code == 200
assert b'got it' in res.read()
Also I am using url_for. The point is every time live server starts on a random port and url_for function generates url with correct port internally. So now live server is running and it is possible to implement selenium tests.
I have a application running which some where in the midst uses some rest API call. Now for stress test I want to replace this API call with some mock server. Is there any way to do it.
Let me try to put it programmatically so it gets some clarity. I've a some server running at port say 8080
# main server
from flask import Flask
from myapp import Myapp
app = Flask(__name__)
#app.route("/find_solution", methods=["GET"])
def solution() :
return app.sol.find_solution(), 200
def start():
app.sol = Myapp()
return app
Now this Myapp
#myapp
import requests
class Myapp:
def __init__():
self.session = requests.Session()
def find_solution():
myparameters = {"Some parameter that I filled"}
return self.session.request('GET', 'http://api.weatherstack.com/current', params=myparameters)
Now here I want to replace behavior of http://api.weatherstack.com/current without modifying code. i.e some way where I can replace call to http:api.weatherstack.com/current to my local system server.
Any help of lead is appreciated. I am using ubuntu 20.04
So for your scenario if you want to test your api flask comes with mock test client feature.
test_client = app.test_client()
test_client.post('/find_solution', headers={"Content-Type": "application/json"}, data=data)
So for this scenario you can create test cases and get test client instance inside your test case and perform tests at api level. This is a light weight test method rather than the one proposed by you
Refer to the following link for official flask documentation
https://flask.palletsprojects.com/en/1.1.x/testing/#keeping-the-context-around
Cheers
I have a website build using python3.4 and flask...I have generated my own self-signed certificate and I am currently testing my website through localhost.
I am using the python ssl module along with this flask extension: https://github.com/kennethreitz/flask-sslify
context = ('my-cert.pem', 'my-key.pem')
app = Flask(__name__)
sslify = SSLify(app)
...
if __name__ == '__main__':
app.debug = False
app.run(
host="127.0.0.1",
port=int("5000"),
ssl_context=context
)
This does not seem to be working however. I took a look in the sslify source code and this line does not seem to be working
def init_app(self, app):
"""Configures the configured Flask app to enforce SSL."""
app.before_request(self.redirect_to_ssl)
app.after_request(self.set_hsts_header)
Specifically the function call to redirect_to_ssl (I added my own print statement under the redirect_to_ssl function and my statement was never printed)
def redirect_to_ssl(self):
print("THIS IS WORKING")
"""Redirect incoming requests to HTTPS."""
Should we redirect?
criteria = [
request.is_secure,
current_app.debug,
request.headers.get('X-Forwarded-Proto', 'http') == 'https'
]
if not any(criteria) and not self.skip:
if request.url.startswith('http://'):
url = request.url.replace('http://', 'https://', 1)
code = 302
if self.permanent:
code = 301
r = redirect(url, code=code)
return r
I am pretty new to python. Any ideas?
To me, it appears you're making it more complicated than it needs to be. Here is the code I use in my views.py script to force user to HTTPS connections:
#app.before_request
def before_request():
if not request.is_secure:
url = request.url.replace('http://', 'https://', 1)
code = 301
return redirect(url, code=code)
According with the docs, after pip install Flask-SSLify you only need to insert the following code:
from flask import Flask
from flask_sslify import SSLify
app = Flask(__name__)
sslify = SSLify(app)
I have done it and it works very well. Am I missing something in the discussion ?
The Flask Security Guide recommends using Flask-Talisman.
$ pip install flask-talisman
Usage example:
from flask import Flask
from flask_talisman import Talisman
app = Flask(__name__)
Talisman(app)
It forces HTTPS by default (from the README):
force_https, default True, forces all non-debug connects to https.
Personally, I got some errors relating to CSP (Content Security Policy) which I disabled with:
Talisman(app, content_security_policy=None)
But use this at your own risk :)
Thanks to answer from Kelly Keller-Heikkila and comment by jaysqrd I ended up doing this in my Flask app:
from flask import request, redirect
...
#app.before_request
def before_request():
if app.env == "development":
return
if request.is_secure:
return
url = request.url.replace("http://", "https://", 1)
code = 301
return redirect(url, code=code)
I tried the flask_sslify solution suggested by Rodolfo Alvarez but ran into this issue and went with the above solution instead.
If the app is running in development mode or the request is already on https there's no need to redirect.
Here is a flask solution if you are on aws and behind a load balancer. Place it in your views.py
#app.before_request
def before_request():
scheme = request.headers.get('X-Forwarded-Proto')
if scheme and scheme == 'http' and request.url.startswith('http://'):
url = request.url.replace('http://', 'https://', 1)
code = 301
return redirect(url, code=code)
The standard solution is to wrap the request with an enforce_ssl decorator that after checking some flags in the app configuration (flags you can set depending on your debugging needs) modifies the request's url with request.url.
As it is written here.
You can modify the code to make it working with before_request as suggested by #kelly-keller-heikkila
I use a simple extra app that runs on port 80 and redirect people to https:
from flask import Flask,redirect
app = Flask(__name__)
#app.route('/')
def hello():
return redirect("https://example.com", code=302)
if __name__ == '__main__':
app.run(host='0.0.0.0', port=80)
An alternative to the other answers that I've been able to use with great success:
from http import HTTPStatus
from typing import Optional
from flask import Response, redirect, request, url_for
def https_redirect() -> Optional[Response]:
if request.scheme == 'http':
return redirect(url_for(request.endpoint,
_scheme='https',
_external=True),
HTTPStatus.PERMANENT_REDIRECT)
# ..
if app.env == 'production':
app.before_request(https_redirect)
# ..
On app engine flex, add:
from werkzeug.middleware.proxy_fix import ProxyFix
def create_app(config=None):
app = Flask(__name__)
app.wsgi_app = ProxyFix(app.wsgi_app)
In addition to the solution of:
#app.before_request
def before_request():
if not request.is_secure:
url = request.url.replace('http://', 'https://', 1)
code = 301
return redirect(url, code=code)
Otherwise it'll cause infinite redirects since SSL is unwrapped behind the proxy but is noted in the headers.
I ran into the same solution running a Flask application in AWS Elastic Beanstalk behind a load balancer. The following AWS docs provided two steps to configure the environment for http redirects: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/configuring-https-httpredirect.html Following both steps fixed my issue.
One thing to note is that you'll have to create the .ebextenions folder at the root level of your application source bundle and add the config file to that .ebextensions folder. The readme here: https://github.com/awsdocs/elastic-beanstalk-samples explains this in a bit more detail.
For some reason it seems, requests from a Private AWS API Gateway with a VPC endpoint don't include the "X-Forwarded-Proto" header. This can break some of the other solutions (either it doesn't work or it continuously redirects to the same url). The following middleware forces https on most flask generated internal redirects:
class ForceHttpsRedirects:
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
environ["wsgi.url_scheme"] = "https"
return self.app(environ, start_response)
# Usage
app = flask.Flask(__name__)
app.wsgi_app = ForceHttpsRedirects(app.wsgi_app) # Add middleware to force all redirects to https
Use:
app.run(port="443")
All modern browsers automatically use HTTPS when the port is 443 or 8443.
I'm using cloud foundry python app which is behind a load balancer (like https://stackoverflow.com/users/5270172/kelly-keller-heikkila said) .
This resolution helped me by adding (_external and _Scheme to the url_for function). https://github.com/pallets/flask/issues/773
I had the same issue and mine is a brute-force solution, but it works.
Heroku in the past suggested flask_sslify, which is not maintained anymore. Nowadays the proper way in Flask should be flask-talisman, but I tried it and it has bad interactions with boostrap templates.
I tried the anulaibar solution but it did not always worked for me.
The following is what I came up with:
#app.before_request
def before_request():
# If the request is sicure it should already be https, so no need to redirect
if not request.is_secure:
currentUrl = request.url
if currentUrl.startswith('http://'):
# http://example.com -> https://example.com
# http://www.example.com -> https://www.example.com
redirectUrl = currentUrl.replace('http://', 'https://', 1)
elif currentUrl.startswith('www'):
# Here we redirect the case in which the user access the site without typing any http or https
# www.example.com -> https://www.example.com
redirectUrl = currentUrl.replace('www', 'https://www', 1)
else:
# I do not now when this may happen, just for safety
redirectUrl = 'https://www.example.com'
code = 301
return redirect(redirectUrl, code=code)
I have the domain registered in godaddy which is also redirecting to https://www.example.com.
In my case Flask app is sitting behind AWS API Gateway and solutions with #app.before_request were giving me permanent redirects.
The following simple solution finally worked:
#app.after_request
def adjust_response(response):
....
if response.location:
if app.env != "development":
response.location = response.location.replace("http://", "https://", 1)
return response
I'm needing some help setting up unittests for Google Cloud Endpoints. Using WebTest all requests answer with AppError: Bad response: 404 Not Found. I'm not really sure if endpoints is compatible with WebTest.
This is how the application is generated:
application = endpoints.api_server([TestEndpoint], restricted=False)
Then I use WebTest this way:
client = webtest.TestApp(application)
client.post('/_ah/api/test/v1/test', params)
Testing with curl works fine.
Should I write tests for endpoints different? What is the suggestion from GAE Endpoints team?
After much experimenting and looking at the SDK code I've come up with two ways to test endpoints within python:
1. Using webtest + testbed to test the SPI side
You are on the right track with webtest, but just need to make sure you correctly transform your requests for the SPI endpoint.
The Cloud Endpoints API front-end and the EndpointsDispatcher in dev_appserver transforms calls to /_ah/api/* into corresponding "backend" calls to /_ah/spi/*. The transformation seems to be:
All calls are application/json HTTP POSTs (even if the REST endpoint is something else).
The request parameters (path, query and JSON body) are all merged together into a single JSON body message.
The "backend" endpoint uses the actual python class and method names in the URL, e.g. POST /_ah/spi/TestEndpoint.insert_message will call TestEndpoint.insert_message() in your code.
The JSON response is only reformatted before being returned to the original client.
This means you can test the endpoint with the following setup:
from google.appengine.ext import testbed
import webtest
# ...
def setUp(self):
tb = testbed.Testbed()
tb.setup_env(current_version_id='testbed.version') #needed because endpoints expects a . in this value
tb.activate()
tb.init_all_stubs()
self.testbed = tb
def tearDown(self):
self.testbed.deactivate()
def test_endpoint_insert(self):
app = endpoints.api_server([TestEndpoint], restricted=False)
testapp = webtest.TestApp(app)
msg = {...} # a dict representing the message object expected by insert
# To be serialised to JSON by webtest
resp = testapp.post_json('/_ah/spi/TestEndpoint.insert', msg)
self.assertEqual(resp.json, {'expected': 'json response msg as dict'})
The thing here is you can easily setup appropriate fixtures in the datastore or other GAE services prior to calling the endpoint, thus you can more fully assert the expected side effects of the call.
2. Starting the development server for full integration test
You can start the dev server within the same python environment using something like the following:
import sys
import os
import dev_appserver
sys.path[1:1] = dev_appserver._DEVAPPSERVER2_PATHS
from google.appengine.tools.devappserver2 import devappserver2
from google.appengine.tools.devappserver2 import python_runtime
# ...
def setUp(self):
APP_CONFIGS = ['/path/to/app.yaml']
python_runtime._RUNTIME_ARGS = [
sys.executable,
os.path.join(os.path.dirname(dev_appserver.__file__),
'_python_runtime.py')
]
options = devappserver2.PARSER.parse_args([
'--admin_port', '0',
'--port', '8123',
'--datastore_path', ':memory:',
'--logs_path', ':memory:',
'--skip_sdk_update_check',
'--',
] + APP_CONFIGS)
server = devappserver2.DevelopmentServer()
server.start(options)
self.server = server
def tearDown(self):
self.server.stop()
Now you need to issue actual HTTP requests to localhost:8123 to run tests against the API, but again can interact with GAE APIs to set up fixtures, etc. This is obviously slow as you're creating and destroying a new dev server for every test run.
At this point I use the Google API Python client to consume the API instead of building the HTTP requests myself:
import apiclient.discovery
# ...
def test_something(self):
apiurl = 'http://%s/_ah/api/discovery/v1/apis/{api}/{apiVersion}/rest' \
% self.server.module_to_address('default')
service = apiclient.discovery.build('testendpoint', 'v1', apiurl)
res = service.testresource().insert({... message ... }).execute()
self.assertEquals(res, { ... expected reponse as dict ... })
This is an improvement over testing with CURL as it gives you direct access to the GAE APIs to easily set up fixtures and inspect internal state. I suspect there is an even better way to do integration testing that bypasses HTTP by stitching together the minimal components in the dev server that implement the endpoint dispatch mechanism, but that requires more research time than I have right now.
webtest can be simplified to reduce naming bugs
for the following TestApi
import endpoints
import protorpc
import logging
class ResponseMessageClass(protorpc.messages.Message):
message = protorpc.messages.StringField(1)
class RequestMessageClass(protorpc.messages.Message):
message = protorpc.messages.StringField(1)
#endpoints.api(name='testApi',version='v1',
description='Test API',
allowed_client_ids=[endpoints.API_EXPLORER_CLIENT_ID])
class TestApi(protorpc.remote.Service):
#endpoints.method(RequestMessageClass,
ResponseMessageClass,
name='test',
path='test',
http_method='POST')
def test(self, request):
logging.info(request.message)
return ResponseMessageClass(message="response message")
the tests.py should look like this
import webtest
import logging
import unittest
from google.appengine.ext import testbed
from protorpc.remote import protojson
import endpoints
from api.test_api import TestApi, RequestMessageClass, ResponseMessageClass
class AppTest(unittest.TestCase):
def setUp(self):
logging.getLogger().setLevel(logging.DEBUG)
tb = testbed.Testbed()
tb.setup_env(current_version_id='testbed.version')
tb.activate()
tb.init_all_stubs()
self.testbed = tb
def tearDown(self):
self.testbed.deactivate()
def test_endpoint_testApi(self):
application = endpoints.api_server([TestApi], restricted=False)
testapp = webtest.TestApp(application)
req = RequestMessageClass(message="request message")
response = testapp.post('/_ah/spi/' + TestApi.__name__ + '.' + TestApi.test.__name__, protojson.encode_message(req),content_type='application/json')
res = protojson.decode_message(ResponseMessageClass,response.body)
self.assertEqual(res.message, 'response message')
if __name__ == '__main__':
unittest.main()
I tried everything I could think of to allow these to be tested in the normal way. I tried hitting the /_ah/spi methods directly as well as even trying to create a new protorpc app using service_mappings to no avail. I'm not a Googler on the endpoints team so maybe they have something clever to allow this to work but it doesn't appear that simply using webtest will work (unless I missed something obvious).
In the meantime you can write a test script that starts the app engine test server with an isolated environment and just issue http requests to it.
Example to run the server with an isolated environment (bash but you can easily run this from python):
DATA_PATH=/tmp/appengine_data
if [ ! -d "$DATA_PATH" ]; then
mkdir -p $DATA_PATH
fi
dev_appserver.py --storage_path=$DATA_PATH/storage --blobstore_path=$DATA_PATH/blobstore --datastore_path=$DATA_PATH/datastore --search_indexes_path=$DATA_PATH/searchindexes --show_mail_body=yes --clear_search_indexes --clear_datastore .
You can then just use requests to test ala curl:
requests.get('http://localhost:8080/_ah/...')
If you don't want to test the full HTTP stack as described by Ezequiel Muns, you can also just mock out endpoints.method and test your API definition directly:
def null_decorator(*args, **kwargs):
def decorator(method):
def wrapper(*args, **kwargs):
return method(*args, **kwargs)
return wrapper
return decorator
from google.appengine.api.users import User
import endpoints
endpoints.method = null_decorator
# decorator needs to be mocked out before you load you endpoint api definitions
from mymodule import api
class FooTest(unittest.TestCase):
def setUp(self):
self.api = api.FooService()
def test_bar(self):
# pass protorpc messages directly
self.api.foo_bar(api.MyRequestMessage(some='field'))
My solution uses one dev_appserver instance for the entire test module, which is faster than restarting the dev_appserver for each test method.
By using Google's Python API client library, I also get the simplest and at the same time most powerful way of interacting with my API.
import unittest
import sys
import os
from apiclient.discovery import build
import dev_appserver
sys.path[1:1] = dev_appserver.EXTRA_PATHS
from google.appengine.tools.devappserver2 import devappserver2
from google.appengine.tools.devappserver2 import python_runtime
server = None
def setUpModule():
# starting a dev_appserver instance for testing
path_to_app_yaml = os.path.normpath('path_to_app_yaml')
app_configs = [path_to_app_yaml]
python_runtime._RUNTIME_ARGS = [
sys.executable,
os.path.join(os.path.dirname(dev_appserver.__file__),
'_python_runtime.py')
]
options = devappserver2.PARSER.parse_args(['--port', '8080',
'--datastore_path', ':memory:',
'--logs_path', ':memory:',
'--skip_sdk_update_check',
'--',
] + app_configs)
global server
server = devappserver2.DevelopmentServer()
server.start(options)
def tearDownModule():
# shutting down dev_appserver instance after testing
server.stop()
class MyTest(unittest.TestCase):
#classmethod
def setUpClass(cls):
# build a service object for interacting with the api
# dev_appserver must be running and listening on port 8080
api_root = 'http://127.0.0.1:8080/_ah/api'
api = 'my_api'
version = 'v0.1'
discovery_url = '%s/discovery/v1/apis/%s/%s/rest' % (api_root, api,
version)
cls.service = build(api, version, discoveryServiceUrl=discovery_url)
def setUp(self):
# create a parent entity and store its key for each test run
body = {'name': 'test parent'}
response = self.service.parent().post(body=body).execute()
self.parent_key = response['parent_key']
def test_post(self):
# test my post method
# the tested method also requires a path argument "parent_key"
# .../_ah/api/my_api/sub_api/post/{parent_key}
body = {'SomeProjectEntity': {'SomeId': 'abcdefgh'}}
parent_key = self.parent_key
req = self.service.sub_api().post(body=body,parent_key=parent_key)
response = req.execute()
etc..
After digging through the sources, I believe things have changed in endpoints since Ezequiel Muns's (excellent) answer in 2014. For method 1 you now need to request from /_ah/api/* directly and use the correct HTTP method instead of using the /_ah/spi/* transformation. This makes the test file look like this:
from google.appengine.ext import testbed
import webtest
# ...
def setUp(self):
tb = testbed.Testbed()
# Setting current_version_id doesn't seem necessary anymore
tb.activate()
tb.init_all_stubs()
self.testbed = tb
def tearDown(self):
self.testbed.deactivate()
def test_endpoint_insert(self):
app = endpoints.api_server([TestEndpoint]) # restricted is no longer required
testapp = webtest.TestApp(app)
msg = {...} # a dict representing the message object expected by insert
# To be serialised to JSON by webtest
resp = testapp.post_json('/_ah/api/test/v1/insert', msg)
self.assertEqual(resp.json, {'expected': 'json response msg as dict'})
For searching's sake, the symptom of using the old method is endpoints raising a ValueError with Invalid request path: /_ah/spi/whatever. Hope that saves someone some time!
I'd like to unit test responses from the Google App Engine webapp.WSGIApplication, for example request the url '/' and test that the responses status code is 200, using GAEUnit. How can I do this?
I'd like to use the webapp framework and GAEUnit, which runs within the App Engine sandbox (unfortunately WebTest does not work within the sandbox).
I have added a sample application to the GAEUnit project which demonstrates how to write and execute a web test using GAEUnit. The sample includes a slightly modified version of the 'webtest' module ('import webbrowser' is commented out, as recommended by David Coffin).
Here's the 'web_tests.py' file from the sample application 'test' directory:
import unittest
from webtest import TestApp
from google.appengine.ext import webapp
import index
class IndexTest(unittest.TestCase):
def setUp(self):
self.application = webapp.WSGIApplication([('/', index.IndexHandler)], debug=True)
def test_default_page(self):
app = TestApp(self.application)
response = app.get('/')
self.assertEqual('200 OK', response.status)
self.assertTrue('Hello, World!' in response)
def test_page_with_param(self):
app = TestApp(self.application)
response = app.get('/?name=Bob')
self.assertEqual('200 OK', response.status)
self.assertTrue('Hello, Bob!' in response)
Actually WebTest does work within the sandbox, as long as you comment out
import webbrowser
in webtest/__init__.py