We Keep Coding

Testing a Flask app with pytest-flask + pytest-selenium (docker)

I am trying to test a Flask web app within a docker container, which is new for me. My stack is the following:
firefox
selenium
pytest-selenium
pytest-flask
Here is my Flask app file:
from flask import Flask
def create_app():
app = Flask(__name__)
return app
app = create_app()
#app.route('/')
def index():
return render_template('index.html')
Now, my test file which verifies the title of my index page:
import pytest
from app import create_app
# from https://github.com/pytest-dev/pytest-selenium/issues/135
#pytest.fixture
def firefox_options(request, firefox_options):
firefox_options.add_argument('--headless')
return firefox_options
# from https://pytest-flask.readthedocs.io/en/latest/tutorial.html#step-2-configure
#pytest.fixture
def app():
app = create_app()
return app
# from https://pytest-flask.readthedocs.io/en/latest/features.html#start-live-server-start-live-server-automatically-default
#pytest.mark.usefixtures('live_server')
class TestLiveServer:
def test_homepage(self, selenium):
selenium.get('http://0.0.0.0:5000')
h1 = selenium.find_element_by_tag_name('h1')
assert h1 == 'title'
When I run my tests with:
pytest --driver Firefox --driver-path /usr/local/bin/firefox test_app.py
I get the following error (which seems due to firefox not in headless mode).
selenium.common.exceptions.WebDriverException: Message: Service /usr/local/bin/firefox unexpectedly exited. Status code was: 1
Error: no DISPLAY environment variable specified
I am able to run firefox --headless but it seems my pytest fixture didn't manage to do the setup. Is there a better way to do this?
Now, if I replace selenium.get() by urlopen just to try the correct initialization of the app and its connection:
def test_homepage(self):
res = urlopen('http://0.0.0.0:5000')
assert b'OK' in res.read()
assert res.code == 200
I get the error:
urllib.error.URLError:
Do I need to boot the live server differently? Or should I change my host + port config somewhere?

Regarding the problem with a direct call with urllib:
Pytest's live server uses random port by default. You can add this parameter to pytest invocation:
--live-server-port 5000
Or without this parameter you can make direct calls to live server like:
import pytest
import requests
from flask import url_for
#pytest.mark.usefixtures('live_server')
def test_something():
r = requests.get(url_for('index', _external=True))
assert r.status_code == 200
I suppose you have view function called index. It would add a correct port number automatically.
But this doesn't have to do anything with docker, how do you run it?
Regarding the problem with Selenium itself - I can imagine docker networks related problem. How do you use it? Do you have eg. docker-compose configuration? Can you share it?

The referenced pytest-selenium issue has:
#pytest.fixture
def firefox_options(firefox_options, pytestconfig):
if pytestconfig.getoption('headless'):
firefox_options.add_argument('-headless')
return firefox_options
Note the - (single dash) preceding headless in add_argument()
(Source)

For late comers, it might be worthwhile taking a look at Xvfb and even more helpful can be this tutorial
Then (in Linux shell) you can enter:
Xvfb :99 &
export DISPLAY=:99
pytest --driver Firefox --driver-path /usr/local/bin/firefox test_app.py
This provides a virtual frame buffer (fake screen) for the application and it outputs all the graphical content there.
Note that I did not encountered this problem, just providing a solution that helped me overcome the mentioned error with an another app.

How Django get the live_server_url?

I learned Django functional test from the TDD with Python and adjust to my project.
My FT is really simple, check the title of url.
I use live_server_url to test by selenium.
But it goes to another port number(56458), not 8000.
(When I follow the book, it wasn't)
$ python manage.py runserver &
...
Starting development server at http://127.0.0.1:8000/
...
$ python manage.py test functional_test
...
http://localhost:56458
E
======================================================================
...
My functional_tests/tests.py is :
from django.test import LiveServerTestCase
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import WebDriverException
from time import time, sleep
class NewVistorTest(LiveServerTestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
def tearDown(self):
self.browser.quit()
def test_render_a_list_of_candiates_in_home(self):
self.browser.get(self.live_server_url)
h1_text = self.browser.find_element_by_tag_name('h1').text
self.assertEqual(self.browser.title, 'Voting Dapp')
self.assertEqual(h1_text, 'A Simple Voting Application')
Doc says:
The live server listens on localhost and binds to port 0 which uses a free port assigned by the operating system. The server’s URL can be accessed with self.live_server_url during the tests.
So I try to look listening ports(I think I'm immature for this part):
$ netstat | grep LISTEN
$ # nothing printed!

You use a LiveServerTestCase. It launches a Django server for you. No need to start a server like you did in the previous chapter.
LiveServerTestCase does basically the same as TransactionTestCase with one extra feature: it launches a live Django server in the background on setup, and shuts it down on teardown. This allows the use of automated test clients other than the Django dummy client such as, for example, the Selenium client, to execute a series of functional tests inside a browser and simulate a real user’s actions.
https://docs.djangoproject.com/en/2.0/topics/testing/tools/#django.test.LiveServerTestCase
So it is expected that your test server has some other port than the development server. Also the test server is an empty project with an empty db. So your test needs to create the needed content before you execute the actual test case.
Alternatively you can point to tests to some other environment with --liveserver LIVESERVER. See python manage.py test -h.
I think it is wrong to test against the development server as this data can be altered (manually and by previous tests) and therefore is not reproducible. I believe tests should be entirely self contained, such that it can be run either in isolation or in arbitrary combination with any number of other test cases.

from selenium import webdriver
from django.urls import reverse
import time
class TestApiPages(StaticLiveServerTestCase):
def setUp(self):
self.browser = webdriver.Chrome('functional_test/chromedriver.exe')
def tearDown(self):
self.browser.close()
def test_api_list_displayed(self):
self.browser.get(('%s%s' % (self.live_server_url, '/admin/')))

I got stuck same problem like you. Same exeception.
So what actually problem is my home page url. Where my home page url is like : http://127.0.0.1:8000/api/ but server is trying with http://localhost:56458/.
self.browser.get(('%s%s' % (self.live_server_url, '/api/')))

I am also reading the TDD Python book. I am using Django 2.1, instead of Django 1.11.
I faced the same problem as you described. I found out in setUpClass(), have to call super().setUpClass().
#classmethod
def setUpClass(cls):
super().setUpClass()
Similar in tearDownClass().

Viewing Google App Engine Python tracebacks in local development server due to error in Flask route

I'm developing a Flask app using Google App Engine and testing it on my local machine using the Local Development Server (ie dev_appserver.py).
When my app encounters a runtime exception at the top level of the app.py script, I see the traceback in the terminal running dev_appserver.py. However, when the exception is encountered inside a Flask route, it seems like either the dev server is catching and handling the exception. All I can see in the terminal output is that the route returns a 500 response status:
INFO 2017-06-20 13:18:31,998 devappserver2.py:116] Skipping SDK update check.
INFO 2017-06-20 13:18:32,025 api_server.py:297] Starting API server at: http://localhost:43843
INFO 2017-06-20 13:18:32,037 dispatcher.py:209] Starting module "default" running at: http://localhost:8080
INFO 2017-06-20 13:18:32,037 admin_server.py:116] Starting admin server at: http://localhost:8000
INFO 2017-06-20 13:18:36,048 module.py:809] default: "GET /cron/batch/basket HTTP/1.1" 500 291
One hacky way I can locate the traceback is by passing dev_appserver.py the flag --logs_path=sqlite.db and then viewing the contents of the AppLogs table in the sqlite file, which contains the traceback. But this is hardly a useful workflow.
Anyone know how I can get the traceback for an exception encountered in the handling of a Flask route?

Something like this might help:
app = Flask(__name__)
production_environment = os.getenv('SERVER_SOFTWARE').startswith('Google App Engine/')
if not production_environment:
app.debug = True

My solution to this problem was to create a decorator which catches any exception, prints its traceback to stdout, and then re-raises the exception, which I then add to whichever routes I want to debug:
import traceback
import sys
def print_traceback(func):
"""Decorator that prints exception tracebacks to stdout"""
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
except Exception, e:
print "Exception:"
print "============================="
traceback.print_exc(file=sys.stdout)
print "============================="
raise e
return wrapper
#app.route('/some/route')
#print_traceback
def some_route(task):
pass

Google App Engine deferred.defer() error 404

I'm trying to get running a task in the Task Queue using deferred.defer(). The task is added to the default task queue, but the task fail with a 404 error.
This is the handler:
import webapp2
import models
import defer_ajust_utils
from google.appengine.ext import ndb
from google.appengine.ext import deferred
class ajust_utils(webapp2.RequestHandler):
def get(self):
deferred.defer(defer_ajust_utils.DoTheJob)
application = webapp2.WSGIApplication([('/ajust_utils', ajust_utils)], debug=True)
This is the module defer_ajust_utils :
import logging
import models
from google.appengine.ext import ndb
def DoTheJob():
logging.info("Debut de la mise a jour des utilisateurs")
utilisateurs = models.Utilisateur.query()
utilisateurs = utilisateurs.fetch()
for utilisateur in utilisateurs:
utilisateur.produire_factures_i = False
utilisateur.put()
logging.info("Fin de la mise a jour des utilisateurs")
And my app.yaml file :
application: xxxx
version: dev
runtime: python27
api_version: 1
threadsafe: yes
builtins:
- deferred: on
handlers:
- url: /ajust_utils
script : tempo_ajuster_utils.application
login: admin
Here's the log :
0.1.0.2 - - [10/Mar/2014:17:50:45 -0700] "POST /_ah/queue/deferred HTTP/1.1" 404 113
"http://xxxx.appspot.com/ajust_utils" "AppEngine-Google;
(+http://code.google.com/appengine)" "xxxx.appspot.com" ms=6 cpu_ms=0
cpm_usd=0.000013 queue_name=default task_name=17914595085560382799
app_engine_release=1.9.0 instance=00c61b117c0b3648693af0563b92051423b3cb
Thank you for help!!

If you are using push-to-deploy with git, when you add in a 'builtin' part to the app.yaml, such as in
builtins:
- deferred: on
you need to do a 'normal' gcloud deploy before you run the app. Otherwise it will not update the running app, which causes 404 errors for /_ah/queue/deferred
There is an open bug for this, so vote for it and it may get fixed. https://code.google.com/p/googleappengine/issues/detail?id=10139

I was receiving the same error.
It appears to be a documentation defect with https://cloud.google.com/appengine/articles/deferred
I looked in the source code and found the following, which wasn't in any of the documentation:
In order for tasks to be processed, you need to set up the handler. Add the
following to your app.yaml handlers section:
handlers:
- url: /_ah/queue/deferred
script: $PYTHON_LIB/google/appengine/ext/deferred/handler.py
login: admin
As I have threadsafe: true set in my app.yaml, I had to add the following handler:
- url: /_ah/queue/deferred
script: google.appengine.ext.deferred.deferred.application
login: admin
and then the deferred task queue began working and stopped 404'ing.

I think you have to add the option deferred: on to your apps "builtin" options in the app.yaml Here is an excerpt from
https://developers.google.com/appengine/docs/python/config/appconfig#Python_app_yaml_Builtin_handlers
builtins:
- deferred: on
The following builtin handlers are available:
admin_redirect - For production App Engine, this results in a redirect from /_ah/admin to the admin console. This builtin has no effect in the development web server.
appstats - Enables Appstats at /_ah/stats/, which you can use to measure your application's performance. In order to use Appstats, you also need to install the event recorder.
deferred - Enables the deferred handler at /_ah/queue/deferred. This builtin allows developers to use deferred.defer() to simplify the creation of Task Queue tasks. Also see Background work with the deferred library.

ProtoRPC App Engine Hello World test app not working

I have been trying to get the rather simple Hello World ProtoRPC App Engine example to work, but to no avail. The code from the website just doesn't seem to work unfortunately. I have looked at a number of possible solutions, but couldn't find a complete working set. Any help would be much appreciated! You can see the error (or lack thereof) below:
app.yaml
application: proto-test
version: 1
runtime: python27
api_version: 1
threadsafe: false
handlers:
- url: /hello.*
script: hello.py
hello.py
from protorpc import messages
from protorpc import remote
from protorpc.wsgi import service
package = 'hello'
# Create the request string containing the user's name
class HelloRequest(messages.Message):
my_name = messages.StringField(1, required=True)
# Create the response string
class HelloResponse(messages.Message):
hello = messages.StringField(1, required=True)
# Create the RPC service to exchange messages
class HelloService(remote.Service):
#remote.method(HelloRequest, HelloResponse)
def hello(self, request):
return HelloResponse(hello='Hello there, %s!' % request.my_name)
# Map the RPC service and path (/hello)
app = service.service_mappings([('/hello', HelloService)])
curl command
curl -H 'content-type:application/json' -d '{"my_name":"test1"}' http://proto-test.appspot.com/hello.hello
When I run the above command in the command line, it just returns the prompt without an error. My logs suggest that the curl command sort of worked, but it just didn't provide a response. This is what appears in the logs:
2013-05-08 22:27:07.409 /hello.hello 200 522ms 0kb curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
2620:0:10c8:1007:a800:1ff:fe00:33af - - [08/May/2013:14:27:07 -0700] "POST /hello.hello HTTP/1.1" 200 0 - "curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3" "proto-test.appspot.com" ms=523 cpu_ms=133 loading_request=1 app_engine_release=1.8.0 instance=00c61b117c66197ad84ad9bc61485b292e5129
I 2013-05-08 22:27:07.409
This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application.
An Ajax call through the Chrome JS Console returned the following: SyntaxError: Unexpected token ILLEGAL:
$.ajax({url: ‘/hello.hello’, type: 'POST', contentType: 'application/json', data: ‘{ "my_name": "Bob" }’,dataType: 'json',success: function(response){alert(response.hello);}});

The Java script you posted seems to have syntax errors. Mainly, it looks like you are using the ` character in places instead of the ' character.
The reason why your request is not working is because of how you wrote the app.yaml file. You are using the old Python 2.5 way of calling applications by referring to a script rather than a WSGI application. You can correct it by changing the url handler in app.yaml to:
handlers:
- url: /hello.*
script: hello.app