We Keep Coding

Testing a Flask app with pytest-flask + pytest-selenium (docker)

I am trying to test a Flask web app within a docker container, which is new for me. My stack is the following:
firefox
selenium
pytest-selenium
pytest-flask
Here is my Flask app file:
from flask import Flask
def create_app():
app = Flask(__name__)
return app
app = create_app()
#app.route('/')
def index():
return render_template('index.html')
Now, my test file which verifies the title of my index page:
import pytest
from app import create_app
# from https://github.com/pytest-dev/pytest-selenium/issues/135
#pytest.fixture
def firefox_options(request, firefox_options):
firefox_options.add_argument('--headless')
return firefox_options
# from https://pytest-flask.readthedocs.io/en/latest/tutorial.html#step-2-configure
#pytest.fixture
def app():
app = create_app()
return app
# from https://pytest-flask.readthedocs.io/en/latest/features.html#start-live-server-start-live-server-automatically-default
#pytest.mark.usefixtures('live_server')
class TestLiveServer:
def test_homepage(self, selenium):
selenium.get('http://0.0.0.0:5000')
h1 = selenium.find_element_by_tag_name('h1')
assert h1 == 'title'
When I run my tests with:
pytest --driver Firefox --driver-path /usr/local/bin/firefox test_app.py
I get the following error (which seems due to firefox not in headless mode).
selenium.common.exceptions.WebDriverException: Message: Service /usr/local/bin/firefox unexpectedly exited. Status code was: 1
Error: no DISPLAY environment variable specified
I am able to run firefox --headless but it seems my pytest fixture didn't manage to do the setup. Is there a better way to do this?
Now, if I replace selenium.get() by urlopen just to try the correct initialization of the app and its connection:
def test_homepage(self):
res = urlopen('http://0.0.0.0:5000')
assert b'OK' in res.read()
assert res.code == 200
I get the error:
urllib.error.URLError:
Do I need to boot the live server differently? Or should I change my host + port config somewhere?

Regarding the problem with a direct call with urllib:
Pytest's live server uses random port by default. You can add this parameter to pytest invocation:
--live-server-port 5000
Or without this parameter you can make direct calls to live server like:
import pytest
import requests
from flask import url_for
#pytest.mark.usefixtures('live_server')
def test_something():
r = requests.get(url_for('index', _external=True))
assert r.status_code == 200
I suppose you have view function called index. It would add a correct port number automatically.
But this doesn't have to do anything with docker, how do you run it?
Regarding the problem with Selenium itself - I can imagine docker networks related problem. How do you use it? Do you have eg. docker-compose configuration? Can you share it?

The referenced pytest-selenium issue has:
#pytest.fixture
def firefox_options(firefox_options, pytestconfig):
if pytestconfig.getoption('headless'):
firefox_options.add_argument('-headless')
return firefox_options
Note the - (single dash) preceding headless in add_argument()
(Source)

For late comers, it might be worthwhile taking a look at Xvfb and even more helpful can be this tutorial
Then (in Linux shell) you can enter:
Xvfb :99 &
export DISPLAY=:99
pytest --driver Firefox --driver-path /usr/local/bin/firefox test_app.py
This provides a virtual frame buffer (fake screen) for the application and it outputs all the graphical content there.
Note that I did not encountered this problem, just providing a solution that helped me overcome the mentioned error with an another app.

Importing CherryPy fails on openshift

I'm running a cherrypy based app on an openshift gear. Recently I've been getting a "503 service temporarily unavailable" error whenever I try to go to the site. Inspecting the logs, I see I'm getting an ImportError where I try to import CherryPy. This is strange - CherryPy is listed as a dependency in my requirements.txt and used to be imported just fine. I double checked to make sure I'm getting the right path to the openshift activate_this.py and it seems to be correct. I'm not quite sure where to look next; any help would be appreciated. Thanks!
The failed import is at line 14 of app.py:
import os
import files
virtenv = os.path.join(os.environ['OPENSHIFT_PYTHON_DIR'], 'virtenv')
virtualenv = os.path.join(virtenv, 'bin', 'activate_this.py')
conf = os.path.join(files.get_root(), "conf", "server.conf")
try:
execfile(virtualenv, dict(__file__=virtualenv))
print virtualenv
except IOError:
pass
import cherrypy
import wsgi
def mount():
def CORS():
cherrypy.response.headers["Access-Control-Allow-Origin"] = os.environ['OPENSHIFT_APP_DNS']
cherrypy.config.update({"tools.staticdir.root": files.get_root()})
cherrypy.tools.CORS = cherrypy.Tool('before_handler', CORS)
cherrypy.tree.mount(wsgi.application(), "/", conf)
def start():
cherrypy.engine.start()
def end():
cherrypy.engine.exit()
if __name__ == "__main__":
mount()
start()
UPDATE
I eventually saw (when pushing to the openshift repo using git bash CLI) that the dependency installation from requirements.txt was failing with some exceptions I haven't bothered to look into yet. It then goes on to try to install dependencies in setup.py, and that works just fine.
Regarding the port in use issue...I have no idea. I changed my startup from tree.mount and engine.start to quickstart, and everything worked when I pushed to openshift. Just for kicks (and because I need it to run my tests), I switched back to cherrypy.tree.mount, pushed it, and it worked just fine.
Go figure.

I use the app.py entry point for Openshift. Here are several examples on how I start my server using the pyramid framework on Openshift. I use waitress as the server but I have also used the cherrypy wsgi server. Just comment out the code you don't want.
app.py
#Openshift entry point
import os
from pyramid.paster import get_app
from pyramid.paster import get_appsettings
if __name__ == '__main__':
here = os.path.dirname(os.path.abspath(__file__))
if 'OPENSHIFT_APP_NAME' in os.environ: #are we on OPENSHIFT?
ip = os.environ['OPENSHIFT_PYTHON_IP']
port = int(os.environ['OPENSHIFT_PYTHON_PORT'])
config = os.path.join(here, 'production.ini')
else:
ip = '0.0.0.0' #localhost
port = 6543
config = os.path.join(here, 'development.ini')
app = get_app(config, 'main') #find 'main' method in __init__.py. That is our wsgi app
settings = get_appsettings(config, 'main') #don't really need this but is an example on how to get settings from the '.ini' files
# Waitress (remember to include the waitress server in "install_requires" in the setup.py)
from waitress import serve
print("Starting Waitress.")
serve(app, host=ip, port=port, threads=50)
# Cherrypy server (remember to include the cherrypy server in "install_requires" in the setup.py)
# from cherrypy import wsgiserver
# print("Starting Cherrypy Server on http://{0}:{1}".format(ip, port))
# server = wsgiserver.CherryPyWSGIServer((ip, port), app, server_name='Server')
# server.start()
#Simple Server
# from wsgiref.simple_server import make_server
# print("Starting Simple Server on http://{0}:{1}".format(ip, port))
# server = make_server(ip, port, app)
# server.serve_forever()
#Running 'production.ini' method manually. I find this method the least compatible with Openshift since you can't
#easily start/stop/restart your app with the 'rhc' commands. Mabye somebody can suggest a better way :)
# #Don't forget to set the Host IP in 'production.ini'. Use 8080 for the port for Openshift
# You will need to use the 'pre_build' action hook(pkill python) so it stops the existing running instance of the server on OS
# You also will have to set up another custom action hook so rhc app-restart, stop works.
# See Openshifts Origin User's Guide ( I have not tried this yet)
#Method #1
# print('Running pserve production.ini')
# os.system("pserve production.ini &")
#Method #2
#import subprocess
#subprocess.Popen(['pserve', 'production.ini &'])

arduino yun uhttpd flask setup

I'm trying to set up python and flask on the arduino yun. I've managed to run python files via the /etc/config/uhttpd configuration file:
...
list interpreter ".py=/usr/bin/python"
...
The default path for the website's root is: /www in which I've placed a soft link (apps) to the sd card. So now I can run python programs: http://[ip arduino]/apps/helloworld.py
And when I make my first helloflask.py program and run that via python helloflask.py I can see the result at: http://[ip arduino]:5000
But now I want to configure the uhttpd mini webserver (which is capable to exchange information via CGI) to use the flask setup. The URI: http://flask.pocoo.org/docs/deploying/cgi/#server-setup shows some instructions... but I just don't get it. I've made a directory ../apps/uno in which I've placed a __init__.py file with the following content:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "He Flask!"
In the apps dir I've put a file: cgi.py with this content:
from wsgiref.handlers import CGIHandler
from uno import app
CGIHandler().run(app)
Now I when I browse: http://[ip arduino]/cgi.py get a server error occured, contact the administrator (I think this is the CGI interface from uhttpd).
I just don't grasp the CGI configuration for Flask/uhttpd

I looked into this too and got a little further, I was able to setup a simple hello world but once I tried to do something non-trivial I ran into a big issue that uhttpd doesn't support URL rewriting/aliasing. This means your flask app can only be served at the URL of its .py file instead of at a root like http:// (arduino IP) /flaskapp/. None of the routes inside the app will be visible and makes the whole thing unusable.
However, instead of trying to force flask into uhttpd I had great success running the built in server that flask provides. Take a look at this guide I wrote up that uses flask to serve data from a Yun: https://learn.adafruit.com/smart-measuring-cup/overview
The thing to do is add a call to app.run when the script is run, for example make your flask app look like:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "Hello Flask!"
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=True, threaded=True)
Then log in to the Yun and run the script using python. Flask's built in server should start serving the app on http:// (arduino IP) :5000/. Make sure to include the host='0.0.0.0' as it's required to listen on the Yun's external network interface. You probably also want debug=True so there are better error messages (and live reloading of the server when the code changes), and I found threaded=True helps because the default server only handles one connection at a time. The Yun is a relatively slow processor so don't expect to service a lot of concurrent requests, however it's quite capable for providing a simple REST API or web application for a few users.
If you want this server to always run on bootup, edit the /etc/rc.local file to include a call to python and your script.

How can I use a single server to deploy multiple WSGI apps on multiple domains?

Hypothetical
I've got a Linux VPS server named myserver, mounted at myserver.com.
On this server I have two Flask WSGI apps app-one and app-two.
I'd like to deploy these two apps on myserver, but I want them mounted on the domains app-one.com and app-two.com.
The apps have no external dependencies (no databases, caches, etc). They're stand-alone, single-file apps.
I do not have Apache, NGinX or any other webserver software installed; just Python, Flask and two WSGI apps.
I have Python/Flask experience, but I don't have a lot of experience with WSGI deployment or multiple-domain work. Basic instructions and/or reading material appreciated.
Question / TL;DR
How can I use a server mounted at one domain to deploy two WSGI apps to two domains? Do I need to install software especially for this case, or is it just a matter of pointing the apps at my chosen domains?
Thank you for any and all advice.

Once you have DNS set up to point both app-one.com and app-two.com to myserver.com's IP address then you need to set up something to route requests coming in on port 80 (or 443 if you are going to use SSL) to each of your apps. This is normally done with virtual hosts in Apache or nginx.
If you need to run both applications in the same Python process (whether you are using a non-Python webserver as your application container or not) then you will need to dispatch to each of your apps by hand:
from werkzeug.exceptions import NotImplemented
from werkzeug.wsgi import get_host
class DomainDispatcher(object):
"""Simple domain dispatch"""
def __init__(self, domain_handlers, default_handler=None):
self.domain_handlers = domain_handlers
self.default_handler = domain_handlers.get("default", default_handler)
if self.default_handler is None:
self.default_handler = NotImplemented()
def __call__(self, environ, start_response):
host = get_host(environ)
handler = self.domain_handlers.get(host, self.default_handler)
return handler(environ, start_response)
An example of usage:
from app_one import app as app1
from app_two import app as app2
from domain_dispatcher import DomainDispatcher
dispatcher = DomainDispatcher({'app-one.com': app1, 'app-two.com': app2})
if __name__ == '__main__':
# Wrap dispatcher in a WSGI container
# such as CherryPy

Exclude One URL From Python URL Masking

I have never written a python script in my life, but I have a question that can hopefully be solved pretty quickly...
I'm using Google App Engine and Dropbprox. The script uses a custom domain to point to your public DropBox folder for better DropBox URLs. I'd like to be able to redirect users to my main site (jacob.bearce.me) if they visit my dropbox url (dl.bearce.me).
The problems that I'm having:
I've never used GAE or Python before, so I have no idea where to even begin
Putting a index.html file in my GAE project didn't fix it (I was hoping it'd just default to that if there was no filename specified, like it would on a normal site, but no cigar.)
Just a simple redirect if a users visits the main URL is all I'm after, nothing fancy.
My Python file: http://dl.bearce.me/mirror.py

Here's a main.py that issues a redirect for all requests, using the Python 2.5 runtime environment:
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
class MainHandler(webapp.RequestHandler):
def get(self):
self.redirect('http://jacob.bearce.me/')
application = webapp.WSGIApplication([('/.*', MainHandler)],
debug=True)
def main():
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
And here's the app.yaml file you need to route URLs to this handler:
application: myapp
version: 1
runtime: python
api_version: 1
handlers:
- url: .*
script: main.py
(Replace myapp with your actual app ID.)
For more information about creating and uploading an App Engine app in Python, see the Getting Started tutorial.