We Keep Coding

Flask keep live server working for tests (Selenium)

First of all I am aware of flask-testing library with LiveServerTestCase class but it hasn't updated since 2017 and GitHub full of issues of it not working neither on Windows or MacOs and I haven't found any other solutions.
I am trying to write some tests for flask app using selenium to validate FlaskForms inside this app.
Simple test like this:
def test_start(app):
driver.get("http://127.0.0.1:5000/endpoint")
authenticate(driver)
falls on selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_CONNECTION_REFUSED error. (As far as I understood in my case app creates in #pytest.fixtures and immediately shuts down and I need to find a way to keep it running for the whole test duration)
My question is: Is it possible to to create some live server in each test that will remain working so I could call API endpoints via selenium?
Simple fixtures if it helps:
#pytest.fixture
def app():
app = create_app()
...
with app.context():
# creating db
...
yield app
also:
#pytest.fixture
def client(app):
"""Test client"""
return app.test_client()

Finally got it all working. My conftest.py
import multiprocessing
import pytest
from app import create_app
#pytest.fixture(scope="session")
def app():
app = create_app()
multiprocessing.set_start_method("fork")
return app
#pytest.fixture
def client(app):
return app.test_client()
Important note that using python <3.8 line multiprocessing.set_start_method("fork") is not necessary (as far as I understood in v.3.8 they refactored multiprocessing module so further upon without this line you would get pickle Error on windows and Mac).
And one simple test looks like
def test_add_endpoint_to_live_server(live_server):
#live_server.app.route('/tests-endpoint')
def test_endpoint():
return 'got it', 200
live_server.start()
res = urlopen(url_for('.te', _external=True))# ".te is a method path I am calling"
assert url_for('.te', _external=True) == "some url"
assert res.code == 200
assert b'got it' in res.read()
Also I am using url_for. The point is every time live server starts on a random port and url_for function generates url with correct port internally. So now live server is running and it is possible to implement selenium tests.

Testing a Flask app with pytest-flask + pytest-selenium (docker)

I am trying to test a Flask web app within a docker container, which is new for me. My stack is the following:
firefox
selenium
pytest-selenium
pytest-flask
Here is my Flask app file:
from flask import Flask
def create_app():
app = Flask(__name__)
return app
app = create_app()
#app.route('/')
def index():
return render_template('index.html')
Now, my test file which verifies the title of my index page:
import pytest
from app import create_app
# from https://github.com/pytest-dev/pytest-selenium/issues/135
#pytest.fixture
def firefox_options(request, firefox_options):
firefox_options.add_argument('--headless')
return firefox_options
# from https://pytest-flask.readthedocs.io/en/latest/tutorial.html#step-2-configure
#pytest.fixture
def app():
app = create_app()
return app
# from https://pytest-flask.readthedocs.io/en/latest/features.html#start-live-server-start-live-server-automatically-default
#pytest.mark.usefixtures('live_server')
class TestLiveServer:
def test_homepage(self, selenium):
selenium.get('http://0.0.0.0:5000')
h1 = selenium.find_element_by_tag_name('h1')
assert h1 == 'title'
When I run my tests with:
pytest --driver Firefox --driver-path /usr/local/bin/firefox test_app.py
I get the following error (which seems due to firefox not in headless mode).
selenium.common.exceptions.WebDriverException: Message: Service /usr/local/bin/firefox unexpectedly exited. Status code was: 1
Error: no DISPLAY environment variable specified
I am able to run firefox --headless but it seems my pytest fixture didn't manage to do the setup. Is there a better way to do this?
Now, if I replace selenium.get() by urlopen just to try the correct initialization of the app and its connection:
def test_homepage(self):
res = urlopen('http://0.0.0.0:5000')
assert b'OK' in res.read()
assert res.code == 200
I get the error:
urllib.error.URLError:
Do I need to boot the live server differently? Or should I change my host + port config somewhere?

Regarding the problem with a direct call with urllib:
Pytest's live server uses random port by default. You can add this parameter to pytest invocation:
--live-server-port 5000
Or without this parameter you can make direct calls to live server like:
import pytest
import requests
from flask import url_for
#pytest.mark.usefixtures('live_server')
def test_something():
r = requests.get(url_for('index', _external=True))
assert r.status_code == 200
I suppose you have view function called index. It would add a correct port number automatically.
But this doesn't have to do anything with docker, how do you run it?
Regarding the problem with Selenium itself - I can imagine docker networks related problem. How do you use it? Do you have eg. docker-compose configuration? Can you share it?

The referenced pytest-selenium issue has:
#pytest.fixture
def firefox_options(firefox_options, pytestconfig):
if pytestconfig.getoption('headless'):
firefox_options.add_argument('-headless')
return firefox_options
Note the - (single dash) preceding headless in add_argument()
(Source)

For late comers, it might be worthwhile taking a look at Xvfb and even more helpful can be this tutorial
Then (in Linux shell) you can enter:
Xvfb :99 &
export DISPLAY=:99
pytest --driver Firefox --driver-path /usr/local/bin/firefox test_app.py
This provides a virtual frame buffer (fake screen) for the application and it outputs all the graphical content there.
Note that I did not encountered this problem, just providing a solution that helped me overcome the mentioned error with an another app.

How to run server as fixture for py.test

I want to write Selenium tests with server as fixture:
import pytest
#pytest.fixture()
def driver(request):
from selenium import webdriver
d = webdriver.Firefox()
request.addfinalizer(d.close)
return d
#pytest.fixture()
def server():
from server import run
run(host="localhost", port=8080)
def test_can_see_echo(driver,server):
page = TestPage(driver)
page.fill_text_in_input("test")
page.click_send()
print page.get_returnet_value()
Function run in server fixture is bottle run function. The problem is that, when I call run() programs go into infinite loop and body of test is not executed. Should I call run in same thread? Is my design fine? In future I want use server fixture to integrate to server state. For example make test "add comment" using Selenium and in the end use server fixture to ask server if this action really happened.

The tests hang because your run(host="localhost", port=8080) starts a server which waits forever. You should start that server in a different thread/process.
Look into something like pytest-xprocess for running external server processes for your tests.

Python holding an webpage active

I am trying have a set of python scripts report their status to a set of micro controllers.
So my idea for this is to have the python scripts each create their own webpage that can be viewed by the micro controllers, but is there anyway to have the script itself keeping the page served, i.e. an apache library so that if the script crashes or is not running the page is not served or a way to make the page have a default value if the script is not running.

You can also have a look at twisted.web
A very basic example:
from twisted.web.server import Site
from twisted.web.resource import Resource
from twisted.internet import reactor
class StatusPageResource(Resource):
isLeaf = True
def __init__(self, param1):
self.param1 = param1
# Call the constructor of the super class
Resource.__init__(self)
def render_GET(self, request):
return "<html><body>%s</body></html>" % self.param1
my_res = Resource()
my_res.putChild('GetStatusPage1', StatusPageResource(param1='abc'))
my_res.putChild('GetStatusPage2', StatusPageResource(param1='xyz'))
factory = Site(my_res)
reactor.listenTCP(8080, factory)
print 'Runnning on port 8080'
reactor.run()
Now point your browser to http://localhost:8080/GetStatusPage1 (for example)

You could use http://docs.python.org/library/simplehttpserver.html or some minimal http server framework like http://flask.pocoo.org/ or http://www.cherrypy.org/.
If you want to feed "live" information to your micro controllers also have a look at comet style long polling requests. You essentially keep downloading "the page" forever and analyse it as a data stream while the server keeps adding updated info at the "end of the page".

Automating Selenium tests in Python

I have a Django project for which I'm trying to write browser interaction tests with Selenium. My goal is to have the tests automated from Hudson/Jenkins. So far I'm able to get the test hitting the Django server, but from the server logs I see it's hitting the url /selenium-server/driver instead of the right path.
Here's my code (based on what was generated by the Selenium IDE plugin for Firefox:
from selenium import selenium
class AccountAdminPageTests(unittest.TestCase):
def setUp(self):
self.selenium = selenium("localhost",
8000,
"*chrome",
"http://localhost:8000/")
self.selenium.start()
self.selenium.open("/")
def test_ok(self):
self.assertTrue(self.selenium.is_text_present('OK'))
def tearDown(self):
self.selenium.stop()
if __name__ == "__main__":
unittest.main()
Any clues?

Never seen the exact error, but I think that Selenium is trying to connect to your app rather than the selenium Server ( a .jar file).
Port of the selenium server should be the first argument to selenium()
That should default to port 4444, you probably have to start it with
$ java -jar selenium-server.jar
FWIW here's how I got selenium tests running on a CI server...
from multiprocessing import Process
from django.test import TestCase
from selenium import selenium
class SeleniumFixtureCase(TestCase):
"""
Wrapper to multiprocess localhost server and selenium instance on one
test run.
"""
def setUp(self):
"Make the selenium connection"
TestCase.setUp(self)
self.server = Process(target=serve)
self.server.start()
self.verificationErrors = []
self.selenium = selenium("localhost", 4444, "*firefox",
"http://localhost:8000/")
self.selenium.start()
def tearDown(self):
"Kill processes"
TestCase.tearDown(self)
self.server.terminate()
self.selenium.stop()
self.assertEqual([], self.verificationErrors)
def _login(self):
"Login as Albert Camus"
self.selenium.open("http://localhost:8000/admin/")
self.selenium.wait_for_page_to_load("30000")
self.selenium.type("id_username", "albert")
self.selenium.type("id_password", "albert")
self.selenium.click("//input[#value='Log in']")
self.selenium.wait_for_page_to_load("30000")

a co-worker and myself created some automated selenium tests using django and selenium 2. It works without having to use the jar files. Here's a link to the code that shows our test cases.

We currently run django tests successfully from Jenkins using django-jenkins:
https://github.com/kmmbvnr/django-jenkins
FWIW nowadays django provides support for Selenium in the form of LiveServerTestCase:
https://docs.djangoproject.com/en/1.4/topics/testing/#django.test.LiveServerTestCase
LiveServerTestCase launches a django server which allows clients like Selenium to connect to it.
Furthermore, you can now use PhantomJs as a web driver for headless testing. This makes CI integration way easier.

The second argument of the selenium() call is supposed to be the Selenium server port number (as written in David's answer), not the tested application's port number. The default port number is 4444. I would replace the call with :
self.selenium = selenium("localhost", 4444, ....

For automating Selenium tests I would definitely use a CI solution like Jenkins. You can configure Jenkins to pull your code repository and trigger the Selenium tests from your server. I have been using Pytest for doing so from Jenkins.
You can find a step by step tutorial of configuring Jenkins with Github and Selenium here: http://www.6020peaks.com/2015/12/how-to-build-a-test-automation-solution-for-your-web-projects/