I want to write Selenium tests with server as fixture:
import pytest
#pytest.fixture()
def driver(request):
from selenium import webdriver
d = webdriver.Firefox()
request.addfinalizer(d.close)
return d
#pytest.fixture()
def server():
from server import run
run(host="localhost", port=8080)
def test_can_see_echo(driver,server):
page = TestPage(driver)
page.fill_text_in_input("test")
page.click_send()
print page.get_returnet_value()
Function run in server fixture is bottle run function. The problem is that, when I call run() programs go into infinite loop and body of test is not executed. Should I call run in same thread? Is my design fine? In future I want use server fixture to integrate to server state. For example make test "add comment" using Selenium and in the end use server fixture to ask server if this action really happened.
The tests hang because your run(host="localhost", port=8080) starts a server which waits forever. You should start that server in a different thread/process.
Look into something like pytest-xprocess for running external server processes for your tests.
Related
First of all I am aware of flask-testing library with LiveServerTestCase class but it hasn't updated since 2017 and GitHub full of issues of it not working neither on Windows or MacOs and I haven't found any other solutions.
I am trying to write some tests for flask app using selenium to validate FlaskForms inside this app.
Simple test like this:
def test_start(app):
driver.get("http://127.0.0.1:5000/endpoint")
authenticate(driver)
falls on selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_CONNECTION_REFUSED error. (As far as I understood in my case app creates in #pytest.fixtures and immediately shuts down and I need to find a way to keep it running for the whole test duration)
My question is: Is it possible to to create some live server in each test that will remain working so I could call API endpoints via selenium?
Simple fixtures if it helps:
#pytest.fixture
def app():
app = create_app()
...
with app.context():
# creating db
...
yield app
also:
#pytest.fixture
def client(app):
"""Test client"""
return app.test_client()
Finally got it all working. My conftest.py
import multiprocessing
import pytest
from app import create_app
#pytest.fixture(scope="session")
def app():
app = create_app()
multiprocessing.set_start_method("fork")
return app
#pytest.fixture
def client(app):
return app.test_client()
Important note that using python <3.8 line multiprocessing.set_start_method("fork") is not necessary (as far as I understood in v.3.8 they refactored multiprocessing module so further upon without this line you would get pickle Error on windows and Mac).
And one simple test looks like
def test_add_endpoint_to_live_server(live_server):
#live_server.app.route('/tests-endpoint')
def test_endpoint():
return 'got it', 200
live_server.start()
res = urlopen(url_for('.te', _external=True))# ".te is a method path I am calling"
assert url_for('.te', _external=True) == "some url"
assert res.code == 200
assert b'got it' in res.read()
Also I am using url_for. The point is every time live server starts on a random port and url_for function generates url with correct port internally. So now live server is running and it is possible to implement selenium tests.
I learned Django functional test from the TDD with Python and adjust to my project.
My FT is really simple, check the title of url.
I use live_server_url to test by selenium.
But it goes to another port number(56458), not 8000.
(When I follow the book, it wasn't)
$ python manage.py runserver &
...
Starting development server at http://127.0.0.1:8000/
...
$ python manage.py test functional_test
...
http://localhost:56458
E
======================================================================
...
My functional_tests/tests.py is :
from django.test import LiveServerTestCase
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import WebDriverException
from time import time, sleep
class NewVistorTest(LiveServerTestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
def tearDown(self):
self.browser.quit()
def test_render_a_list_of_candiates_in_home(self):
self.browser.get(self.live_server_url)
h1_text = self.browser.find_element_by_tag_name('h1').text
self.assertEqual(self.browser.title, 'Voting Dapp')
self.assertEqual(h1_text, 'A Simple Voting Application')
Doc says:
The live server listens on localhost and binds to port 0 which uses a free port assigned by the operating system. The server’s URL can be accessed with self.live_server_url during the tests.
So I try to look listening ports(I think I'm immature for this part):
$ netstat | grep LISTEN
$ # nothing printed!
You use a LiveServerTestCase. It launches a Django server for you. No need to start a server like you did in the previous chapter.
LiveServerTestCase does basically the same as TransactionTestCase with one extra feature: it launches a live Django server in the background on setup, and shuts it down on teardown. This allows the use of automated test clients other than the Django dummy client such as, for example, the Selenium client, to execute a series of functional tests inside a browser and simulate a real user’s actions.
https://docs.djangoproject.com/en/2.0/topics/testing/tools/#django.test.LiveServerTestCase
So it is expected that your test server has some other port than the development server. Also the test server is an empty project with an empty db. So your test needs to create the needed content before you execute the actual test case.
Alternatively you can point to tests to some other environment with --liveserver LIVESERVER. See python manage.py test -h.
I think it is wrong to test against the development server as this data can be altered (manually and by previous tests) and therefore is not reproducible. I believe tests should be entirely self contained, such that it can be run either in isolation or in arbitrary combination with any number of other test cases.
from selenium import webdriver
from django.urls import reverse
import time
class TestApiPages(StaticLiveServerTestCase):
def setUp(self):
self.browser = webdriver.Chrome('functional_test/chromedriver.exe')
def tearDown(self):
self.browser.close()
def test_api_list_displayed(self):
self.browser.get(('%s%s' % (self.live_server_url, '/admin/')))
I got stuck same problem like you. Same exeception.
So what actually problem is my home page url. Where my home page url is like : http://127.0.0.1:8000/api/ but server is trying with http://localhost:56458/.
self.browser.get(('%s%s' % (self.live_server_url, '/api/')))
I am also reading the TDD Python book. I am using Django 2.1, instead of Django 1.11.
I faced the same problem as you described. I found out in setUpClass(), have to call super().setUpClass().
#classmethod
def setUpClass(cls):
super().setUpClass()
Similar in tearDownClass().
I'm having trouble figuring out how to write a unit test for a Flask route that's decorated with #Sockets.route. With normal Flask routes, I can set up a test client as a context manager and use that to send requests, but I'm not sure what the analogue is for web sockets.
This is the code I'm trying to write a unit test for:
from flask_sockets import Sockets
sockets = Sockets(app)
#sockets.route('/test')
def echo_socket(ws):
while not ws.closed:
message = ws.receive()
ws.send('success')
The documentation for Flask-Sockets says I should use the gunicorn worker, but I'm not really clear how to get access to one of those from a unit test (just using the unittest module). I have something like:
def test_echo_socket:
# this is all incorrect
with my_app.app.test_client() as context:
context.get('/test')
self.assertEqual(context.response_code, '101')
Does anyone have any ideas how to get a websocket context for unit testing? Thanks.
I'm trying to write some unit tests for a flask application i'm building, this is the file that contains the flask application
app.py
from flask import Flask
app = Flask(__name__)
#app.route('/')
def home():
return 'hello world'
and then the file that contains the tests
tests.py
from unittest import TestCase, main
from app import app
from multiprocessing import Process
import requests
class Test(TestCase):
#classmethod
def setUpClass(cls):
cls.hostname = 'http://localhost:8000'
with app.app_context():
p = Process(target=app.run, kwargs={'port': 8000})
p.start()
cls.p = p
def test_options(self):
# print(self.p.is_alive()) # returns True but requests doesn't connect
result = requests.post(self.hostname).text
self.assertEqual(result, 'hello world')
#classmethod
def tearDownClass(cls):
cls.p.terminate() # works fine if i comment it out
pass
if __name__ == '__main__':
main()
I came up with an idea to use the requests module to test the application instead of the test_client that comes with flask.
From my understanding of the unittest module,
The setUpClass method is called once, that creates a new process and starts the application.
After that i'm free to run various tests with request.post or request.get.
Then at the end of the day, when all the tests have been run tearDownClass is supposed to terminate the process.
When i try to run the tests this is what i get (The stacktrace is actually longer than this, but it all boils down to this)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7faeeba183c8>: Failed to establish a new connection: [Errno 111] Connection refused',))
Because the flask process doesn't start and requests is unable to connect to localhost via port 8000.
but the strange part is if i comment out cls.p.terminate everything works properly and the test passes, the only problem is that i won't be able to kill the flask process again.
I know i can make the process daemonic or use the test_client like this:
class Test(TestCase):
#classmethod
def setUpClass(cls):
cls.hostname = 'http://localhost:8000'
with app.app_context():
app.config['TESTING'] = True
cls.app = app.test_client()
def test_options(self):
# print(self.p.is_live())
result = self.app.get().data
self.assertEqual(result, b'hello world')
and just move on with my life, but i really want to understand why my code doesn't work when i try to kill the process when i'm tearing down the test suite.
I've tried to use pdb to debug, but for some reason i don't know my terminal gets messed up when i try to type anything in the pdb prompt (Some letters don't show up in the terminal as i type, but when i hit enter i discover that the letters were actually there but were not visible, E.g to type print i might end up having prrrinnnttt because r and t refused to show up when i was pressing them)
I don't know what's happening in your case, but in one of my projects I use a bash script like this to test:
#!/bin/bash
set -eux
python app.py &
sleep 3
set +e
python tests.py
result=$?
kill $(ps aux | grep app.py | grep -v grep | awk '{print $2}')
exit $result
This handles bringing up the server, running the tests, and then killing the server, and there is no mention of handling the server in the python tests.
I have a Django project for which I'm trying to write browser interaction tests with Selenium. My goal is to have the tests automated from Hudson/Jenkins. So far I'm able to get the test hitting the Django server, but from the server logs I see it's hitting the url /selenium-server/driver instead of the right path.
Here's my code (based on what was generated by the Selenium IDE plugin for Firefox:
from selenium import selenium
class AccountAdminPageTests(unittest.TestCase):
def setUp(self):
self.selenium = selenium("localhost",
8000,
"*chrome",
"http://localhost:8000/")
self.selenium.start()
self.selenium.open("/")
def test_ok(self):
self.assertTrue(self.selenium.is_text_present('OK'))
def tearDown(self):
self.selenium.stop()
if __name__ == "__main__":
unittest.main()
Any clues?
Never seen the exact error, but I think that Selenium is trying to connect to your app rather than the selenium Server ( a .jar file).
Port of the selenium server should be the first argument to selenium()
That should default to port 4444, you probably have to start it with
$ java -jar selenium-server.jar
FWIW here's how I got selenium tests running on a CI server...
from multiprocessing import Process
from django.test import TestCase
from selenium import selenium
class SeleniumFixtureCase(TestCase):
"""
Wrapper to multiprocess localhost server and selenium instance on one
test run.
"""
def setUp(self):
"Make the selenium connection"
TestCase.setUp(self)
self.server = Process(target=serve)
self.server.start()
self.verificationErrors = []
self.selenium = selenium("localhost", 4444, "*firefox",
"http://localhost:8000/")
self.selenium.start()
def tearDown(self):
"Kill processes"
TestCase.tearDown(self)
self.server.terminate()
self.selenium.stop()
self.assertEqual([], self.verificationErrors)
def _login(self):
"Login as Albert Camus"
self.selenium.open("http://localhost:8000/admin/")
self.selenium.wait_for_page_to_load("30000")
self.selenium.type("id_username", "albert")
self.selenium.type("id_password", "albert")
self.selenium.click("//input[#value='Log in']")
self.selenium.wait_for_page_to_load("30000")
a co-worker and myself created some automated selenium tests using django and selenium 2. It works without having to use the jar files. Here's a link to the code that shows our test cases.
We currently run django tests successfully from Jenkins using django-jenkins:
https://github.com/kmmbvnr/django-jenkins
FWIW nowadays django provides support for Selenium in the form of LiveServerTestCase:
https://docs.djangoproject.com/en/1.4/topics/testing/#django.test.LiveServerTestCase
LiveServerTestCase launches a django server which allows clients like Selenium to connect to it.
Furthermore, you can now use PhantomJs as a web driver for headless testing. This makes CI integration way easier.
The second argument of the selenium() call is supposed to be the Selenium server port number (as written in David's answer), not the tested application's port number. The default port number is 4444. I would replace the call with :
self.selenium = selenium("localhost", 4444, ....
For automating Selenium tests I would definitely use a CI solution like Jenkins. You can configure Jenkins to pull your code repository and trigger the Selenium tests from your server. I have been using Pytest for doing so from Jenkins.
You can find a step by step tutorial of configuring Jenkins with Github and Selenium here: http://www.6020peaks.com/2015/12/how-to-build-a-test-automation-solution-for-your-web-projects/