First of all I am aware of flask-testing library with LiveServerTestCase class but it hasn't updated since 2017 and GitHub full of issues of it not working neither on Windows or MacOs and I haven't found any other solutions.
I am trying to write some tests for flask app using selenium to validate FlaskForms inside this app.
Simple test like this:
def test_start(app):
driver.get("http://127.0.0.1:5000/endpoint")
authenticate(driver)
falls on selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_CONNECTION_REFUSED error. (As far as I understood in my case app creates in #pytest.fixtures and immediately shuts down and I need to find a way to keep it running for the whole test duration)
My question is: Is it possible to to create some live server in each test that will remain working so I could call API endpoints via selenium?
Simple fixtures if it helps:
#pytest.fixture
def app():
app = create_app()
...
with app.context():
# creating db
...
yield app
also:
#pytest.fixture
def client(app):
"""Test client"""
return app.test_client()
Finally got it all working. My conftest.py
import multiprocessing
import pytest
from app import create_app
#pytest.fixture(scope="session")
def app():
app = create_app()
multiprocessing.set_start_method("fork")
return app
#pytest.fixture
def client(app):
return app.test_client()
Important note that using python <3.8 line multiprocessing.set_start_method("fork") is not necessary (as far as I understood in v.3.8 they refactored multiprocessing module so further upon without this line you would get pickle Error on windows and Mac).
And one simple test looks like
def test_add_endpoint_to_live_server(live_server):
#live_server.app.route('/tests-endpoint')
def test_endpoint():
return 'got it', 200
live_server.start()
res = urlopen(url_for('.te', _external=True))# ".te is a method path I am calling"
assert url_for('.te', _external=True) == "some url"
assert res.code == 200
assert b'got it' in res.read()
Also I am using url_for. The point is every time live server starts on a random port and url_for function generates url with correct port internally. So now live server is running and it is possible to implement selenium tests.
Related
Assuming I have this listener defined in my Sanic app:
#app.before_server_start
async def db_setup(*args):
# ... set up the DB as I wish for the app
If I want to unit test this function (with pytest) and I import it in a unit test file with from my.app import db_setup, it seems the test actually starts serving the app, as pytest outputs:
[INFO] Goin' Fast # http://0.0.0.0:8000
[INFO] Starting worker [485]
Now, I know that I can remove the effects of the decorator by doing db_setup = db_setup.__wrapped__, but in order to do this I actually need to import db_setup, which is where the Sanic server fires up.
Is there a way of removing the effects of the decorator at import time?
LE: I've tried patching the Sanic app as follows:
async def test_stuff(mocker):
mocker.patch('myapp.app.app') # last '.app' being `app = Sanic('MyApp')`
imp = importlib.import_module('myapp.app')
db_setup = getattr(imp, 'db_setup')
await db_setup()
but now I get a RuntimeError: Cannot run the event loop while another loop is running for the mocker.patch('myapp.app.app') line.
I am going to make a few assumptions here, so I may need to amend this answer if there are some clarifications.
Before starting, it should be noted that the decorator itself will not start your web server. That will run in one of two scenarios:
You are running app.run() somewhere in the global scope
You are using the Sanic TestClient, which specifically operates by running your application's web server
Now, from what I can understand, you are trying to run db_setup in a test manually by calling it as a function, but you do not want it to attach as a listener to the application in your tests.
You can get access to all of your application instance's listeners in the app.listeners property. Therefore one solution would be something like this:
# conftest.py
import pytest
from some.place import app as myapp
#pytest.fixture
def app():
myapp.listeners = {}
return myapp
Like I said earlier, this will just empty out your listeners. It does not actually impact your application starting, so I am not sure it has the utility that you are looking for.
You should be able to have something like this:
from unittest.mock import Mock
import pytest
from sanic import Request, Sanic, json
app = Sanic(__name__)
#app.get("/")
async def handler(request: Request):
return json({"foo": "bar"})
#app.before_server_start
async def db_setup(app, _):
app.ctx.db = 999
#pytest.mark.asyncio
async def test_sample():
await db_setup(app, Mock())
assert app.ctx.db == 999
For the sake of ease, it is all in the same scope, but even if the test functions, the application instance, and the listener are spread across different modules, the end result is the same: You can run db_setup as any other function and it does not matter if it is registered as a listener or not.
I have a application running which some where in the midst uses some rest API call. Now for stress test I want to replace this API call with some mock server. Is there any way to do it.
Let me try to put it programmatically so it gets some clarity. I've a some server running at port say 8080
# main server
from flask import Flask
from myapp import Myapp
app = Flask(__name__)
#app.route("/find_solution", methods=["GET"])
def solution() :
return app.sol.find_solution(), 200
def start():
app.sol = Myapp()
return app
Now this Myapp
#myapp
import requests
class Myapp:
def __init__():
self.session = requests.Session()
def find_solution():
myparameters = {"Some parameter that I filled"}
return self.session.request('GET', 'http://api.weatherstack.com/current', params=myparameters)
Now here I want to replace behavior of http://api.weatherstack.com/current without modifying code. i.e some way where I can replace call to http:api.weatherstack.com/current to my local system server.
Any help of lead is appreciated. I am using ubuntu 20.04
So for your scenario if you want to test your api flask comes with mock test client feature.
test_client = app.test_client()
test_client.post('/find_solution', headers={"Content-Type": "application/json"}, data=data)
So for this scenario you can create test cases and get test client instance inside your test case and perform tests at api level. This is a light weight test method rather than the one proposed by you
Refer to the following link for official flask documentation
https://flask.palletsprojects.com/en/1.1.x/testing/#keeping-the-context-around
Cheers
I'm using a application factory pattern, and when I tried to run my test, I get "Attempted to generate a URL without the application context being". I created a fixture to create the application:
#pytest.fixture
def app():
yield create_app()
but when I run my test
def test_get_activation_link(self, app):
user = User()
user.set_password(self.VALID_PASS)
generated_link = user.get_activation_link()
I get the above error (from the line of code url = url_for("auth.activate")). I'm also trying to figure out to have the app creation run for every test, without having to import it into every test, but I can't seem to find if that's possible.
This works for my app
import pytest
from xxx import create_app
#pytest.fixture
def client():
app = create_app()
app.config['TESTING'] = True
with app.app_context():
with app.test_client() as client:
yield client
def smoke_test_homepage(client):
"""basic tests to make sure test setup works"""
rv = client.get("/")
assert b"Login" in rv.data
So, you missed the application context.
At this year's Flaskcon there was an excellent talk about the Flask context - I highly recommend this video.
https://www.youtube.com/watch?v=fq8y-9UHjyk
I have been trying to follow the tutorials to get flask apps to run on Heroku, like this one: https://dev.to/emcain/how-to-set-up-a-twitter-bot-with-python-and-heroku-1n39.
They all tell you to put this in your code in a file server.py:
from flask import Flask
app = Flask(__name__)
app.run(host='0.0.0.0')
And then run the app via the following command:
python3 server.py
But the tutorials don't explain how to connect the actual function you want to run using the app. In my case, I have a File testbot.py that has the function test(arg1) that contains the code I want to execute:
def test(arg1):
while(1):
#do stuff with arg1 on twitter
I want to do something like this:
from flask import Flask
from testbot import test
from threading import Thread
app = Flask(__name__)
app.addfunction(test(arg1='hardcodedparameter'))
app.run(host='0.0.0.0')
So that when the app runs my test() function executes with the argument. Right now my server is starting, but nothing is happening.
Am I thinking about this correctly?
*Edit: I got it working with the solution, so my server.py now looks like this:
from flask import Flask
from testbot import test
def main_process():
test("hardcodeparam")
app = Flask(__name__)
Thread(target=main_process).start()
app.run(debug=True,host='0.0.0.0')
And now test runs as expected.
Before app.run, register the function with a path, e.g.
#app.route('/')
def test(): # no argument
... do one iteration
return 'ok'
Then visiting the URL will trigger the function. Sites such as https://cron-job.org/ can automate that visiting on a regular basis for free, as suggested here.
If the regular intervals aren't good enough, then you could try:
#app.route('/')
def index(): # no argument
return 'ok'
def test():
while True:
# do stuff
from threading import Thread
Thread(target=test).start()
app.run(...)
You will probably still need to have a job regularly visiting the URL so that Heroku sees that the server is alive and in use.
I am trying to test a Flask web app within a docker container, which is new for me. My stack is the following:
firefox
selenium
pytest-selenium
pytest-flask
Here is my Flask app file:
from flask import Flask
def create_app():
app = Flask(__name__)
return app
app = create_app()
#app.route('/')
def index():
return render_template('index.html')
Now, my test file which verifies the title of my index page:
import pytest
from app import create_app
# from https://github.com/pytest-dev/pytest-selenium/issues/135
#pytest.fixture
def firefox_options(request, firefox_options):
firefox_options.add_argument('--headless')
return firefox_options
# from https://pytest-flask.readthedocs.io/en/latest/tutorial.html#step-2-configure
#pytest.fixture
def app():
app = create_app()
return app
# from https://pytest-flask.readthedocs.io/en/latest/features.html#start-live-server-start-live-server-automatically-default
#pytest.mark.usefixtures('live_server')
class TestLiveServer:
def test_homepage(self, selenium):
selenium.get('http://0.0.0.0:5000')
h1 = selenium.find_element_by_tag_name('h1')
assert h1 == 'title'
When I run my tests with:
pytest --driver Firefox --driver-path /usr/local/bin/firefox test_app.py
I get the following error (which seems due to firefox not in headless mode).
selenium.common.exceptions.WebDriverException: Message: Service /usr/local/bin/firefox unexpectedly exited. Status code was: 1
Error: no DISPLAY environment variable specified
I am able to run firefox --headless but it seems my pytest fixture didn't manage to do the setup. Is there a better way to do this?
Now, if I replace selenium.get() by urlopen just to try the correct initialization of the app and its connection:
def test_homepage(self):
res = urlopen('http://0.0.0.0:5000')
assert b'OK' in res.read()
assert res.code == 200
I get the error:
urllib.error.URLError:
Do I need to boot the live server differently? Or should I change my host + port config somewhere?
Regarding the problem with a direct call with urllib:
Pytest's live server uses random port by default. You can add this parameter to pytest invocation:
--live-server-port 5000
Or without this parameter you can make direct calls to live server like:
import pytest
import requests
from flask import url_for
#pytest.mark.usefixtures('live_server')
def test_something():
r = requests.get(url_for('index', _external=True))
assert r.status_code == 200
I suppose you have view function called index. It would add a correct port number automatically.
But this doesn't have to do anything with docker, how do you run it?
Regarding the problem with Selenium itself - I can imagine docker networks related problem. How do you use it? Do you have eg. docker-compose configuration? Can you share it?
The referenced pytest-selenium issue has:
#pytest.fixture
def firefox_options(firefox_options, pytestconfig):
if pytestconfig.getoption('headless'):
firefox_options.add_argument('-headless')
return firefox_options
Note the - (single dash) preceding headless in add_argument()
(Source)
For late comers, it might be worthwhile taking a look at Xvfb and even more helpful can be this tutorial
Then (in Linux shell) you can enter:
Xvfb :99 &
export DISPLAY=:99
pytest --driver Firefox --driver-path /usr/local/bin/firefox test_app.py
This provides a virtual frame buffer (fake screen) for the application and it outputs all the graphical content there.
Note that I did not encountered this problem, just providing a solution that helped me overcome the mentioned error with an another app.