I'd like to unittest web script on HTTPServer.
But mock.patch isn't working via HTTPServer.
it seems kicking subprocess inside.
For example, my web scripts has some external web access.
web script:
#!/usr/bin/python3
import requests
class Script:
def main(self):
res = requests.put('http://www.google.co.jp') # get response code 405
print('Content-type: text/html; charset=UTF-8\n')
print(res.content)
if __name__ == '__main__':
Script().main()
And my test script seems it can't mock the external web access.
test script:
import unittest
import requests
from http.server import HTTPServer, CGIHTTPRequestHandler
from threading import Thread
from unittest import TestCase, mock
class MockTreadTest(TestCase):
def test_default(self):
server = HTTPServer(('', 80), CGIHTTPRequestHandler)
server_thread = Thread(target=server.serve_forever)
server_thread.start()
try:
with mock.patch('requests.put', return_value='<html>mocked response</html>') as put:
res = requests.get('http://localhost/cgi-bin/script.py')
self.assertRegex(str(res.content), 'mocked response') # fail
self.assertRegex(put.call_args_list[0][0][0], 'http://www.google.co.jp')
finally:
server.shutdown()
server_thread.join()
if __name__ == "__main__":
unittest.main()
The MockTreadTest is presently not testing the webscript. It is right now starting a WSGI server and it looks like the try block is calling a non-existent script. I recommend reading more about testing and mocking. I think you are trying to test the main() function in the Script class. Here is a test function to help you with:
from unittest.mock import TestCase, patch
# replace `your_script` with the name of your script
from your_script import Script
# import the requests object from your script. IMPORTANT, do not do import request
from your_script import requests
class ScriptTestCase(TestCase):
def test_main(self):
script = Script()
# Now patch the requests.put function call going to the server
with patch('requests.put', return_value="<html>mocked response</html>") as mockput:
script.main() # now call the main function <-- this is what we are testing
mockput.assert_called_with('http://www.google.co.jp')
Presently you are just printing in the response in you script. So there is no way to test the return value. Use return statement in the main() function to achieve it. Then you can do as given below in the with block.
response = script.main()
self.assertIn('mocked response', response.content)
Related
Here's a tiny example. When running under functions framework on a GCE VM with an appropriate service account,
import logging
import google.cloud.logging
logging_client = google.cloud.logging.Client()
logging_client.setup_logging()
def test(request):
logging.warning("testing warning")
return "done"
does not log a warning message.
Changing it to:
import logging
import google.cloud.logging
logging_client = google.cloud.logging.Client()
def test(request):
logging_client.setup_logging()
logging.warning("testing warning")
return "done"
does produce the correct message.
And if I take the version that doesn't work, add a call to the function and run it normally without functions framework, it produces a warning as expected.
import logging
import google.cloud.logging
logging_client = google.cloud.logging.Client()
logging_client.setup_logging()
def test(request):
logging.warning("testing warning")
return "done"
test("foo")
Can someone explain what's going on, either with how the functions frameworks works, or how the logging handler works?
Thanks!
I had a similar issue using gcsfs with functions framework that provided the answer. It looks like when you use functions framework the function call runs in a different process from the initialization.
I mimicked this with:
import logging
import google.cloud.logging
import os
import time
logging_client = google.cloud.logging.Client()
logging_client.setup_logging()
def test(request):
logging.warning("testing warning")
return "done"
pid = os.fork()
if pid > 0:
time.sleep(10)
else:
test("foo")
and it does not work. Move the setup_logging to inside of the function call, and it does. So it seems that you have to do the setup_logging() in the subprocess for it to work, and that's why it has to be done in the function call rather than as part of initialization in function framework.
I am trying to create unittests for my flask application which should assert exceptions properly.
I am attaching simplified code sample on what i want to test. The below test should finish as success.
def my_function():
abort(400,"error")
import unittest
from werkzeug import exceptions
class Tests(unittest.TestCase):
def test_event_link(self):
self.assertRaises(exceptions.BadRequest,my_function)
unittest.main(argv=[''], verbosity=2, exit=False)
I would simply patch the Flask abort function and ensure it is called with the correct value, this is preferable as it only tests your code not the behaviour of Flasks abort function which could change with future versions of Flask and break your tests.
See below example based on your code which also includes examples of testing the exception if this is what you would prefer to do.
# code.py
from flask import abort
def my_function():
abort(400, "error")
# test.py
import unittest
from unittest.mock import patch
from werkzeug import exceptions
import code # Your code file code.py
class Tests(unittest.TestCase):
#patch('code.abort')
def test_one(self, mock_abort):
code.my_function()
mock_abort.assert_called_once_with(400, 'error')
def test_two(self):
with self.assertRaises(exceptions.BadRequest):
code.my_function()
def test_three(self):
with self.assertRaisesRegexp(exceptions.BadRequest, '400 Bad Request: error'):
code.my_function()
unittest.main(argv=[''], verbosity=2, exit=False)
I'm writing unit tests for a piece of code that uses zeep to access a SOAP API so I want to mock out zeep. In my actual code, it looks something like this:
from zeep import Client
def do_something():
client = Client("...")
In my test, I'm doing this:
from unittest import mock
#mock.patch('zeep.Client')
def test_do_somethi(self, MockedClient):
do_something()
The Client that the actual function is obtaining, is the actual zeep client and not my mock. I also tried:
#mock.patch('zeep.client.Client')
and the result was the same.
I also tried:
def test_do_something(self):
with mock.patch('zeep.client.Client') as MockedClient:
do_something()
with no difference.
Any ideas why this isn't working?
When mock doesn't work, the first thing to look for is if you are patching the right name. Three common import scenarios:
(a) If you want to mock out zeep but you import like
from zeep import Client
and your tests are in the same file, you patch Client not zeep.Client.
(b) If instead you import it like
import zeep
and then use zeep.Client in SUT code, then you patch zeep.Client.
(c) If you are testing code that lies in some other module (like mymodule) and you import zeep there with
from zeep import Client # (1)
then in your test module you
import mymodule
then you patch mymodule.Client, ... or mymodule.zeep.Client if you used the alternative import zeep form in (1).
You must patch the method/class from file you are using it. If you want to patch Client and it is imported in some_file.py you must import it from it, and not from lib (zeep.Client)
Here is an example using the official doc from Zeep.
some_lib.py
from zeep import Client
def do_something():
wsdl = 'http://www.soapclient.com/xml/soapresponder.wsdl'
client = zeep.Client(wsdl=wsdl)
return client.service.Method1('Zeep', 'is cool')
test_connection.py
from some_lib import do_something
from unittest.mock import patch
#patch('some_lib.Client')
def test_do_something(mock_zeep):
res = do_something()
assert mock_zeep.call_count == 1
if __name__ == '__main__':
test_soap_conn()
I’m relatively new to python and am looking for a pythonic way to handle this practice.
I’ve inherited a fairly trivial Python 2.7 Flask app that runs under uwsgi that I want to add some unit tests to. It does some initialization at indentation level 0 that is required when it’s running in uwsgi but needs to be skipped when under test.
I’m given to understand that often python apps use the
if __name__ == '__main__':
pattern to isolate code that should run when the script is run on its own and should not run when it’s imported. In this case, however, both when the script is run under uwsgi and when the script is imported into the unit tests, __name__ is the same; the name of the script, so I can’t use that to differentiate between uwsgi and unit-testing environments.
This sample code illustrates what I'm working with.
In the Flask application (flask_app.py):
import logging
import bcrypt
from flask import Flask, jsonify, abort, make_response, request
from ConfigParser import SafeConfigParser
# some initialization that only makes sense when running from the uwsgi context on the server
PARSER = SafeConfigParser()
PARSER.read('config.ini')
LOG_FILE = PARSER.get('General', 'logfile')
APP = Flask(__name__)
#APP.route('/', methods=['GET'])
def index
...
#APP.route('/<p1>/<p2>', methods=['PUT'])
def put(p1, p2):
...
if __name__ == '__main__':
APP.run(debug = True, host='0.0.0.0')
In the unit tests (tests.py):
import os
import unittest
from flask import json
from flask_app import APP
class FlaskAppTestCase(unittest.TestCase):
def setUp(self):
self.APP = APP.test_client()
def test_GET(self):
resp = self.APP.get('/')
assert 'Some Html' in resp.data
def test_PUT(self):
resp = self.APP.put('/1/2')
assert 'Got 1, 2' in resp.data
if __name__ == '__main__':
unittest.main()
What I was thinking of doing was to move the initialization so that it only runs when flask_app is being executed by uwsgi and not when it's running via tests.py, perhaps by checking name and determining which path to execute based on that, but when I examine the output of print(name) either when running flask_app under uwsgi or by executing tests.py the output is "flask_app", so I can't seem to use that as a discriminator.
Is there an idiomatic way in python to handle this?
As it turns out the Python module for uWSGI actually offers a mechanism to determine if the app is being run under uWSGI. The uwsgi module is available for import if you are in a uWSGI context, so I ended up checking whether i could import that module and only executing the initialization code if I could.
# detect if we're running under uWSGI; only init if we are, not if we're testing
try:
import uwsgi
IN_UWSGI = True
except ImportError:
IN_UWSGI = False
Then wrap the init code with
if IN_UWSGI:
This seems much more reliable then checking the module name of the module that's doing the import, which was the only other thing I could think of to do.
I am trying to load a module according to some settings. I have found a working solution but I need a confirmation from an advanced python developer that this solution is the best performance wise as the API endpoint which will use it will be under heavy load.
The idea is to change the working of an endpoint based on parameters from the user and other systems configuration. I am loading the correct handler class based on these settings. The goal is to be able to easily create new handlers without having to modify the code calling the handlers.
This is a working example :
./run.py :
from flask import Flask, abort
import importlib
import handlers
app = Flask(__name__)
#app.route('/')
def api_endpoint():
try:
endpoint = "simple" # Custom logic to choose the right handler
handlerClass = getattr(importlib.import_module('.'+str(endpoint), 'handlers'), 'Handler')
handler = handlerClass()
except Exception as e:
print(e)
abort(404)
print(handlerClass, handler, handler.value, handler.name())
# Handler processing. Not yet implemented
return "Hello World"
if __name__ == "__main__":
app.run(host='0.0.0.0', port=8080, debug=True)
One "simple" handler example. A handler is a module which needs to define an Handler class :
./handlers/simple.py :
import os
class Handler:
def __init__(self):
self.value = os.urandom(5)
def name(self):
return "simple"
If I understand correctly, the import is done on each query to the endpoint. It means IO in the filesystem with lookup for the modules, ...
Is it the correct/"pythonic" way to implement this strategy ?
Question moved to codereview. Thanks all for your help : https://codereview.stackexchange.com/questions/96533/extension-pattern-in-a-flask-controller-using-importlib
I am closing this thread.