Passing args as part of unittests to test pyspark script - python

I have a python script which currently takes a command line argument 'path to the json file' and carries out some cleaning up on the data.
I am writing some unit tests where I am trying to pass path to the json file as an arg. It currently comes up with an error when no arg is passed but when it is passed i get the error:
AttributeError: 'module' object has no attribute 'data' which is data.json.
I want to have three separate unit tests each having a different json file to be passed as an argument.
My code is as follows:
import unittest
import sys
import argparse
class TestTransform(unittest.TestCase):
def test_transform(self,input_filename):
target = __import__("cleaning.py")
transform = target
transform.ARGS(input_filename)
self.assertTrue('Pass')
if __name__ == '__main__':
unittest.main()

If I understood your problem correctly here is what I normally do in this case. I override the setUpClass method and make all the inputs to this class attributes that can be accessed by the tests:
class TestTransform():
#classmethod
def setUpClass(self, file_name):
self.input_filename = file_name
#Some other initialization code here
def test_transform(self):
target = __import__("cleaning.py")
transform = target
transform.ARGS(self.input_filename)
self.assertTrue('Pass')
If you then want to make different tests with different input values you can create other classes by subclassing the TestTransform class (and of course the unittest.TestCase):
class Test1(TestTransform, unittest.TestCase):
#classmethod
def setUpClass(self):
input_filename = 'MyFileName'
#Here call the setUpClass from the TestTransform class
TestTransform.setUpClass(input_filename)

Related

how to test a Python class that requires command line argument?

I have a python class that requires a command line argument:
class SomeClass:
request = sys.argv[1] + ".json"
def __init__(self_:
self.req = request
i'd run someClass.py on the commandline i.e. python someClass 1234, which would set the json to 1234.json.
I want a second class, testClass.py, to be able to test methods inside of the main class. But first, i just want to make sure its connected by printing variables:
from someClass import SomeClass
i = SomeClass()
print(i.req)
if i run python testClass.py (without any input), i get a missing input error,
error: the following arguments are required: input
so if i run python testClass.py 1234, i get
none
i just want to know how to pull the class in and make sure its provided with an argument so i can test individual components inside of it.
Just overwrite request in every test that needs it:
import unittest
from x import SomeClass
class TestClass(unittest.TestCase):
def setUp(self):
SomeClass.request = ''
In general, don't make classes which set themselves up. Make the class take parameters which are not defaulted.
You can always make higher level code which supplies default values.
Don't make your class depend on sys.argv in the first place.
class SomeClass:
def __init__(self, base):
self.req = base + ".json"
Then
from someClass import SomeClass
i = SomeClass(sys.argv[1]) # or any other value
print(i.req)

Python unittest test passed in class method is called with

I have a method that takes two inputs a emailer class and a dict of data.
def send_email(data, email_client):
**** various data checks and formatting *****
response_code = email_client.create_email(recipient=receipient
sender=sender
etc...)
I am trying to write a unit test that will assert email_client.create_email was called with the correct values based on the input data.
In my test file I have
from emailer.email import send_email
class TestEmails(unittest.TestCase):
def test_send_email(self):
email.send_email(get_transactional_email_data, MagicMock())
I normally test what a method is called with by something similar to:
mock.assert_called_with(recipient=receipient
sender=sender
etc..)
However, since this time I'm testing what a passed in class is being called with (and a MagicMock) I don't know how it should be done.
I don't think you need MagicMock. Just create the mocks upfront
from emailer.email import send_email
class TestEmails(unittest.TestCase):
def test_send_email(self):
myclient = Mock()
mycreate = Mock()
myclient.create_email = mycreate
email.send_email(get_transactional_email_data, myclient)
self.assertTrue(
mycreate.called_with(sender='...')
)

Is it possible to invoke the main method of a test class through pytest?

I have a test class test_enrollment.py that defines a set of tests, and has a main method which uses a generator to generate a new custom test class. In a simpler form, it looks like:
import json
import unittest
import front_end_tests.generator
class EnrollmentTests(unittest.TestCase):
def test_new_student_registration(self):
# test definition here
if __name__ == '__main__':
front_end_tests.generator.generate_and_run_tests(EnrollmentTests)
The way I normally run this test class is by calling python test_enrollment.py. If I wanted to run this same test file with pytest instead, is there a way to call the file in a way where the main method still gets called?
I thought you can`t.
To solve this problem, you just need to move
front_end_tests.generator.generate_and_run_tests(EnrollmentTests)
To
def setUpModule():
front_end_tests.generator.generate_and_run_tests(EnrollmentTests)
test_enrollment.py look like this:
import json
import unittest
import front_end_tests.generator
def setUpModule():
front_end_tests.generator.generate_and_run_tests(EnrollmentTests)
class EnrollmentTests(unittest.TestCase):
def test_new_student_registration(self):
# test definition here
pass
if __name__ == '__main__':
unittest.main()
and just type:
pytest

Python internal entity mocking

I'd like to test a method, whether it calls a specific method of a temporary internal object or not. (ConfigParser.read)
So the object is created inside, and it's not accessible from the outside after the method exits.
Using python 2.7
In foobar.py
import ConfigParser
class FooBar:
def method(self, filename):
config=ConfigParser.ConfigParser()
config.read(filename)
do_some_stuff()
I'd like to test whether config.read was called.
As I understand, the patch decorator was made for this, but unfortunately the MagicMock object the testcase receives is not the same that is created inside, and I can't get near the object that lives inside the method.
I tried like this:
class TestFooBar(TestCase):
def setUp(self):
self.myfoobar = FooBar()
#mock.patch('foobar.ConfigParser')
def test_read(self,mock_foobar):
self.myfoobar.method("configuration.ini")
assert mock_foobar.called # THIS IS OKAY
assert mock_foobar.read.called # THIS FAILS
mock_foobar.read.assert_called_with("configuration.ini") # FAILS TOO
The problem is:
- mock_foobar is created before the self.myfoobar.method creates the ConfigReader inside.
- when debugging mock_foobar has internal data about the previous calls, but no "read" property (the inner MagicMock for mocking the read method)
Of course one way out is refactoring and giving the .read() or the init() a ConfigReader object, but it's not always possible to change the code, and I'd like to grasp the internal objects of the method without touching the module under test.
You're so close! The issue is that you are mocking the class, but then your test checks that read() is called on that mock class - but you actually expect read() to be called on the instance that is returned when you call the class. The following works - I find the second test more readable than the first, but they both work:
import ConfigParser
from unittest import TestCase
from mock import create_autospec, patch, Mock
class FooBar(object):
def method(self, filename):
config=ConfigParser.ConfigParser()
config.read(filename)
class TestFooBar(TestCase):
def setUp(self):
self.myfoobar = FooBar()
#patch('ConfigParser.ConfigParser')
def test_method(self, config_parser_class_mock):
config_parser_mock = config_parser_class_mock.return_value
self.myfoobar.method("configuration.ini")
config_parser_class_mock.assert_called_once_with()
config_parser_mock.read.assert_called_once_with("configuration.ini")
def test_method_better(self):
config_parser_mock = create_autospec(ConfigParser.ConfigParser, instance=True)
config_parser_class_mock = Mock(return_value=config_parser_mock)
with patch('ConfigParser.ConfigParser', config_parser_class_mock):
self.myfoobar.method("configuration.ini")
config_parser_class_mock.assert_called_once_with()
config_parser_mock.read.assert_called_once_with("configuration.ini")

python unittest multiple mixins

I'm trying to write a way of testing some XML files. The XML files describe input into a scientific analysis program where various parameters can be defined. I want to write unittests for my XML files so I know that the program is configured correctly.
I'm currently doing this as a library with a base test class containing various tests and some mixins for subcomponents. But the subcomponents are repeated a number of times so I want the tests to run once for each mixin e.g.:
class BaseTest(object):
xmlfile = '...'
...
class ComponentMixin(object):
xmlid = None #
var = None #
def test_var(self):
assert self.var == "whatever_the_value_is_in self.xmlfile"
# ... and a number of other tests and variables.
... now for each analysis there can be a number of components defined with different parameters. I'm hoping to do something like this --
class MyFirstComponentMixin(ComponentMixin):
xmlid = 'component1'
var = 'one'
class MySecondComponentMixin(ComponentMixin):
xmlid = 'component2'
var = 'two'
class MyTest(BaseTest, MyFirstComponentMixin, MySecondComponentMixin, unittest.TestCase):
xmlfile = '...'
... but the problem is that test_var will only be called for component2 and not component2. Is there a way around this, or a better solution?
As you were advised in comment: composition is better solution for your problem than inheritance. The idea is to define multiple standalone TestCases (parts) for pieces of the XML file and then compose them into single TestSuite (composite).
Library
It is a base class for part.
class BaseTestCase(unittest.TestCase):
xmlfile = None # will be set by containing test suite
It is an abstract component test case implementation.
class ComponentTestCase(BaseTestCase):
xmlid = None
var = None
def test_var(self):
assert self.var == "whatever_the_value_is_in self.xmlfile"
It is a base for our composite. It defines convenient copying of the xmlfile from composite to its parts.
class BaseTestSuite(unittest.TestSuite):
xmlfile = None
def addTest(self, test):
if isinstance(test, BaseTestCase):
test.xmlfile = self.xmlfile
super(BaseTestSuite, self).addTest(test)
Usage
It is specific part, which tests some specific aspect of the XML:
class MySpecificTestCase(BaseTestCase):
def test_something_specific(self):
self.assertEqual(4, 2 + 2)
These are parts, which test particular components:
class MyFirstComponentTestCase(ComponentTestCase):
xmlid = 'component1'
var = 'one'
class MySecondComponentTestCase(ComponentTestCase):
xmlid = 'component2'
var = 'two'
Here is a composite with XML you want to test.
class MyTest(BaseTestSuite):
xmlfile = '<some_xml></some_xml>'
We define load_tests to return TestSuite with all TestCases included.
def load_tests(loader, standard_tests, pattern):
return MyTest((
loader.loadTestsFromTestCase(MySpecificTestCase),
loader.loadTestsFromTestCase(MyFirstComponentTestCase),
loader.loadTestsFromTestCase(MySecondComponentTestCase)
))
This approach has one limitation: you can't test few XML files from the single Python file. Basically you can, but output won't help you to identify, which XML file is broken.
Your case is a bit tricky. unittest were designed to test code, not data. Maybe validation against XML schema is what you need.
While the title of this question was exactly what I was looking for, the answer doesn't exactly fit my case.
Perhaps because this question is about testing of data, rather than code.
Still, I found this example (code copy-pasted below) of using multiple inheritances to implement multiple mixins.
Before following this pattern, though, I recommend reading up on multiple inheritance in python is hard - by Ned Batchelder, and this Deep Dive into Python Mixins and Multiple Inheritance.
import unittest
from unittest import TestCase
"""
Showcase how to use mixins and multiple inheritance to write tests
"""
class BaseTest(TestCase):
"""
A base class to be inheritated by actuall test classes.
"""
def setUp(self): # 3
print("BaseTest:setUp called")
self.boo = "gladen sum"
#classmethod
def setUpClass(cls): # 1
print("BaseTest::setUpClass called")
cls.browser = 'musaka'
class FullDBMixin(object):
def setUp(self): # 5
super(FullDBMixin, self).setUp()
print("FullDBMixin::setUp called with instance attribute [boo] = %s" % self.boo)
class LoginMixin(object):
#classmethod
def setUpClass(cls): # 2
super(LoginMixin, cls).setUpClass()
print("LoginMixin::setUpClass called")
def setUp(self): # 4
super(LoginMixin, self).setUp()
print("LoginMixin::setUp called")
self.login()
def login(self):
print("LoginMixin::login called with class attribute [browser] %s" % self.browser)
# order of inheritance **matters**
class TestAuthontecation(LoginMixin, FullDBMixin , BaseTest):
def test_user_dashboard(self):
# test stuff without needing to setup the db or login the user
pass
if __name__ == '__main__':
unittest.main()
# georgi#georgi-laptop:~$ python test.py
# BaseTest::setUpClass called
# LoginMixin::setUpClass called
# BaseTest:setUp called
# FullDBMixin::setUp called with instance attribute [boo] = gladen sum
# LoginMixin::setUp called
# LoginMixin::login called with class attribute [browser] musaka
# .
# ----------------------------------------------------------------------
# Ran 1 test in 0.000s
# OK

Categories