I am writing a unit test to determine if an attribute is properly set during the instantiation of my parser object. Unfortunetly the only way that I can think to do it is to use self.assertTrue(p.soup)
I haven't slung any python in awhile, but that doesn't seem like a very clear way to check that the instance attribute was properly set. Any ideas on how to improve it?
Here is my test class:
class ParserTest(unittest.TestCase):
def setUp(self):
self.uris = ctd.Url().return_urls()
self.uri = self.uris['test']
def test_create_soup(self):
p = ctd.Parser(self.uri)
self.assertTrue(p.soup)
if __name__ == '__main__':
unittest.main()
# suite = unittest.TestLoader().loadTestsFromTestCase(UrlTest)
unittest.TextTestRunner(verbosity=2).run(suite)
Here is my Parser class that I am testing
class Parser():
def __init__(self, uri):
self.uri = uri
self.soup = self.createSoup()
def createSoup(self):
htmlPage = urlopen(self.uri)
htmlText = htmlPage.read()
self.soup = BeautifulSoup(htmlText)
return BeautifulSoup(htmlText)
I got in the bad habot over the past few years of not unit testing, so I am fairly new to the topic. Any good resources to look at for an in depth explaination of unit testing in Python would be appreciated. I look at the standard library unittest documentation, but that really didn't help much...
If p.soup attribute needs to be instance of BeautifulSoup you can explicitly check its type
self.assertIsInstance(p.soup, BeautifulSoup)
Related
Is it possible to have base classes that define tests in tornado that are themselves not run as tests?
Let's say I have the following minimal example as a base class:
from tornado.testing import AsyncTestCase, gen_test
from tornado.httpclient import HTTPRequest
class AbstractTestCase(AsyncTestCase):
def set_parameters(self):
#Set some parameter value here: self.uri = ...
raise NotImplementedError
#gen_test
def test_common_functionality(self):
req = HTTPRequest(self.uri, method = "GET")
response = yield self.client.fetch(req, raise_error=False)
self.assertEqual(200, response.code)
Now, I would like to make several test cases that inherit from this, define their own value for self.uri...and have some specific tests of their own. Like this:
class ConcreteTestCase(AbstractTestCase):
def set_parameters(self):
self.uri = "www.stackoverflow.com"
#gen_test
def test_some_other_thing(self):
self.assertEqual(2, 1+1)
However, when I try to run this, the AbstractTestCase is also run on its own, giving an error (the NotImplementedError). This happens even when I only try to run the inheriting tests.
Is there any way around this issue, or do I have to duplicate the functionality in each test case?
One way to do this is with multiple inheritance. The abstract class doesn't need to extend AsyncTestCase as long as that class is in the inheritance hierarchy at runtime.
class AbstractTestCase(object):
def set_parameters(self):
#Set some parameter value here: self.uri = ...
raise NotImplementedError
#gen_test
def test_common_functionality(self):
req = HTTPRequest(self.uri, method = "GET")
response = yield self.client.fetch(req, raise_error=False)
self.assertEqual(200, response.code)
class ConcreteTestCase(AbstractTestCase, AsyncTestCase):
def set_parameters(self):
self.uri = "www.stackoverflow.com"
#gen_test
def test_some_other_thing(self):
self.assertEqual(2, 1+1)
This is admittedly a little weird and mypy type checking doesn't like it. But it's simple and it works, and I haven't found a mypy-friendly alternative that I like.
CLI
python3 -m tornado.testing ConcreteTestCase.ConcreteTestCase
testmain.py
#!/usr/bin/env python3
import unittest
from tornado.testing import main
def all():
cases = ['ConcreteTestCase.ConcreteTestCase']
return unittest.defaultTestLoader.loadTestsFromNames(cases)
main()
test/runtests.py is a good example.
I am having a bit of trouble with nesting classes in python.
Mind you my code below is a simplified example showing what I want to do, but basically I want to use nested classes to make my code more structured and make sure I don't run into name clashes for certain functions.
See here my example code:
class Chrome:
def __init__(self, url='http://localhost:4723/wd/hub'):
# Capabilities
capabilities = Capabilities.chrome()
# Start session
self.driver = webdriver.Remote(url, capabilities)
def get_url(self, url):
# Go to URL
self.driver.get(url)
class actions:
#staticmethod
def browse_url(url):
# Go to url
Chrome.get_url(url)
if __name__ == '__main__':
browser = Chrome()
browser.actions.browse_url('https://www.google.com')
The goal as you can see in if __name__ == '__main__' is to be able to start a browser instance, and then call functions in a structured way.
However I have no clue on how to correctly achieve the browser.actions.browse_url('https://www.google.com') concept.
How is this done correctly ?
You should call get_url from an instance of Chrome and not the class itself, since it's an instance method and not a static one:
...
#staticmethod
def browse_url(url):
Chrome().get_url(url)
...
if __name__ == '__main__':
Chrome.actions.browse_url('https://www.google.com')
I'm trying to write a way of testing some XML files. The XML files describe input into a scientific analysis program where various parameters can be defined. I want to write unittests for my XML files so I know that the program is configured correctly.
I'm currently doing this as a library with a base test class containing various tests and some mixins for subcomponents. But the subcomponents are repeated a number of times so I want the tests to run once for each mixin e.g.:
class BaseTest(object):
xmlfile = '...'
...
class ComponentMixin(object):
xmlid = None #
var = None #
def test_var(self):
assert self.var == "whatever_the_value_is_in self.xmlfile"
# ... and a number of other tests and variables.
... now for each analysis there can be a number of components defined with different parameters. I'm hoping to do something like this --
class MyFirstComponentMixin(ComponentMixin):
xmlid = 'component1'
var = 'one'
class MySecondComponentMixin(ComponentMixin):
xmlid = 'component2'
var = 'two'
class MyTest(BaseTest, MyFirstComponentMixin, MySecondComponentMixin, unittest.TestCase):
xmlfile = '...'
... but the problem is that test_var will only be called for component2 and not component2. Is there a way around this, or a better solution?
As you were advised in comment: composition is better solution for your problem than inheritance. The idea is to define multiple standalone TestCases (parts) for pieces of the XML file and then compose them into single TestSuite (composite).
Library
It is a base class for part.
class BaseTestCase(unittest.TestCase):
xmlfile = None # will be set by containing test suite
It is an abstract component test case implementation.
class ComponentTestCase(BaseTestCase):
xmlid = None
var = None
def test_var(self):
assert self.var == "whatever_the_value_is_in self.xmlfile"
It is a base for our composite. It defines convenient copying of the xmlfile from composite to its parts.
class BaseTestSuite(unittest.TestSuite):
xmlfile = None
def addTest(self, test):
if isinstance(test, BaseTestCase):
test.xmlfile = self.xmlfile
super(BaseTestSuite, self).addTest(test)
Usage
It is specific part, which tests some specific aspect of the XML:
class MySpecificTestCase(BaseTestCase):
def test_something_specific(self):
self.assertEqual(4, 2 + 2)
These are parts, which test particular components:
class MyFirstComponentTestCase(ComponentTestCase):
xmlid = 'component1'
var = 'one'
class MySecondComponentTestCase(ComponentTestCase):
xmlid = 'component2'
var = 'two'
Here is a composite with XML you want to test.
class MyTest(BaseTestSuite):
xmlfile = '<some_xml></some_xml>'
We define load_tests to return TestSuite with all TestCases included.
def load_tests(loader, standard_tests, pattern):
return MyTest((
loader.loadTestsFromTestCase(MySpecificTestCase),
loader.loadTestsFromTestCase(MyFirstComponentTestCase),
loader.loadTestsFromTestCase(MySecondComponentTestCase)
))
This approach has one limitation: you can't test few XML files from the single Python file. Basically you can, but output won't help you to identify, which XML file is broken.
Your case is a bit tricky. unittest were designed to test code, not data. Maybe validation against XML schema is what you need.
While the title of this question was exactly what I was looking for, the answer doesn't exactly fit my case.
Perhaps because this question is about testing of data, rather than code.
Still, I found this example (code copy-pasted below) of using multiple inheritances to implement multiple mixins.
Before following this pattern, though, I recommend reading up on multiple inheritance in python is hard - by Ned Batchelder, and this Deep Dive into Python Mixins and Multiple Inheritance.
import unittest
from unittest import TestCase
"""
Showcase how to use mixins and multiple inheritance to write tests
"""
class BaseTest(TestCase):
"""
A base class to be inheritated by actuall test classes.
"""
def setUp(self): # 3
print("BaseTest:setUp called")
self.boo = "gladen sum"
#classmethod
def setUpClass(cls): # 1
print("BaseTest::setUpClass called")
cls.browser = 'musaka'
class FullDBMixin(object):
def setUp(self): # 5
super(FullDBMixin, self).setUp()
print("FullDBMixin::setUp called with instance attribute [boo] = %s" % self.boo)
class LoginMixin(object):
#classmethod
def setUpClass(cls): # 2
super(LoginMixin, cls).setUpClass()
print("LoginMixin::setUpClass called")
def setUp(self): # 4
super(LoginMixin, self).setUp()
print("LoginMixin::setUp called")
self.login()
def login(self):
print("LoginMixin::login called with class attribute [browser] %s" % self.browser)
# order of inheritance **matters**
class TestAuthontecation(LoginMixin, FullDBMixin , BaseTest):
def test_user_dashboard(self):
# test stuff without needing to setup the db or login the user
pass
if __name__ == '__main__':
unittest.main()
# georgi#georgi-laptop:~$ python test.py
# BaseTest::setUpClass called
# LoginMixin::setUpClass called
# BaseTest:setUp called
# FullDBMixin::setUp called with instance attribute [boo] = gladen sum
# LoginMixin::setUp called
# LoginMixin::login called with class attribute [browser] musaka
# .
# ----------------------------------------------------------------------
# Ran 1 test in 0.000s
# OK
I' playing with OOP (OOP concept is something totally new for me) in Python 3 and trying to access attribute (list) of one class from another class. Obviously I am doing something wrong but don't understand what.
from urllib import request
from bs4 import BeautifulSoup
class getUrl(object):
def __init__(self):
self.appList = []
self.page = None
def getPage(self, url):
url = request.urlopen(url)
self.page = url.read()
url.close()
def parsePage(self):
soup = BeautifulSoup(self.page)
for link in soup.find_all("a"):
self.appList.append(link.get('href'))
return (self.appList)
class getApp(object):
def __init__(self):
pass
def selectApp(self):
for i in getUrl.appList():
return print(i)
a = getUrl()
a.getPage("http://somepage/page")
a.parsePage()
b = getApp()
b.selectApp()
And I get:
AttributeError: type object 'getUrl' has no attribute 'appList'
Your code seems to confuse classes with functions. Normally a function name is a verb (e.g. getUrl) because it represents an action. A class name is usually a noun, because it represents a class of objects rather than actions. For example, the following is closer to how I would expect to see classes being used:
from urllib import request
from bs4 import BeautifulSoup
class Webpage(object):
def __init__(self, url):
self.app_list = []
url = request.urlopen(url)
self.page = url.read()
def parse(self):
soup = BeautifulSoup(self.page)
for link in soup.find_all("a"):
self.app_list.append(link.get('href'))
return self.app_list
class App(object):
def __init__(self, webpage, number):
self.webpage = webpage
self.link = webpage.app_list[number]
my_webpage = Webpage("http://somepage/page")
my_webpage.parse()
selected_app = App(my_webpage, 1)
print (selected_app.link)
Note that we usually make an instance of a class (e.g. my_webpage) then access methods and properties of the instance rather than of the class itself. I don't know what you intend to do with the links found on the page, so it is not clear if these need their own class (App) or not.
You need to pass in the getUrl() instance; the attributes are not present on the class itself:
class getApp(object):
def __init__(self):
pass
def selectApp(self, geturl_object):
for i in geturl_object.appList:
print(i)
(note the removed return as well; print() returns None and you'd exit the loop early).
and
b = getApp()
b.selectApp(a)
The appList is a variable in an instance of the getUrl class. So you can only access it for each instance (object) of the getUrl class. The problem is here:
class getApp(object):
def __init__(self):
pass
def selectApp(self):
for i in getUrl.appList():
return print(i)
Look at getUrl.appList(). Here you call the class, not an object. You might also want to look at the return print(i) statement.
Use requests instead of urllib, it's more comfortable.
I am using the HTMLParser library of Python 2.7 to process and extract some information from
an HTML content which was fetched from a remote url. I did not quite understand how to know or catch the exact moment when the parser instance finishes parsing the HTML data.
The basic implementation of my parser class looks like this:
class MyParser(HTMLParser.HTMLParser):
def __init__(self, url):
self.url = url
self.users = set()
def start(self):
self.reset()
response = urllib3.PoolManager().request('GET', self.url)
if not str(response.status).startswith('2'):
raise urllib3.HTTPError('HTTP error here..')
self.feed(response.data.decode('utf-8'))
def handle_starttag(self, tag, attrs):
if tag == 'div':
attrs = dict(attrs)
if attrs.get('class') == 'js_userPictureOuterOnRide':
user = attrs.get("data-name")
if user:
self.users.add(user)
def reset(self):
HTMLParser.HTMLParser.reset(self)
self.users.clear()
My question is, how can I detect that parsing process is finished?
Thanks.
HTMLParser is synchronous, that is, once it returns from feed, all data so far has been parsed and all callbacks called.
self.feed(response.data.decode('utf-8'))
print 'ready!'
(if I misunderstood your question, please let me know).