I'd like to setup a nice way to capture screenshots when some of our Robot Framework front-end test fails. Of course I can add:
Run Keyword If Test Failed Capture Page Screenshot
To test Teardown, but considering I have huge and complex test suites with hundreds of tests and a nested structure - I would need to add this to so many Teardowns that it seems ugly to me.
I've experimented a bit. I thought the way forward was to use listener. So I tried this:
class ExtendedSelenium(Selenium2Library):
ROBOT_LISTENER_API_VERSION = 3
def __init__(self, timeout=5.0, implicit_wait=0.0, run_on_failure='Capture Page Screenshot', screenshot_root_directory=None):
super(ExtendedSelenium, self).__init__(timeout, implicit_wait, run_on_failure, screenshot_root_directory)
self.ROBOT_LIBRARY_LISTENER = self
def _end_test(self, data, result):
if not result.passed:
screenshot_name = data.id + data.name.replace(" ", "_") + ".png"
try:
self.capture_page_screenshot(screenshot_name)
except:
pass
It captures the screenshot but the picture is then not visible in the log. I'm able to display it in the test message, adding this step after capturing:
BuiltIn().set_test_message("*HTML* {} <br/> <img src={}>".format(result.message, screenshot_name))
But still, it isn't the best.
Then I tried a different approach with the Visitor interface (used with --prerunmodifier):
from robot.api import SuiteVisitor
class CaptureScreenshot(SuiteVisitor):
def end_test(self, test):
if test.status == 'FAIL':
test.keywords.create('Capture Page Screenshot', type='teardown')
But it replaces any existing Test Teardown by the new one (with only one keyword 'Capture Page Screenshot'). I thought I would be able to modify existing Teardowns by adding the Capture keyword, but I wasn't.
Is there any nice, clean, pythonic way to do this? Did I miss something?
Finally it ended up with library listener as below. I have to agree with Bryan: the code isn't short and nice but it fulfils the desired goal - single point in suite where screenshot capturing is defined.
As a big advantage I see the possibility to capture screenshot for failed setup - in some cases it helps us to identify possible infrastructure problems.
Please note the part with ActionChains - it zooms out in browser. Our front-end app uses partial page scroll and with this zoom we are able to see more content inside of that scroll which is really helpful for us. Result of this ActionChains differs for each browser, so this is truly workaround.
#!/usr/bin/python
# -*- coding: utf-8 -*-
from Selenium2Library import Selenium2Library
from selenium.common.exceptions import StaleElementReferenceException, WebDriverException
import re
class ExtendedSelenium(Selenium2Library):
""" Robot Framework library extending Robot Framework Selenium2Library library.
"""
ROBOT_LISTENER_API_VERSION = 2
DON_NOT_CAPTURE_KEYWORDS = ["Run Keyword And Ignore Error", "Run Keyword And Expect Error", "Run Keyword And Return Status", "Wait Until.*"]
def __init__(self, timeout=5.0, implicit_wait=0.0, run_on_failure='', screenshot_root_directory=None):
super(ExtendedSelenium, self).__init__(timeout, implicit_wait, run_on_failure, screenshot_root_directory)
self.ROBOT_LIBRARY_LISTENER = self
self._is_current_keyword_inside_teardown = False
self._do_not_capture_parent_keywords_count = 0
self._screenshot_was_captured = False
def _start_test(self, name, attributes):
""" Reset flags at the begin of each test.
"""
self._do_not_capture_parent_keywords_count = 0
self._is_current_keyword_inside_teardown = False
self._screenshot_was_captured = False
def _start_keyword(self, name, attributes):
""" Set keyword flag at the beginning of teardown.
If the keyword is one of the 'do not capture keywords' increase _do_not_capture_parent_keywords_count counter.
"""
if attributes["type"] == "Teardown":
self._is_current_keyword_inside_teardown = True
if any(kw for kw in self.DON_NOT_CAPTURE_KEYWORDS if re.match(kw, attributes["kwname"])):
self._do_not_capture_parent_keywords_count += 1
def _end_keyword(self, name, attributes):
"""If the keyword is one of the 'do not capture keywords' decrease _do_not_capture_parent_keywords_count counter.
Capture Screenshot if:
- keyword failed AND
- test is not in teardown phase AND
- the parent keyword isn't one of the 'do not capture keywords'
RuntimeError exception is thrown when no browser is open (backend test), no screenshot is captured in this case.
"""
if any(kw for kw in self.DON_NOT_CAPTURE_KEYWORDS if re.match(kw, attributes["kwname"])):
self._do_not_capture_parent_keywords_count -= 1
if not attributes["status"] == "PASS" and not self._is_current_keyword_inside_teardown and self._do_not_capture_parent_keywords_count == 0 and not self._screenshot_was_captured:
self._screenshot_was_captured = True
try:
self.capture_page_screenshot()
# TODO refactor this so it is reusable and nice!
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
ActionChains(super(ExtendedSelenium, self)._current_browser()).send_keys(Keys.CONTROL, Keys.SUBTRACT, Keys.NULL).perform()
ActionChains(super(ExtendedSelenium, self)._current_browser()).send_keys(Keys.CONTROL, Keys.SUBTRACT, Keys.NULL).perform()
self.capture_page_screenshot()
ActionChains(super(ExtendedSelenium, self)._current_browser()).send_keys(Keys.CONTROL, '0', Keys.NULL).perform()
except RuntimeError:
pass
Any comments are welcome.
I wrapped everything in a try/except block for my tests. That way at any point if the test fails the screen shot is in the except block and I get a screen shot the moment before/after/...whatever.
try:
class UnitTestRunner():
...
except:
driver.save_screenshot('filepath')
Related
A little bit of context: I am using Python and Selenium for QA Web Automation and I am using the Page Object Model pattern.
I am currently in the process of making the logger part of the system, and I want to make the code more efficient, without
having to write a lot of dublicate code. I checked the docs for subTests, but can't find anything in particular and this is the reason I am here. So I am wondering if there is a way to access this part of the code (so I don't have to
write each logger message each time, which is not very practical):
class TestScenario(unittest.TestCase):
.... # set Up class
def logger(self,info):
logger.error(f"Error happened at {info}")
def test_method(self):
with self.subTest("MESSAGE"):---------------------------------------------
|
try: |
... something |
except AssertionError: |
self.logger(#Is there a way to access subTest msg parameter?) <---
raise
.... other subTests which will follow the same pattern
Thank you in advance!
You can do self.logger(self._subtest._message) to get the message.
Beware, you are accessing internals variable of unittest thus this code can be broken in the next release of python without warning.
Possible approach:
class Message:
def __init__(self, message=''):
self.message = message
def __str__(self):
return self.message
# separate message for every subtest or one message continually modified between subtests
msg = Message('qwe')
...
with self.subTest(msg=msg):
msg.message = 'asd' # access or modify
self.assertTrue(True)
...
Or you may create attribute message in the setUp method of the test and use it to pass message into subTest and access/modify when you need.
Trying to find the most elegant way to inform the test fixture of a test failure. This test fixture needs to report the results of the test to saucelabs in order to mark it as pass or fail. I've tried to delete as much irrelevant code from these examples as possible.
The following test uses the fixture browser.
def test_9(browser):
browser.get(MY_CONSTANT)
assert "My Page" in browser.title
browser.find_element_by_css_selector('div > img.THX_IP')
browser.find_element_by_link_text('Some text').click()
... etc
The fixture browser, which currently is hard coded to mark the test as passed:
#pytest.fixture()
def browser(request):
driver_type = request.config.getoption('driver')
if driver_type == 'sauce':
driver = webdriver.Remote(
command_executor = 'MY_CREDENTIALS',
desired_capabilities = caps)
else:
driver = webdriver.Chrome()
driver.implicitly_wait(2)
yield driver
if driver_type == 'sauce':
sauce_client.jobs.update_job(driver.session_id, passed = True)
driver.quit()
I've discovered a few workarounds but I'd really like to know the best way to do it.
I have the following fixture in my conftest.py file which handles all of the pass/fail reporting to sauce. I am not sure if it's "the best way to do it" but it certainly works for us at the moment. We include this fixture for every test that we write.
There's considerably more to this fixture but this section contained at the bottom handles the report really well.
def quit():
try:
if config.host == "saucelabs":
if request.node.result_call.failed:
driver_.execute_script("sauce:job-result=failed")
elif request.node.result_call.passed:
driver_.execute_script("sauce:job-result=passed")
finally:
driver_.quit()
My English is very poor but I'll try my best to describe the problem I encountered.
I used selenium webdriver to test a web site and the language that I used to write my script is python.Because of this,I used Pyunit.
I know that if my test suite have no exceptions,the webdriver instance will be closed correctly,(by the way, I used chrome) however,once a exception was threw,the script will be shut down and I have to close chrome manually.
I wonder that how can I achieved that when a python process quits , any remaining open WebDriver instances will also be closed.
By the way, I used Page Object Design Pattern,and the code below is a part of my script:
class personalcenter(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Chrome()
self.page = personalCenter(self.driver,"admin","123456")
def testAddWorkExp(self):
blahblahblah...
def tearDown(self):
self.page.quit()
self.driver.quit()
if __name__ == "__main__":
unittest.main()
I haved searched the solution of this problem for a long time ,but almost every answer is depended on java and junit or testNG.How can I deal with this issue with Pyunit?
Thanks for every answer.
From the tearDown() documentation:
Method called immediately after the test method has been called and
the result recorded. This is called even if the test method raised an
exception, so the implementation in subclasses may need to be
particularly careful about checking internal state. Any exception
raised by this method will be considered an error rather than a test
failure. This method will only be called if the setUp() succeeds,
regardless of the outcome of the test method. The default
implementation does nothing.
So, the only case where tearDown will not be called is when setUp fails.
Therefore I would simply catch the exception inside setUp, close the driver, and re-raise it:
def setUp(self):
self.driver = webdriver.Chrome()
try:
self.page = personalCenter(self.driver,"admin","123456")
except Exception:
self.driver.quit()
raise
Hope!! this will help you. you might need to close the driver with boolean expression as False
from selenium import webdriver
import time
class Firefoxweb(object):
def __init__(self):
print("this is to run the test")
def seltest(self):
driver=webdriver.Chrome(executable_path="C:\\Users\Admin\PycharmProjects\Drivers\chromedriver.exe")
driver.get("URL")
driver.find_element_by_xpath("//*[#id='btnSkip']").click()
driver.find_element_by_xpath("//*[#id='userInput']").send_keys("ABC")
driver.find_element_by_xpath("//*[#id='passwordInput']").send_keys("*******")
driver.find_element_by_xpath("//*[#id='BtnLogin']").click()
driver.close(False)
FF=Firefoxweb()
FF.seltest()
My apologies if this is not clear at any point (or if the vernacular isn't quite right) but I will do my best!
First time actually posting in a forum ;) ...
Any help from those in the know would be greatly appreciated.
I am using Selenium with Python bindings to conduct automated testing of a web based application across a number of platforms (OS) and browsers.
As part of this I am using the HTMLTestRunner module to generate a report at the end of each test run. It is here in that my problem lies.
The result of my code as it currently reads is that for each platform/ browser combination within respective lists, the HTMLTestRunner module is initialised and conducts a single test case... in turn generating the report and closing.
This creates problems with the generated report overwriting it self (or leads to formatting problems) as HTMLTestRunner is designed to be initialised, then conduct all test cases, then create a single report using all the test results.
Incidently if I use 'unittest.TextTestRunner()' instead of the HTMLTestRunner then essentially the same thing is happening only the reults are obviously displayed in the shell. ran 1 test... OK... ran 1 test... OK... et
I have tried using the line 'unittest.main(exit=False) which actually appears to work for results displyed within the shell eg all tests are run before any report is provided.
Unfortunatly I have not found a way of using this functionality with HTMLTestRunner.
(I suspect that someone is going to come back with using a meta class to dynamically generate tests instead of looping the execution code line like this. While I have looked into this I found myself at a complete loss as to how to implement this and a number of developer types have told me that this is something to steer clear of us (don't want to start a debate here!).)
Anyway a simplified example of the code that I am using is as per below:
global platform_list
platform_list = ['XP','VISTA','LINUX','WIN8','MAC']
global browser_list
browser_list = ['internet explorer','firefox','chrome','safari','opera']
class configuration(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Remote(command_executor= executor, desired_capabilities={'platform': platform, 'browserName': browser})
self.driver.implicitly_wait(30)
self.base_url = environment
self.verificationErrors = []
self.accept_next_alert = True
def is_element_present(self, how, what):
try: self.driver.find_element(by=how, value=what)
except NoSuchElementException, e: return False
return True
def is_alert_present(self):
try: self.driver.switch_to_alert()
except NoAlertPresentException, e: return False
return True
def close_alert_and_get_its_text(self):
try:
alert = self.driver.switch_to_alert()
alert_text = alert.text
if self.accept_next_alert:
alert.accept()
else:
alert.dismiss()
return alert_text
finally: self.accept_next_alert = True
def tearDown(self):
self.driver.quit()
self.assertEqual([], self.verificationErrors)
class test_case_A(configuration):
def test_case_A(self):
try:
some code
except:
some code
class test_case_B(test_case_A):
def test_case_B(self):
self.test_case_A()
try:
some code
except:
some code
unit_test_suite = unittest.TestSuite()
unit_test_suite.addTest(test_case_A('test_case_A')), (test_case_B('test_case_B'))
results = file('results.html', 'wb')
runner = HTMLTestRunner.HTMLTestRunner(stream = results, title = 'Whatever', description = 'Whatever')
global platform
global browser
for platform in platform_list:
for browser in browser_list:
unittest.TextTestRunner().run(unit_test_suite)
I just want to state that while certain elements of the above code may be unnessecary such as global declarations they are required in the actual!
Any help would be greatly appreciated! Many thanks...
I'm not sure if this answer will assist you as it may require a substantial restructure of your code, as far as I can understand.
For running my regression suite I have a particular module that builds the regression suite and then runs the HTMLtestrunner.
So each of my test-modules with multiple test-cases is imported, added to the TestSuite and then the suite itself is launched.
Here is a chopped down example:
from unittest import TestLoader, TestSuite
import HTMLTestRunner
import datetime
from root.nested.tests.test_class_1 import TestClass1
from root.nested.tests.test_class_2 import TestClass2
class RegressionSuite():
if __name__ == "__main__":
file_name = datetime.datetime.now().strftime("%Y_%m_%d_%H%M_report.html")
output = open(file_name, "wb")
loader = TestLoader()
suite = TestSuite((
loader.loadTestsFromTestCase(TestClass1),
loader.loadTestsFromTestCase(TestClass2)
))
runner = HTMLTestRunner(stream = output, verbosity = 1, title="Regression Suite")
runner.run(suite)
When this module is executed, the resulting HTML results file lists each test-class with a summary and the detail can be shown to display the result for each test-case as well.
I'm attempting to call test_check_footer_visible in another .py file.
origin.py has:
class FooterSection(unittest.TestCase):
"""
"""
browser = None
site_url = None
def setUp(self):
self.browser.open(self.site_url)
self.browser.window_maximize()
# make footer section
self._footer = FooterPage(self.browser)
def test_check_footer_visible(self):
#press the page down
self.browser.key_down_native("34")
self.browser.key_down_native("34")
self.browser.key_down_native("34")
print "Page down"
time.sleep (10)
Now in dest.py I need to call test_check_footer_visible(). How do I do that?
dest.py has the following format:
class TestPageErrorSection(unittest.TestCase):
"""
"""
browser = None
site_url = None
def setUp(self):
global SITE_URL, BROWSER
if not BROWSER:
BROWSER = self.browser
if not SITE_URL:
SITE_URL = self.site_url
BROWSER.open(SITE_URL)
BROWSER.window_maximize()
print 'page not found 404 error'
self._pageerror_section = ErrorSection(BROWSER)
def _primary_navigation(self):
# I want to call test_check_footer_visible here.
I tried everything in Call Class Method From Another Class
You can't (without doing some really shady things -- see comments by #glglgl) -- at least not without changing your code somewhat. The reason you can't is because test_check_footer_visible will assert that the first element passed in is an instance of the class FooterSection (which it isn't -- it's an instance of TestPageErrorSection). You have (at least) a couple of options for the re-factor.
1) Make TestPageErrorSection inherit from FooterSection and use that function directly (e.g. self.test_check_footer_visible() This doesn't seem like a very good option to me based on the class names
2) Make test_check_footer_visible() a regular function. It can then be called from either class.
3) Create a 3rd class (e.g. SectionBase), put test_check_footer_visible on that class and have your other 2 classes inherit from it.