My apologies if this is not clear at any point (or if the vernacular isn't quite right) but I will do my best!
First time actually posting in a forum ;) ...
Any help from those in the know would be greatly appreciated.
I am using Selenium with Python bindings to conduct automated testing of a web based application across a number of platforms (OS) and browsers.
As part of this I am using the HTMLTestRunner module to generate a report at the end of each test run. It is here in that my problem lies.
The result of my code as it currently reads is that for each platform/ browser combination within respective lists, the HTMLTestRunner module is initialised and conducts a single test case... in turn generating the report and closing.
This creates problems with the generated report overwriting it self (or leads to formatting problems) as HTMLTestRunner is designed to be initialised, then conduct all test cases, then create a single report using all the test results.
Incidently if I use 'unittest.TextTestRunner()' instead of the HTMLTestRunner then essentially the same thing is happening only the reults are obviously displayed in the shell. ran 1 test... OK... ran 1 test... OK... et
I have tried using the line 'unittest.main(exit=False) which actually appears to work for results displyed within the shell eg all tests are run before any report is provided.
Unfortunatly I have not found a way of using this functionality with HTMLTestRunner.
(I suspect that someone is going to come back with using a meta class to dynamically generate tests instead of looping the execution code line like this. While I have looked into this I found myself at a complete loss as to how to implement this and a number of developer types have told me that this is something to steer clear of us (don't want to start a debate here!).)
Anyway a simplified example of the code that I am using is as per below:
global platform_list
platform_list = ['XP','VISTA','LINUX','WIN8','MAC']
global browser_list
browser_list = ['internet explorer','firefox','chrome','safari','opera']
class configuration(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Remote(command_executor= executor, desired_capabilities={'platform': platform, 'browserName': browser})
self.driver.implicitly_wait(30)
self.base_url = environment
self.verificationErrors = []
self.accept_next_alert = True
def is_element_present(self, how, what):
try: self.driver.find_element(by=how, value=what)
except NoSuchElementException, e: return False
return True
def is_alert_present(self):
try: self.driver.switch_to_alert()
except NoAlertPresentException, e: return False
return True
def close_alert_and_get_its_text(self):
try:
alert = self.driver.switch_to_alert()
alert_text = alert.text
if self.accept_next_alert:
alert.accept()
else:
alert.dismiss()
return alert_text
finally: self.accept_next_alert = True
def tearDown(self):
self.driver.quit()
self.assertEqual([], self.verificationErrors)
class test_case_A(configuration):
def test_case_A(self):
try:
some code
except:
some code
class test_case_B(test_case_A):
def test_case_B(self):
self.test_case_A()
try:
some code
except:
some code
unit_test_suite = unittest.TestSuite()
unit_test_suite.addTest(test_case_A('test_case_A')), (test_case_B('test_case_B'))
results = file('results.html', 'wb')
runner = HTMLTestRunner.HTMLTestRunner(stream = results, title = 'Whatever', description = 'Whatever')
global platform
global browser
for platform in platform_list:
for browser in browser_list:
unittest.TextTestRunner().run(unit_test_suite)
I just want to state that while certain elements of the above code may be unnessecary such as global declarations they are required in the actual!
Any help would be greatly appreciated! Many thanks...
I'm not sure if this answer will assist you as it may require a substantial restructure of your code, as far as I can understand.
For running my regression suite I have a particular module that builds the regression suite and then runs the HTMLtestrunner.
So each of my test-modules with multiple test-cases is imported, added to the TestSuite and then the suite itself is launched.
Here is a chopped down example:
from unittest import TestLoader, TestSuite
import HTMLTestRunner
import datetime
from root.nested.tests.test_class_1 import TestClass1
from root.nested.tests.test_class_2 import TestClass2
class RegressionSuite():
if __name__ == "__main__":
file_name = datetime.datetime.now().strftime("%Y_%m_%d_%H%M_report.html")
output = open(file_name, "wb")
loader = TestLoader()
suite = TestSuite((
loader.loadTestsFromTestCase(TestClass1),
loader.loadTestsFromTestCase(TestClass2)
))
runner = HTMLTestRunner(stream = output, verbosity = 1, title="Regression Suite")
runner.run(suite)
When this module is executed, the resulting HTML results file lists each test-class with a summary and the detail can be shown to display the result for each test-case as well.
Related
Trying to find the most elegant way to inform the test fixture of a test failure. This test fixture needs to report the results of the test to saucelabs in order to mark it as pass or fail. I've tried to delete as much irrelevant code from these examples as possible.
The following test uses the fixture browser.
def test_9(browser):
browser.get(MY_CONSTANT)
assert "My Page" in browser.title
browser.find_element_by_css_selector('div > img.THX_IP')
browser.find_element_by_link_text('Some text').click()
... etc
The fixture browser, which currently is hard coded to mark the test as passed:
#pytest.fixture()
def browser(request):
driver_type = request.config.getoption('driver')
if driver_type == 'sauce':
driver = webdriver.Remote(
command_executor = 'MY_CREDENTIALS',
desired_capabilities = caps)
else:
driver = webdriver.Chrome()
driver.implicitly_wait(2)
yield driver
if driver_type == 'sauce':
sauce_client.jobs.update_job(driver.session_id, passed = True)
driver.quit()
I've discovered a few workarounds but I'd really like to know the best way to do it.
I have the following fixture in my conftest.py file which handles all of the pass/fail reporting to sauce. I am not sure if it's "the best way to do it" but it certainly works for us at the moment. We include this fixture for every test that we write.
There's considerably more to this fixture but this section contained at the bottom handles the report really well.
def quit():
try:
if config.host == "saucelabs":
if request.node.result_call.failed:
driver_.execute_script("sauce:job-result=failed")
elif request.node.result_call.passed:
driver_.execute_script("sauce:job-result=passed")
finally:
driver_.quit()
I'd like to setup a nice way to capture screenshots when some of our Robot Framework front-end test fails. Of course I can add:
Run Keyword If Test Failed Capture Page Screenshot
To test Teardown, but considering I have huge and complex test suites with hundreds of tests and a nested structure - I would need to add this to so many Teardowns that it seems ugly to me.
I've experimented a bit. I thought the way forward was to use listener. So I tried this:
class ExtendedSelenium(Selenium2Library):
ROBOT_LISTENER_API_VERSION = 3
def __init__(self, timeout=5.0, implicit_wait=0.0, run_on_failure='Capture Page Screenshot', screenshot_root_directory=None):
super(ExtendedSelenium, self).__init__(timeout, implicit_wait, run_on_failure, screenshot_root_directory)
self.ROBOT_LIBRARY_LISTENER = self
def _end_test(self, data, result):
if not result.passed:
screenshot_name = data.id + data.name.replace(" ", "_") + ".png"
try:
self.capture_page_screenshot(screenshot_name)
except:
pass
It captures the screenshot but the picture is then not visible in the log. I'm able to display it in the test message, adding this step after capturing:
BuiltIn().set_test_message("*HTML* {} <br/> <img src={}>".format(result.message, screenshot_name))
But still, it isn't the best.
Then I tried a different approach with the Visitor interface (used with --prerunmodifier):
from robot.api import SuiteVisitor
class CaptureScreenshot(SuiteVisitor):
def end_test(self, test):
if test.status == 'FAIL':
test.keywords.create('Capture Page Screenshot', type='teardown')
But it replaces any existing Test Teardown by the new one (with only one keyword 'Capture Page Screenshot'). I thought I would be able to modify existing Teardowns by adding the Capture keyword, but I wasn't.
Is there any nice, clean, pythonic way to do this? Did I miss something?
Finally it ended up with library listener as below. I have to agree with Bryan: the code isn't short and nice but it fulfils the desired goal - single point in suite where screenshot capturing is defined.
As a big advantage I see the possibility to capture screenshot for failed setup - in some cases it helps us to identify possible infrastructure problems.
Please note the part with ActionChains - it zooms out in browser. Our front-end app uses partial page scroll and with this zoom we are able to see more content inside of that scroll which is really helpful for us. Result of this ActionChains differs for each browser, so this is truly workaround.
#!/usr/bin/python
# -*- coding: utf-8 -*-
from Selenium2Library import Selenium2Library
from selenium.common.exceptions import StaleElementReferenceException, WebDriverException
import re
class ExtendedSelenium(Selenium2Library):
""" Robot Framework library extending Robot Framework Selenium2Library library.
"""
ROBOT_LISTENER_API_VERSION = 2
DON_NOT_CAPTURE_KEYWORDS = ["Run Keyword And Ignore Error", "Run Keyword And Expect Error", "Run Keyword And Return Status", "Wait Until.*"]
def __init__(self, timeout=5.0, implicit_wait=0.0, run_on_failure='', screenshot_root_directory=None):
super(ExtendedSelenium, self).__init__(timeout, implicit_wait, run_on_failure, screenshot_root_directory)
self.ROBOT_LIBRARY_LISTENER = self
self._is_current_keyword_inside_teardown = False
self._do_not_capture_parent_keywords_count = 0
self._screenshot_was_captured = False
def _start_test(self, name, attributes):
""" Reset flags at the begin of each test.
"""
self._do_not_capture_parent_keywords_count = 0
self._is_current_keyword_inside_teardown = False
self._screenshot_was_captured = False
def _start_keyword(self, name, attributes):
""" Set keyword flag at the beginning of teardown.
If the keyword is one of the 'do not capture keywords' increase _do_not_capture_parent_keywords_count counter.
"""
if attributes["type"] == "Teardown":
self._is_current_keyword_inside_teardown = True
if any(kw for kw in self.DON_NOT_CAPTURE_KEYWORDS if re.match(kw, attributes["kwname"])):
self._do_not_capture_parent_keywords_count += 1
def _end_keyword(self, name, attributes):
"""If the keyword is one of the 'do not capture keywords' decrease _do_not_capture_parent_keywords_count counter.
Capture Screenshot if:
- keyword failed AND
- test is not in teardown phase AND
- the parent keyword isn't one of the 'do not capture keywords'
RuntimeError exception is thrown when no browser is open (backend test), no screenshot is captured in this case.
"""
if any(kw for kw in self.DON_NOT_CAPTURE_KEYWORDS if re.match(kw, attributes["kwname"])):
self._do_not_capture_parent_keywords_count -= 1
if not attributes["status"] == "PASS" and not self._is_current_keyword_inside_teardown and self._do_not_capture_parent_keywords_count == 0 and not self._screenshot_was_captured:
self._screenshot_was_captured = True
try:
self.capture_page_screenshot()
# TODO refactor this so it is reusable and nice!
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
ActionChains(super(ExtendedSelenium, self)._current_browser()).send_keys(Keys.CONTROL, Keys.SUBTRACT, Keys.NULL).perform()
ActionChains(super(ExtendedSelenium, self)._current_browser()).send_keys(Keys.CONTROL, Keys.SUBTRACT, Keys.NULL).perform()
self.capture_page_screenshot()
ActionChains(super(ExtendedSelenium, self)._current_browser()).send_keys(Keys.CONTROL, '0', Keys.NULL).perform()
except RuntimeError:
pass
Any comments are welcome.
I wrapped everything in a try/except block for my tests. That way at any point if the test fails the screen shot is in the except block and I get a screen shot the moment before/after/...whatever.
try:
class UnitTestRunner():
...
except:
driver.save_screenshot('filepath')
I'm trying to create a few tests in appium, but when i first run the application i have to pass through terms of use screen and first run screens.
Since i only have to run these steps just once, i'm having some hard time to run other tests (when i already ran these screens).
Any ideas of how can i do it?
import os, unittest, time
from appium import webdriver
from time import sleep
class Tests(unittest.TestCase):
def setUp(self):
desired_caps = {}
desired_caps['platformName'] = 'Android'
desired_caps['platformVersion'] = '5.1'
desired_caps['deviceName'] = '0429058934'
desired_caps['app'] = os.path.abspath(os.path.join(os.path.dirname(__file__),'mypath'))
desired_caps['appPackage'] = ''
desired_caps['appActivity'] = '.MainActivity'
desired_caps['fullReset'] = True
self.driver = webdriver.Remote('http://localhost:4723/wd/hub', desired_caps)
self.driver.implicitly_wait(30)
def tearDown(self):
self.driver.quit()
def first_run(self):
self.driver.find_element_by_id("p_agreement_btn").click()
next = self.driver.find_element_by_id("next_text")
next.click()
next.click()
next.click()
next.click()
self.driver.find_element_by_id("btn_ok").click()
def test_optimization(self):
self.driver.find_element_by_id("new_powerpro_tab_view_text").click()
self.driver.find_element_by_id("optimization_button").click()
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(Tests)
unittest.TextTestRunner(verbosity=2).run(suite)
You are using UnitTest framework for python which is not support dependency like TestNg. if you want to create dependent test then you user Proboscis testing framework, it has same feature like TestNg. Go though above link on read document specially annotations like " #test(depends_on=[UserTests.successful_login])"
For that you can used Proboscis.
Refer link : https://pythonhosted.org/proboscis/
I am running a a selenium functional test using nose from inside a django function using:
arg = sys.argv[:1]
arg.append('--verbosity=2')
arg.append('-v')
out = nose.run(module=ft1.testy1, argv=arg, exit=False)
I have created the functional test using the selenium IDE. Part of the test looks like:
class y1(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Firefox()
self.driver.implicitly_wait(30)
self.base_url = "https://www.yahoo.com/"
self.verificationErrors = []
self.accept_next_alert = True
def test_y1(self):
driver = self.driver
driver.get(self.base_url)
driver.find_element_by_link_text("Weather").click()
driver.save_screenshot('out1.png')
return " this is a returned value"
I want to return a string value (" this is a returned value") to the calling function. How can I do this?
The output of your nose.run() does not correspond to your test method. The default behavior for unittest.TestCase is to throw an exception. If you would like to signal an external code some specific detail, you can always do it through global/class variables, files, etc.
For example, this is how to do it with a class variable (results)
ft1_runner.py:
import nose
import ft1_test
if __name__ == '__main__':
out = nose.run(module=ft1_test, exit=False)
print 'Y1.test_y1 test results returned:', ft1_test.Y1.results['test_y1']
ft1_test.py:
import unittest
class Y1(unittest.TestCase):
results = {}
def test_y1(self):
Y1.results['test_y1'] = "this is a returned value"
I think it would help if you can describe the problem you are trying to solve: this seems a little awkward and error prone (what if the test is called twice, or test skipped, etc.)
My English is very poor but I'll try my best to describe the problem I encountered.
I used selenium webdriver to test a web site and the language that I used to write my script is python.Because of this,I used Pyunit.
I know that if my test suite have no exceptions,the webdriver instance will be closed correctly,(by the way, I used chrome) however,once a exception was threw,the script will be shut down and I have to close chrome manually.
I wonder that how can I achieved that when a python process quits , any remaining open WebDriver instances will also be closed.
By the way, I used Page Object Design Pattern,and the code below is a part of my script:
class personalcenter(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Chrome()
self.page = personalCenter(self.driver,"admin","123456")
def testAddWorkExp(self):
blahblahblah...
def tearDown(self):
self.page.quit()
self.driver.quit()
if __name__ == "__main__":
unittest.main()
I haved searched the solution of this problem for a long time ,but almost every answer is depended on java and junit or testNG.How can I deal with this issue with Pyunit?
Thanks for every answer.
From the tearDown() documentation:
Method called immediately after the test method has been called and
the result recorded. This is called even if the test method raised an
exception, so the implementation in subclasses may need to be
particularly careful about checking internal state. Any exception
raised by this method will be considered an error rather than a test
failure. This method will only be called if the setUp() succeeds,
regardless of the outcome of the test method. The default
implementation does nothing.
So, the only case where tearDown will not be called is when setUp fails.
Therefore I would simply catch the exception inside setUp, close the driver, and re-raise it:
def setUp(self):
self.driver = webdriver.Chrome()
try:
self.page = personalCenter(self.driver,"admin","123456")
except Exception:
self.driver.quit()
raise
Hope!! this will help you. you might need to close the driver with boolean expression as False
from selenium import webdriver
import time
class Firefoxweb(object):
def __init__(self):
print("this is to run the test")
def seltest(self):
driver=webdriver.Chrome(executable_path="C:\\Users\Admin\PycharmProjects\Drivers\chromedriver.exe")
driver.get("URL")
driver.find_element_by_xpath("//*[#id='btnSkip']").click()
driver.find_element_by_xpath("//*[#id='userInput']").send_keys("ABC")
driver.find_element_by_xpath("//*[#id='passwordInput']").send_keys("*******")
driver.find_element_by_xpath("//*[#id='BtnLogin']").click()
driver.close(False)
FF=Firefoxweb()
FF.seltest()