Recently I have started working with multiple tabs in Selenium and I have encountered a strange problem. When I'm executing this code:
WebDriverWait(driver, 10).until(EC.number_of_windows_to_be(2))
driver.switch_to.window(driver.window_handles[-1])
time.sleep(1)
url_in_page_source= eu.look_for_url_in_page_source(
page_html=driver.page_source,
left_delimiter='placeholder',
right_delimiter='placeholder'
)
driver.close()
driver.switch_to.window(driver.window_handles[0])
# time.sleep(10) # <--- this is fixing the error
return url_in_page_source
and immediately after the return statement when I'm trying to visit extracted url by driver.get() I'm getting this error:
Message: no such window: target window already closed
from unknown error: web view not found
However I have found out that adding a simple time.sleep(10) just before return statement is fixing the issue, but what is more strange to me - when I have tried to lower the wait time just below 10 secs the error is still existing. I have no idea why it is happening. Maybe I'm doing something wrong. I will be very grateful for any help and explanations.
Edit:
Here's source code of the eu.look_for_url_in_page_source() as per #JeffC request
def look_for_url_in_page_source(page_html, left_url_delimiter, right_url_delimiter):
print('Processing URL with f:look_for_url_in_page_source()')
# extracts multiple URLs from page_source
extracted_urls = []
while True:
# check if delimiters are present in page_html
find_left_url_delimiter = page_html.find(left_url_delimiter)
find_right_url_delimiter = page_html.find(right_url_delimiter)
if find_left_url_delimiter == -1 or find_right_url_delimiter == -1:
if len(extracted_urls) > 0:
return extracted_urls
print('f:look_for_url_in_page_source() was not able to get any text.')
return False
left_url_delimiter_pos = find_left_url_delimiter + len(left_url_delimiter)
right_url_delimiter_pos = page_html[left_url_delimiter_pos:].find(right_url_delimiter) + left_url_delimiter_pos
extracted_url = page_html[left_url_delimiter_pos:right_url_delimiter_pos].strip()
extracted_urls.append(extracted_url)
page_html = page_html[right_url_delimiter_pos:]
There are a lot many process involved in the process of closing a tab and switching to the parent Browsing Context which is beyond the scope of our naked eyes. Having said that, neither
driver.switch_to.window(driver.window_handles[-1])
is an ideal way to switch to the new tab, nor
driver.switch_to.window(driver.window_handles[0])
is an ideal way to switch to the parent tab.
You can find a detailed discussion on tab switching in Open web in new tab Selenium + Python
References
A few relevant discussions:
"NoSuchWindowException: no such window: window was already closed" while switching tabs using Selenium and WebDriver through Python3
I am using an automation script with selenium web driver. It usually works fine, but sometimes the content on a page is changing.
If the element it is looking for is not there, it is crashing instead of executing the else statement.
I Tried to use another function with try and NoSuchElementException, but I do get another error about NoSuchElementException.
This is the if else statement:
#Look for link text "Stores"
find_store = driver.find_element_by_link_text('Stores')
if find_store:
find_store.click()
else:
driver.get("https://example.com/myaccount")
time.sleep(5)
This is the try statement:
try:
find_store = driver.find_element_by_link_text('Stores')
if find_store.is_displayed():
find_store.click() # this will click the element if it is there
print("Store found, all good.")
except NoSuchElementException:
driver.get("https://example.com/myaccount")
print("Store was not found, visiting base URL to search for store")
time.slep(5)
In the first script I expect it to search for the element. If its not there, then visit the URL so I can search for the element there. But it never opens the URL and I get this error:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element:
In the second script I expect to look for the element. If the element is not there, it should visit the base url and try again later in the code. However here I get this error:
except NoSuchElementException:
NameError: name 'NoSuchElementException' is not defined
I would like to learn how I can avoid the script crashing and actually execute an alternative action if the element is not there. Thank you.
Use Following :
if len(driver.find_elements_by_link_text('Stores'))> 0:
driver.find_element_by_link_text('Stores').click()
else:
driver.get("https://example.com/myaccount")
time.sleep(5)
I am trying to print the XPath data in the console. But the process stops after loading the first page even without no errors.
This is my code:
browser.get('https://stackoverflow.com/questions?pagesize=10')
while True:
try:
elm = browser.find_element_by_link_text("next")
elm.click()
labels = browser.find_element_by_xpath('/html/body/div[3]/div[2]/div[1]/div[3]/div/div/div[1]/div[2]/h3/a')
for label in labels:
print label.text
except:
break
What am I missing?
The reason why are not getting any errors is because you are catching them and then just using break.
You also had an issue with your XPATH to the question labels. I included a scroll to the next link in case you are receiving the cookies notification at the bottom like I was. Here is a working example:
NOTE: This was testing in Python 3 using current Chrome build 67 and
chromedriver 2.40
import traceback
from selenium.common.exceptions import NoSuchElementException
browser.get('https://stackoverflow.com/questions?pagesize=10')
while True:
try:
elm = browser.find_element_by_link_text("next")
browser.execute_script("return arguments[0].scrollIntoView();", elm)
elm.click()
labels = browser.find_elements_by_xpath('.//a[#class="question-hyperlink"]')
for label in labels:
print(label.text)
except NoSuchElementException:
print("only catching this exception now when you run out of the next elements, other exceptions will raise")
print(traceback.format_exc())
break
for i in driver.find_elements_by_class_name("endorse-count"):
try:
i.click()
except:
continue
elem = WebDriverWait(driver, 100).until(EC.presence_of_element_located((By.CLASS_NAME, "dialog-window")))
src = elem.get_attribute("innerHTML")
add_skill(name, src)
WebDriverWait(driver, timeout=10)
I'm getting the following error while running the above code -
selenium.common.exceptions.StaleElementReferenceException: Message: u'Element is no longer attached to the DOM' ; Stacktrace:
at fxdriver.cache.getElementAt (resource://fxdriver/modules/web_element_cache.js:7646)
for line -
src = elem.get_attribute("innerHTML")
I'm running this code on LinkedIn user profile page, after logging in.
I tried putting the following line of code after "i.click()" -
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
But then I see that function "add_skill(name, src)" is not called and none of the code after driver.manage() is called, though for loop and further i.click() work fine.
Selenium is trying to complete actions (such as clicking a button or link) before verifying that the target element has rendered on the page. Selenium can be more patient, but you have to explicitly ask him to be.
For example, if you are testing something that makes an AJAX request, you can try something like this (in Ruby):
# timeout is in seconds
def wait_for_ajax(timeout=x)
time_limit, interval = (Time.now + timeout), 0.5
loop do
break if #driver.execute_script "return jQuery.active == 0"
sleep interval
raise "Wait for AJAX timed out after waiting for #{timeout} seconds" if Time.now > time_limit
end
end
To ensure your tests are fully comprehensive, always make Selenium waits for elements to load before running a task.
I had faced a similar issue and tried refreshing the page before finding that element, and it worked...
driver.navigate().refresh();
Though I couldnt reason out how this worked.
If this works for you as well, please let me know. I just want to learn more about this exception.
you can refer this page to learn about a similar issue
I had a similar problem when trying to execute some javascript (IJavaScripExecutor). I created an IWebElement and passed that to the JSE and that failed for me. When I moved the driver.FindElement(BySelector) into my JSE call, then it worked. (C# code ahead.)
Instead of:
IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
IWebElement tableEl = driver.FindElement(selector);
js.ExecuteScript(script, tableEl);
I had to do:
IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
js.ExecuteScript(script, driver.FindElement(selector));
You may have to do something similar: move your selector or element creation onto the same line as what you are trying to do. Or, maybe, in your case:
src = driver.find_element_by_class_name("dialog-window").get_attribute("innerHTML")
Upon closer inspection, that's what looks to be your problem, there's a stale web element object when you try to use the get_attribute method.
This question has been asked over and over again - and in-spite of trying all the hacks I still can't seem to figure out what's wrong.
I tried increasing the implicitly_wait to 30 (and even increased it upto 100) - yet it did not work.
Use case -: I am trying to create a list that wil populate all the items in the page here, as a base case - and I intend to bind this to a mini-module that I already have with scrapy which has all (pages with similar web elements) crawled links - so essentially will be building the whole pipeline, post I am done with this.
###My source code - generated via Selenium IDE, exported to a Python webdriver and manipulated a little later ###
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support.wait import WebDriverWait
import unittest, time, re
class Einstein(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Firefox()
self.driver.implicitly_wait(30)
self.base_url = "http://shopap.lenovo.com/in/en/laptops/"
self.verificationErrors = []
self.accept_next_alert = True
def test_einstein(self):
driver = self.driver
driver.get(self.base_url)
print driver.title
driver.find_element_by_link_text("T430").click()
print driver.title
# driver.find_element_by_xpath("id('facetedBrowseWrapper')/div/div/div[1]/div[2]/ul[1]/li[1]/a").click()
driver.find_element_by_xpath("//div[#id='subseries']/div[2]/div/p[3]/a").click()
print driver.title
# driver.find_element_by_xpath("//div[#id='subseries']/div[2]/div/p[3]/a").click()
try: self.assertEqual("Thinkpad Edge E530 (Black)", driver.find_element_by_link_text("Thinkpad Edge E530 (Black)").text)
except AssertionError as e: self.verificationErrors.append(str(e))
# Everything ok till here
#**THE CODE FAILS HERE**#
laptop1 = driver.find_element_by_link_text("Thinkpad Edge E530 (Black)").text
print laptop1
price1 = driver.find_element_by_css_selector("span.price").text
print price1
detail1 = self.is_element_present(By.CSS_SELECTOR, "div.desc.std")
print detail1
def is_element_present(self, how, what):
try: self.driver.find_element(by=how, value=what)
except NoSuchElementException, e: return False
return True
def is_alert_present(self):
try: self.driver.switch_to_alert()
except NoAlertPresentException, e: return False
return True
def close_alert_and_get_its_text(self):
try:
alert = self.driver.switch_to_alert()
alert_text = alert.text
if self.accept_next_alert:
alert.accept()
else:
alert.dismiss()
return alert_text
finally: self.accept_next_alert = True
def tearDown(self):
self.driver.quit()
self.assertEqual([], self.verificationErrors)
if __name__ == "__main__":
unittest.main()
Errors & output :
ekta#ekta-VirtualBox:~$ python einstein.py
Laptops & Ultrabooks | Lenovo (IN)
ThinkPad T430 Laptop PC for Business Computing | Lenovo (IN)
Buy Lenovo Thinkpad Laptops | Lenovo Thinkpad Laptops Price India
E
======================================================================
ERROR: test_einstein (__main__.Einstein)
----------------------------------------------------------------------
Traceback (most recent call last):
File "einstein.py", line 27, in test_einstein
try: self.assertEqual("Thinkpad Edge E530 (Black)", driver.find_element_by_link_text("Thinkpad Edge E530 (Black)").text)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 246, in find_element_by_link_text
return self.find_element(by=By.LINK_TEXT, value=link_text)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 680, in find_element
{'using': by, 'value': value})['value']
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 165, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 158, in check_response
raise exception_class(message, screen, stacktrace)
NoSuchElementException: Message: u'Unable to locate element: {"method":"link text","selector":"Thinkpad Edge E530 (Black)"}' ; Stacktrace:
at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmphli5Jg/extensions/fxdriver#googlecode.com/components/driver_component.js:8444)
at fxdriver.Timer.prototype.setTimeout/<.notify (file:///tmp/tmphli5Jg/extensions/fxdriver#googlecode.com/components/driver_component.js:386)
----------------------------------------------------------------------
Ran 1 test in 79.348s
FAILED (errors=1)
Questions & comments:
If you are answering this question - please mention why this specific "find_element_by_link_text" does not work.
(Very Basic) In the GUI of my selenium IDE -> Show all available commands - why dont I see the css (find_element_by_css_selector) for all the web elements - is there a way to force feed an element to be read as a CSS selector ?
In case you suggest using some other locator - please mention if that will be consistent way to fetch elements, given (1)
Does assert work to capture the exceptions and "move on" - since even after trying "verify" , "assert" loops, I still cant fetch this "find_element_by_link_text"
I tried using Xpath to build this "element" , but in the view Xpath (in firefox) - I see nothing, to clue why that happens (Of course I removed the namespace ":x" )
Other things I tried apart from implicity_wait(30):
find_element_by_partial_link(“Thinkpad”) and appending Unicode to this (wasn’t sure if it was reading the brackets ( , driver.find_element_by_link_text(u"Thinkpad Edge E530 (Black)").text, still did not work.
Related questions:
How to use find_element_by_link_text() properly to not raise NoSuchElementException?
NoSuchElement Exception using find_element_by_link_text when implicitly_wait doesn't work?
It happened to me before that the find_element_by_link_text method sometimes works and sometimes doesn't work; even in a single case. I think it's not a reliable way to access elements; the best way is to use find_element_by_id.
But in your case, as I visit the page, there is no id to help you. Still you can try find_elements_by_xpath in 3 ways:
1- Accessing title: find_element_by_xpath["//a[contains(#title = 'T430')]"]
2- Accessing text: find_element_by_xpath["//a[contains(text(), 'T430')]"]
3- Accessing href: find_element_by_xpath["//a[contains(#href = 'http://www.thedostore.com/laptops/thinkpad-laptops/thinkpad-t430-u-black-627326q.html')]"].
Hope it helps.
NoSuchElementException is thrown when the element could not be found.
If you encounter this exception, please check the followings:
Check your selector used in your find_by...
Element may not yet be on the screen at the time of the find operation.
If webpage is still loading, check for selenium.webdriver.support.wait.WebDriverWait() and write a wait wrapper to wait for an element to appear.
Troubleshooting and code samples
You can add breakpoint just before your failing line pdb.set_trace() (don't forget to import pdb), then run your test and once your debugger stops, then do the following tests.
You could try:
driver.find_element_by_xpath(u'//a[text()="Foo text"]')
instead. This is more reliable test, so if this would work, use it instead.
If above won't help, please check if your page has been loaded properly via:
(Pdb) driver.execute_script("return document.readyState")
'complete'
Sometimes when the page is not loaded, you're actually fetching the elements from the old page. But even though, readyState could still indicate the state of the old page (especially when using click()). Here is how this is explained in this blog:
Since Selenium webdriver has become more advanced, clicks are much more like "real" clicks, which has the benefit of making our tests more realistic, but it also means it's hard for Selenium to be able to track the impact that a click has on the browsers' internals -- it might try to poll the browser for its page-loaded status immediately after clicking, but that's open to a race condition where the browser was multitasking, hasn't quite got round to dealing with the click yet, and it gives you the .readyState of the old page.
If you think this is happening because the page wasn't loaded properly, the "recommended" (however still ugly) solution is an explicit wait:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
old_value = browser.find_element_by_id('thing-on-old-page').text
browser.find_element_by_link_text('my link').click()
WebDriverWait(browser, 3).until(
expected_conditions.text_to_be_present_in_element(
(By.ID, 'thing-on-new-page'),
'expected new text'
)
)
The naive attempt would be something like this:
def wait_for(condition_function):
start_time = time.time()
while time.time() < start_time + 3:
if condition_function():
return True
else:
time.sleep(0.1)
raise Exception(
'Timeout waiting for {}'.format(condition_function.__name__)
)
def click_through_to_new_page(link_text):
browser.find_element_by_link_text('my link').click()
def page_has_loaded():
page_state = browser.execute_script(
'return document.readyState;'
)
return page_state == 'complete'
wait_for(page_has_loaded)
Another, better one would be (credits to #ThomasMarks):
def click_through_to_new_page(link_text):
link = browser.find_element_by_link_text('my link')
link.click()
def link_has_gone_stale():
try:
# poll the link with an arbitrary call
link.find_elements_by_id('doesnt-matter')
return False
except StaleElementReferenceException:
return True
wait_for(link_has_gone_stale)
And the final example includes comparing page ids as below (which could be bulletproof):
class wait_for_page_load(object):
def __init__(self, browser):
self.browser = browser
def __enter__(self):
self.old_page = self.browser.find_element_by_tag_name('html')
def page_has_loaded(self):
new_page = self.browser.find_element_by_tag_name('html')
return new_page.id != self.old_page.id
def __exit__(self, *_):
wait_for(self.page_has_loaded)
And now we can do:
with wait_for_page_load(browser):
browser.find_element_by_link_text('my link').click()
Above code samples are from Harry's blog.
Here is the version proposed by Tommy Beadle (by using staleness approach):
import contextlib
from selenium.webdriver import Remote
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.expected_conditions import staleness_of
class MyRemote(Remote):
#contextlib.contextmanager
def wait_for_page_load(self, timeout=30):
old_page = self.find_element_by_tag_name('html')
yield
WebDriverWait(self, timeout).until(staleness_of(old_page))
If you think it isn't about page load, double check if your element isn't in iframe or different window. If so, you've to switch to it first. To check list of available windows, run: driver.window_handles.
From viewing the source of the page that you provided a link to, it seems you are using an incorrect selector.
You should use instead find_elements_by_link_text(u'text here')[0] to select the first occurrence instead as there seems to be the potential for multiple links with the same link text.
So instead of:
self.assertEqual("Thinkpad Edge E530 (Black)", driver.find_element_by_link_text("Thinkpad Edge E530 (Black)").text)
You should use:
self.assertEqual("Thinkpad Edge E530 (Black)", driver.find_elements_by_link_text("Thinkpad Edge E530 (Black)")[0].text)
Solution posted by OP:
Hack 1: Instead of identifying the element as a text-link, I identified the "bigger frame" in which this element was present.
itemlist_1 = driver.find_element_by_css_selector("li.item.first").text
This will give the whole item along with the name, price and detail (and the unwanted add to cart and compare"
See the attached image for more .
Hack 2: I found that the "Buy Now" which was an image element with xPath (driver.find_element_by_xpath("//div[#id='subseries']/div[2]/div/p[3]/a").click()
, in the code above) , could be made to click/identified faster if I added the following line, before finding this by xpath. I think this sort of narrows down where the Webdriver is looking for an element. This is what I added " driver.find_element_by_css_selector("#subseries").text"
This must have decreased my wait by at least 20 seconds, on that page .Hope that helps.