How to write appropriate Xpath to locate text value - python

I am trying to use python, selenium and xpath to grab the text $0.180 in the following block of HTML,
<div class="wlp-values-div">
<span class="wlp-last-price-span">$0.180</span>
This code is deep into a websites HTML and I tried to use the following Xpath to locate it:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get("https://smallcaps.com.au/stocks/?symbol=AWJ")
time.sleep(3)
webdriver.ActionChains(driver).send_keys(Keys.ESCAPE).perform()
ele = driver.find_element_by_xpath("//div[#class='wlp-values-div']/span[#class='wlp-last-price-span']")
But I receive the error:
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[#class='wlp-values-div']/span[#class='wlp-last-price-span']"}
Any help would be greatly appreciated

The last price field is within nested <iframe> elements so you have to:
Induce WebDriverWait for the parent frame to be available and switch to it.
Induce WebDriverWait for the child frame to be available and switch to it.
Induce WebDriverWait for the desired element to be visible.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get('https://smallcaps.com.au/stocks/?symbol=AWJ')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "svg[data-name='close']"))).click()
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#wl-summary")))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe.wl-quote-frame")))
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span.wlp-last-price-span"))).get_attribute("innerHTML"))
driver.quit()
Using XPATH:
driver.get('https://smallcaps.com.au/stocks/?symbol=AWJ')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//*[name()='svg' and #data-name='close']"))).click()
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='wl-summary']")))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#class='wl-quote-frame']")))
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[#class='wlp-last-price-span']"))).get_attribute("innerHTML"))
driver.quit()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Console Output:
$0.190
You can find a detailed relevant discussion in How To sign in to Applemusic With Python Using Chrome Driver With Selenium
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
selenium in python : NoSuchElementException: Message: no such element: Unable to locate element

<iframe class="wl-summary" id="wl-summary" src="https://cloud.weblink.com.au/clients/smallcaps/summary/summary.aspx?symbol=AWJ&" width="100%" height="25" frameborder="0" scrolling="no" allowtransparency="true" style="height: 1077px;"></iframe>
<iframe class="wl-quote-frame" src="quoteFrame.aspx?symbol=AWJ&" scrolling="no" frameborder="0" height="1" style="height: 177px;"></iframe>
It's in a double nested iframe so you have to switch to it. This also has the appropriate waits instead of using time.sleep() and hoping the elements load. It also clicks the popup to get out of the way and prints the spans text.
wait = WebDriverWait(driver, 10)
driver.get("https://smallcaps.com.au/stocks/?symbol=AWJ")
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#tve_editor > div'))).click()
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CLASS_NAME, 'wl-summary')))
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CLASS_NAME, 'wl-quote-frame')))
elem=wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, 'span.wlp-last-price-span')))
print(elem.text)
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Related

How to click pop up window button within a iframe using Selenium with Pyhton

I want to click the "Akkoord" button but I am unable to do that. I already tried different methods but they are not working. Any help will be appreciated.
one of the codes I tried.
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
import time
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get('https://www.demorgen.be/nieuws')
time.sleep(20)
driver.find_elements_by_class_name('message-component message-button no-children pg-accept-button')[0].click()
The element Akkoord is within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("https://www.demorgen.be/nieuws")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[id^='sp_message_iframe']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-label='Akkoord']"))).click()
Using XPATH:
driver.get("https://www.demorgen.be/nieuws")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[starts-with(#id, 'sp_message_iframe')]")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#aria-label='Akkoord']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
selenium in python : NoSuchElementException: Message: no such element: Unable to locate element
use xpath or css
css :
[class="message-component message-button no-children pg-accept-button"]
driver.find_elements_by_css_selector('[class="message-component message-button no-children pg-accept-button"]')[0].click()
xpath:
//*[#class="message-component message-button no-children pg-accept-button"]
driver.find_elements_by_xpath('//*[#class="message-component message-button no-children pg-accept-button"]')[0].click()
find_elements_by_class_name expects single class name as argument thats why its not working as space in class indicates multiple classes.
THe find by class actually uses css under the hood. So if you want to find element having multiple class . You can replace space with '.' (THis works only in python)
driver.find_elements_by_class_name('message-component.message-button.no-children.pg-accept-button')[0].click()
Also
The element is inside iframe
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
frame = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR,
"#sp_message_container_404503 iframe")))
driver.switch_to_frame(frame)
driver.find_element_by_css_selector(
'[class="message-component message-button no-children pg-accept-button"]').click()

Message: element click intercepted: Element <span>...</span> is not clickable at point (657, 594). Other element would receive the click with Selenium

I was making a webscraper to get gpu stocks from https://www.nvidia.com/en-us/shop/geforce/?page=1&limit=9&locale=en-us to get my hands on a 30 series card, I am using python with bs4 and selenium for this.
I want to load more shopping items, on the website it has this load more button. So I grabbed its class and made it so selenium clicks it:
driver.find_element_by_class_name("buy-link").click()
but it says that the element in non interactiable, HTML for the button
The exact error it gives me is:
Message: element click intercepted: Element <span class="extra_style buy-link" data-color="#76b900" data-secondary-color="#fff" style="visibility: visible; cursor: pointer;" data-mpn-code="NVGFT070">...</span> is not clickable at point (657, 594). Other element would receive the click: <div class="popBg" id="nv-buy-now-model" style="display: block;">...</div>
I don't know much HTML, how can I achieve clicking this button
The LOAD MORE element is a Angular element so to click on it you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("https://www.nvidia.com/en-us/shop/geforce/?page=1&limit=9&locale=en-us")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div#cookiePolicy-btn-close>span"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.buy-link.load-more-btn[value='LOAD MORE']"))).click()
Using XPATH:
driver.get("https://www.nvidia.com/en-us/shop/geforce/?page=1&limit=9&locale=en-us")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#id='cookiePolicy-btn-close']/span"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#class='buy-link load-more-btn' and #value='LOAD MORE']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
The problem is that there are some 20 elements on that page that match your first locator, .find_element_by_class_name("buy-link"). In this case, if you make your locator more specific, you can isolate it down to just the "LOAD MORE" INPUT button.
Try this instead
driver.find_element_by_css_selector("input.buy-link").click()
It's generally a good practice to add a wait unless you are sure you won't ever need it.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[text()='CLOSE']")).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.buy-link")).click()
See the docs for more info and details.

Selenium returning NoSuchElementException but the element exists

I am having this code
driver.implicitly_wait(300) # I have tried different timers
driver.find_element_by_xpath("(//button[#aria-pressed='false'])[1]").click()
But I get this error:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"(//button[#aria-pressed='false'])[1]"}
However, this exists in my HTML content, like:
<div class="webix_el_box" style="width:90px; height:26px">
<button aria-pressed="false" type="button" class="webix_img_btn" style="line-height:22px;">
<span class="webix_icon_btn fa-lock red" style="max-width:22px;">
</span> Read Only
</button>
</div>
I have also tried to use the XPath expression:
//button[#class='webix_img_btn']//child::span[#class='webix_icon_btn fa-lock red']
Both XPath expressions worked fine in Google Chrome.
In the website I am able to find the button using Ctrl + F and 'Read Only', can I use it in Selenium?
It is not working because of Selenium Driver is one page behind ajax call after large json data download.
To click on the element with text as Read Only you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.webix_el_box > button.webix_img_btn"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='webix_el_box']/button[#class='webix_img_btn' and contains(., 'Read Only')]"))).click()
Note: You have to add the following imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
References
You can find a couple of relevant discussions on NoSuchElementException in:
"selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element" while clicking a 'Next' button with Selenium
Selenium in Python: "NoSuchElementException: Message: no such element: Unable to locate element"
Try doing just
driver.implicitly_wait(300)
driver.find_element_by_xpath("//button[#aria-pressed='false']").click()
This will click on the first element that satisfies the requirements.

How to download PDF with selenium WebDriver

I would like to download a PDF in Python using Selenium WebDriver. However, for some reason I cannot select / click the download button. It might be due to the fact that the button is not focusable?
This is the button:
<svg aria-hidden="true" focusable="false" data-prefix="fas" data-icon="spinner-third" class="svg-inline--fa fa-spinner-third fa-w-16 fa-icon menubar-h-14" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path fill="currentColor" d="M456.433 371.72l-27.79-16.045c-7.192-4.152-10.052-13.136-6.487-20.636 25.82-54.328 23.566-118.602-6.768-171.03-30.265-52.529-84.802-86.621-144.76-91.424C262.35 71.922 256 64.953 256 56.649V24.56c0-9.31 7.916-16.609 17.204-15.96 81.795 5.717 156.412 51.902 197.611 123.408 41.301 71.385 43.99 159.096 8.042 232.792-4.082 8.369-14.361 11.575-22.424 6.92z"></path></svg>
And here is my current code to reproduce:
from selenium import webdriver
import time
website = "https://onlinelibrary.wiley.com/doi/epdf/10.1111/jofi.12895"
driver = webdriver.Chrome()
driver.get(website)
driver.set_window_size(1024, 768)
time.sleep(4)
As the the desired element is within an <iframe> so to invoke click() on the element you have to:
Induce WebDriverWait for the desired frame_to_be_available_and_switch_to_it().
Induce WebDriverWait for the desired element_to_be_clickable().
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe.rc-reader-frame")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "menu-button.download span"))).click()
Using XPATH:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe[#class='rc-reader-frame']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//menu-button[#class='download']//span[text()='PDF']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser snapshot:
Reference
You can find a couple of relevant discussions in:
Switch to an iframe through Selenium and python
Ways to deal with #document under iframe

Python Selenium: Element Is Not Clickable At Point

I am using the code below to click on the highlighted tab that you see on the attached screenshot with the corresponding HTML code. The tab is always visible on the bottom of the script so there is no need to scroll down to it.
element = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.ID, "tab-button-settings")))
element.click()
The error that I get is below. What am I doing wrong?
Message: unknown error: Element <a class="tab-button has-icon icon-only" href="#" role="tab" ng-
reflect-tab="[object Object]" id="tab-button-settings" aria-controls="tabpanel-t0-1" aria-
selected="false">...</a> is not clickable at point (1440, 1017). Other element would receive the
click: <div class="click-block click-block-enabled click-block-active"></div>
(Session info: chrome=78.0.3904.108)
you can use action class to avoid above issue
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
element = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.ID, "tab-button-settings"))
ActionChains(driver).move_to_element(element).click().perform()
Another way alternative to javascript is to use touch actions
from selenium.webdriver.common.touch_actions import TouchActions
element = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.ID, "tab-button-settings")))
touchactions = TouchActions(driver)
touchactions.tap(element).perform()
The desired element is an Angular element so to locate and click() on the element you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.tab-button.has-icon.icon-only#tab-button-settings"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='tab-button has-icon icon-only' and #id='tab-button-settings']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
In case the unknown error still occurs you may have to induce WebDriverWait additionally for the invisibility_of_element_located() and you can use either of the following solutions:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.invisibility_of_element_located((By.CSS_SELECTOR, "div.click-block.click-block-enabled.click-block-active")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.tab-button.has-icon.icon-only#tab-button-settings"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.invisibility_of_element_located((By.XPATH, "//div[#class='click-block click-block-enabled click-block-active']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='tab-button has-icon icon-only' and #id='tab-button-settings']"))).click()
References
You can find a couple of relevant discussions in:
Selenium Web Driver & Java. Element is not clickable at point (x, y). Other element would receive the click
Element MyElement is not clickable at point (x, y)… Other element would receive the click
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element is not clickable with Selenium and Python

Categories