I have been working on this problem for quite a bit now, and can't figure out why this is happening. I am trying to click a button. The button and the corresponding text always changes, but the XPATH stays the same.
Button I am trying to click
It doesn't work with CSS Selector either. I am using Chrome web version 106. Does anyone know why?
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
PATH = r"C:\Users\###\Downloads\chromedriver.exe"
driver = webdriver.Chrome(PATH)
# time to login
driver.get("https://clever.com")
time.sleep(60)
try:
driver.find_element(By.XPATH, '//*[#id="root"]/div/div/div[2]/div[3]/div/div/div/div[2]/button[2]').click()
except NoSuchElementException:
print("no such element")
pass
time.sleep(5)
Capturing the XPATH for the element
Try this:
driver.find_element("xpath", '//*[#id="root"]/div/div/div[2]/div[3]/div/div/div/div[2]/button[2]').click()
May be you using not correct syntax
Related
I'm trying use selenium and firefox, but it's just open the browser and set this error:
Error
With Chrome is ok.
This is the code:
from selenium import webdriver
driver = webdriver.Firefox(executable_path = "c:/my-apps/geckodriver.exe")
Thanks very much
When you try to find any element in an HTML page that does not exist, NoSuchElementException will be raised.
First Possible Solution:
You need to import
from selenium.common.exceptions import NoSuchElementException
Then you can use try except block
try:
your_element = driver.find_element_by_xpath(".//*[#id='loginForm:username']")
your_element.click()
except NoSuchElementException:
pass
Second Possible Solution:
Without importing anything, checking whether that element exist, if it does then it will be clicked
your_element = driver.find_elements_by_xpath(".//*[#id='loginForm:username']")
if len(your_element) > 0:
elem[0].click()
HEY You can use this code as given below
driver = webdriver.Firefox()
driver.implicitly_wait(5)
driver.maximize_window()
driver.get('URL')
I think you should put your geckodriver in the same folder where you are coding
and if this does not work you need to upgrade geckodriver
HEY According to your ERROR
when you open the website your 1 element is not correct
you need to change your 1 element of your website
wait=WebDriverWait(self.driver,60)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"#loginForm:username"))).send_keys("a")
Generally Firefox is a tad slower than Chrome so your issue may be timing for find elements. What is recommended is waiting for the element to be clickable and then proceeding to send keys to it.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I am trying to scrape data from the Sunshine List website (http://www.sunshinelist.ca/) using the Selenium package but I get the following error mentioned below. From several other related posts I understand that I need to use the WebDriverWait to explicitly ask the driver to wait/refresh but I am unable to identify where and how I should call the function.
Screenshot of Error
StaleElementReferenceException: Message: The element reference
of (tr class="even") stale: either the element is no longer attached to the DOM or the
page has been refreshed
import numpy as np
import pandas as pd
import requests
import time
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
ffx_bin = FirefoxBinary(r'C:\Users\BhagatM\AppData\Local\Mozilla Firefox\firefox.exe')
ffx_caps = DesiredCapabilities.FIREFOX
ffx_caps['marionette'] = True
driver = webdriver.Firefox(capabilities=ffx_caps,firefox_binary=ffx_bin)
driver.get("http://www.sunshinelist.ca/")
driver.maximize_window()
tablewotags1=[]
while True:
divs = driver.find_element_by_id('datatable-disclosures')
divs1=divs.find_elements_by_tag_name('tbody')
for d1 in divs1:
div2=d1.find_elements_by_tag_name('tr')
for d2 in div2:
tablewotags1.append(d2.text)
try:
driver.find_element_by_link_text('Next →').click()
except NoSuchElementException:
break
year1=tablewotags1[0::10]
name1=tablewotags1[3::10]
position1=tablewotags1[4::10]
employer1=tablewotags1[1::10]
df1=pd.DataFrame({'Year':year1,'Name':name1,'Position':position1,'Employer':employer1})
df1.to_csv('Sunshine List-1.csv', index=False)
If your problem is to click the "Next" button, you can do that with the xpath:
driver = webdriver.Firefox(executable_path=r'/pathTo/geckodriver')
driver.get("http://www.sunshinelist.ca/")
wait = WebDriverWait(driver, 20)
el=wait.until(EC.presence_of_element_located((By.XPATH,"//ul[#class='pagination']/li[#class='next']/a[#href='#' and text()='Next → ']")))
el.click()
For each click on the "Next" button -- you should find that button and click on it.
Or do something like this:
max_attemps = 10
while True:
next = self.driver.find_element_by_css_selector(".next>a")
if next is not None:
break
else:
time.sleep(0.5)
max_attemps -= 1
if max_attemps == 0:
self.fail("Cannot find element.")
And after this code does click action.
PS: Also try to add just time.sleep(x) after fiding element and then do click action.
Try this code below.
When the element is no longer attached to the DOM and the StaleElementReferenceException is invoked, search for the element again to reference the element.
Please do note I checked with Chrome:
try:
driver.find_element_by_css_selector('div[id="datatable-disclosures_wrapper"] li[class="next"]>a').click()
except StaleElementReferenceException:
driver.find_element_by_css_selector('div[id="datatable-disclosures_wrapper"] li[class="next"]>a').click()
except NoSuchElementException:
break
>>>Stale Exceptions can be handled using **StaleElementReferenceException** to continue to execute the for loop. When you try to get the element by any find_element method in a for loop.
from selenium.common import exceptions
and customize your code of for loop as:
for loop starts:
try:
driver.find_elements_by_id("data") //method to find element
//your code
except exceptions.StaleElementReferenceException:
pass
When you raise the StaleElementException that means that somthing changed in the site, but not in the list you have. So the trick is to refresh that list every time, inside the loop like this:
while True:
driver.implicitly_wait(4)
for d1 in driver.find_element_by_id('datatable-disclosures').find_element_by_tag_name('tbody').find_elements_by_tag_name('tr'):
tablewotags1.append(d1.text)
try:
driver.switch_to.default_content()
driver.find_element_by_xpath('//*[#id="datatable-disclosures_wrapper"]/div[2]/div[2]/div/ul/li[7]/a').click()
except NoSuchElementException:
print('Don\'t be so cryptic about error messages, they are good\n
...Script broke clicking next') #jk aside put some info there
break
Hope this help you, cheers.
Edit:
So I went to the said website, the layout is pretty straight forward, but the structure repeats itself like four times. So when you go about crawling the site like that something is bound to change.
So I’ve edited the code to only scrap one tbody tree. This tree comes from the first datatable-disclousure. And added some waits.
import webbrowser
from selenium import webdriver
browser = webdriver.Chrome()
browser.maximize_window()
browser.get('https://www.suntrust.com/')
browser.implicitly_wait(15)
elem = browser.find_element_by_css_selector('input#sign-on-3A69E29D-79E0-
403E-9352-5261239ADD89-user')
elem.click().send_keys('your-username')
element not visible exception error message:
I'm trying to sign into the login/password field automatically, but I keep getting this error message.
I have tried various "find_element_by" locators, but this one was recommended, so I don't think the css selector is the problem. What am I doing wrong?
It happens usually because the dom wouldn't have loaded and the Selenium script tries to find that element .. Make sure this element is not inside an Iframe . Use the selenium explicit wait until the element loads and then perform action on that button . You have to do something like this in python . The below code is just creating a wait object and then waiting for the element to load and perform next step
from selenium.webdriver.support.ui import WebDriverWait
myElem = WebDriverWait(browser, delay).until(EC.presence_of_element_located((By.ID, 'IdOfMyElement')))
print "Page is ready!"
except TimeoutException:
print "Loading took too much time!"
Here is the Answer to your Question:
The css_selector you constructed was not unique and was matching to 2 elements on the HTML DOM. The first match was invisible while the second match was our expected element. Selenium was trying to click the first element. Hence the error. Here is your own code with the modified css_selector which works well at my end:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
browser = webdriver.Chrome(chrome_options=options, executable_path="C:\\Utility\\BrowserDrivers\\chromedriver.exe")
browser.get('https://www.suntrust.com/')
browser.implicitly_wait(15)
elem = browser.find_element_by_css_selector('section[role="main"] input[id="sign-on-3A69E29D-79E0-403E-9352-5261239ADD89-user"]')
elem.send_keys('your-username')
Let me know if this Answers your Question.
I'm new to selenium and trying to automate the download of some government data. When using the code below. I manage to navigate to the right page and enter the right parmeter in the form, but then can't find a way to click the 'submit' button. I've tried find_element_by_partial_link_text("Subm").click() and I've tried find_element_by_class_name on a number of class names. Nothing works. Any ideas?
import time
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.keys import Keys
main_url="http://data.stats.gov.cn/english/easyquery.htm?cn=E0101"
driver = webdriver.Firefox()
driver.get(main_url)
time.sleep(8)
driver.find_element_by_partial_link_text("Industry").click()
time.sleep(8)
driver.find_element_by_partial_link_text("Main Economic Indicat").click()
time.sleep(8)
driver.find_element_by_id("mySelect_sj").click()
time.sleep(3)
driver.find_element_by_class_name("dtText").send_keys("last72")
time.sleep(4)
try:
driver.find_element_by_class_name("dtFoot").click()
except:
driver.find_element_by_class_name("dtFoot").submit()
Solved my own problem, the key was using
driver.find_element_by_class_name(`dtTextBtn`)
instead of
driver.find_element_by_class_name(`dtTextBtn f10`)
The latter was what I saw in the source code, but the f10 blocked selenium.
All I am trying to do is: go to "http://news.google.com" and click on the "Technology" link on the side menu. Here's my code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import unittest, time, re
class GoogleTech(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Firefox()
self.driver.implicitly_wait(30)
self.base_url = "https://www.google.com/"
self.verificationErrors = []
self.accept_next_alert = True
def test_googletech(self):
driver = self.driver
driver.get("http://news.google.com")
#driver.find_element_by_xpath("//div[#id='main-wrapper']/div[#id='main-pane']/div[#class='background']/div[#class='centered']/div[#id='nav-menu-wrapper']/div[#class='browse-sidebar']/ul[#id='anchorman-two-browse-nav']/li[#class='nav-item nv-en_us:tc']/a[contains(text(),'Technology')]").click()
#driver.find_element_by_xpath("//ul[#id='anchorman-two-browse-nav']/li/a[contains(text(),'Technology')]").click()
#driver.find_element_by_xpath("//ul[#id='anchorman-two-browse-nav']/li[#class='nav-item nv-en_us:tc']/a[contains(text(),'Technology')]").click()
#driver.find_element_by_xpath("html/body/div[3]/div[1]/div/div/div[2]/div/ul/li[6]/a").click()
#driver.find_element_by_link_text("Technology").click()
wait = WebDriverWait(driver, 10)
link = wait.until(EC.presence_of_element_located((By.XPATH, "//ul[#id='anchorman-two-browse-nav']/li/a[contains(text(),'Technology')]")))
link.click()
time.sleep(10)
...
I've tried many combinations (all the commented out lines plus more!), but still cannot get it to work. I've check to make sure that the element I am using does exists (using Element Inspector) before entering it into the code. But even the "find_element_by_link_text" is giving me a NoSuchElementException exception. Can someone please tell me what I am doing wrong?
UPDATE: After doing further tests, I now have a better understanding of when this error is occuring for the news.google.com page. So, basically, the side menu, that contains the "Technology" link, is set to scrollable (I believe this error is only happening because the link is in scrollable area). The error is show up if the browser (that the Selenium script opens up when you run it), does not show the "Technology" link ie. You have to maximize the browser or scroll down to see the link. You can test this error out yourself - when the test opens up the browser, quickly resize the browser window so that the "Technology" link is not showing, and the test will fail. If your initial browser window doesn't show "Technology" link when it first opens, it will fail unless you maximize the browser window or scroll down really quickly until the "Technology" link is displayed. The error can take two forms - if you are using the wait.until method, you will get a Timeoutexception. If you are not using the wait.until method, you will see the ElementNotVisibleException. I've tested in both Chrome and Firefox and for both, I am seeing this problem. I am new to Selenium, but I believe this is not normal behavior, can someone please confirm? If this normal, then can someone please tell me how I can make sure my test runs correctly each time?
I think you want to use element_to_be_clickable as your ExpectedCondition.
That means that your driver will keep polling Selenium until it finds that the element is visible and enabled source
Try the below solution
link = wait.until(EC.element_to_be_clickable((By.XPATH, "//ul[#id='anchorman-two-browse-nav']/li/a[contains(text(),'Technology')]")))
link.click()
time.sleep(10)