selenium can't locate the element.
The error info:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"loadAllUpcomingPast"}
my code is:
url_base = 'http://www.christies.com/lotfinder/searchresults.aspx&searchtype=p&action=paging&searchFrom=header&lid=1&entry=&pg=all'
driver = webdriver.Chrome()
driver.get(url_base)
time.sleep(2)
driver.switch_to.frame("signupFrame")
driver.find_element_by_id("close_signup").click()
time.sleep(2)
driver.find_element_by_id("loadAllUpcomingPast").click()
the screenshot
Seems that the loadAllUpcomingPast gets loaded later on. You could try something like this:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException
url_base = 'http://www.christies.com/lotfinder/searchresults.aspx?&searchtype=p&action=paging&searchFrom=header&lid=1&entry=lama&pg=all'
driver = webdriver.Chrome()
driver.get(url_base)
wait = WebDriverWait(driver, 5)
try:
element = wait.until(EC.element_to_be_clickable((By.ID, 'loadAllUpcomingPast')))
print(f'Element found: {element}')
except TimeoutException:
print('could not find loadAllUpcomingPast')
In my console I see:
Element found: <selenium.webdriver.remote.webelement.WebElement (session="c9f1b83164f34001aa0a98581618d45b", element="0.6182317130585246-1")>
This will tell Selenium to wait until that element is clickable. You can adjust the wait time if needed within the WebDriverWait 2nd argument.
Related
Error: NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//td[#class='C($primaryColor) W(51%)']"}
(Session info: chrome=77.0.3865.120)
My Code is below:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome('/Users/ryanyee/Desktop/Python Code/Selenium/Launch
Chrome/chromedriver')
driver.get('https://finance.yahoo.com')
search_box = driver.find_element_by_id('yfin-usr-qry')
search_box.send_keys('GOOG')
search_box.submit()
name = driver.find_element_by_xpath("//td[#class='C($primaryColor) W(51%)']").text()
My problem is that it throws an error when I try to scrape the text for name.
Also, I have trouble trying to click the button "Historical Data"
This is the website I am trying to scrape from 'https://finance.yahoo.com/quote/GOOG?p=GOOG'
Please let me know what I am doing wrong! I have been stuck for days!
Try Adding wait statement after search_box.submit()
from selenium.webdriver.support.ui import WebDriverWait
WebDriverWait(driver, 10).until(
EC.presence_of_element_located(By.XPATH, "//td[#class='C($primaryColor) W(51%)']"))
You need wait after submit.
search_box.submit()
name = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//td[#class='C($primaryColor) W(51%)']"))).text
print(name)
#this is for click Historical Data
driver.find_element_by_link_text("Historical Data").click()
Following import:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I am pretty new in selenium and I was looking forward to execute a click() button that expands a table, and after that I was looking forward to scrape the content of the expanded table.
This is what I tried:
url = 'https://www.telegraph.co.uk/markets-hub/assets/shares/'
phantomjs_path = '/usr/local/bin/phantomjs'
driver = webdriver.PhantomJS(executable_path=phantomjs_path,service_args=['--load-images=no'])
driver.get(url)
element = driver.find_element_by_xpath("//div[#class='tmg-show-more']")
html_source = driver.page_source
element.click()
driver.quit()
soup = BeautifulSoup(html_source,"html5lib")
tables=soup.find('table',class_="table-static kurser-table")
link=pd.read_html(str(tables),flavor='html5lib',thousands='.',decimal=',',header=0)
print(link)
The current output is as it follows:
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: {"errorMessage":"Element is not currently visible and may not be manipulated",
"request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-
Type":"application/json;charset=UTF-8","Host":"127.0.0.1:58447","User-Agent":"Python http auth"},
"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1556893378248\", \"sessionId\": \"f223cb80-6dae-11e9-b00b-4bb158952171\"}",
"url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"",
"userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/f223cb80-6dae-11e9-b00b-4bb158952171/element/:wdc:1556893378248/click"}}
Screenshot: available via screen
You would need to scroll down to let the selenium web driver knows where the element is actually present.
Code :
driver.get("https://www.telegraph.co.uk/markets-hub/assets/shares/")
wait = WebDriverWait(driver, 10)
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'div.section-box.kurser-table-container')))
element = driver.find_element_by_xpath("//a[#ng-click='stocksShowMore()']")
actions = ActionChains(driver)
actions.move_to_element(element).perform()
element.click()
Imports :
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
You are getting ElementNotVisibleException means you are trying to click the element before it is visible.
Also, you are getting the page source before clicking the button. You need to get the source after clicking and loading the new data.
Try the following code before getting the page source.
wait = WebDriverWait(driver, 20)
element = wait.until(EC.element_to_be_clickable(driver.find_element_by_xpath("//div[#class='tmg-show-more']")))
element.click()
wait.until(EC.invisibility_of_element(element))
To use the WebDriverWait you need to import these
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I cannot locate the username and password fields. I inspect the elements, and tried finding it by id, xpath or css selector, but it gives me error NoSuchElementException: Message: no such element: Unable to locate element.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
import time
if __name__ == "__main__":
option = webdriver.ChromeOptions()
option.add_argument("--incognito")
option.add_argument("--start-maximized")
if getattr(sys, 'frozen', False):
chromedriver_path = os.path.join(sys._MEIPASS, "chromedriver.exe")
driver = webdriver.Chrome(chromedriver_path, options=option)
else:
driver = webdriver.Chrome(options=option)
driver.get("https://www.wix.com/")
loginPage = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/header/nav/a[2]"))).click()
usernameField = driver.find_element_by_id("input_4")
passwordField = driver.find_element_by_xpath("input_5")
usernameField.send_keys("user")
passwordField.send_keys("pass")
time.sleep(5)
driver.quit()
The error I get is:
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"input_4"}
(Session info: chrome=71.0.3578.98)
(Driver info: chromedriver=2.44.609538 (b655c5a60b0b544917107a59d4153d4bf78e1b90),platform=Windows NT 10.0.17134 x86_64)
If you want to locate email and password fields you can try
usernameField = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.NAME, "email")))
passwordField = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.NAME, "password")))
usernameField.send_keys("user")
passwordField.send_keys("pass")
The reason why you cannot locate fields:
driver.find_element_by_id("input_4") - I see no elements with id="input_4". #id value might be dynamic.
driver.find_element_by_xpath("input_5") - "input_5" is not valid XPath syntax. You might need to use //*[#id="input_5"], but anyway I also see no elements with id="input_4"...
The elements i.e. username and password fields are Angular elements and becomes visible, enabled and clickable once you invoke click() on the element with text as Sign In and JavaScript / AjaxCalls are completed. So you have to induce WebDriverWait for the desired element to be clickable and you can use the following solution:
Minimal Code Block:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_argument("--disable-extensions")
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://www.wix.com/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "Sign In"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='email']"))).send_keys("paul")
driver.find_element_by_css_selector("input[name='password']").send_keys("paul")
Browser Snapshot:
Note: You can find a relevant detailed discussion in Selenium “selenium.common.exceptions.NoSuchElementException” when using Chrome
I have an "onclick" element in a webpage whose HTML reads as follows:
Text 1
I am trying to use selenium to click on this element. I tried the following code (including an implicit wait)
import time
from selenium import webdriver
driver = webdriver.Chrome()
driver.get(webpage);
driver.implicitly_wait(5)
driver.find_element_by_xpath("//a[#onclick='blah.submit()']").click()
However, it returned a NoSuchElementException
NoSuchElementException: Message: no such element: Unable to locate element:
{"method":"xpath","selector":"//a[#onclick='blah.submit()']"}
I then tried using WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get(webpage);
button = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH,
"//a[#onclick='blah.submit()']")))
button.click()
However, this still didn't work and returned the following error:
TimeoutException: Message:
Does anyone know what's going wrong? Thanks in advance.
Why does this piece of code throw an exception selenium.common.exceptions.WebDriverException: Message: unknown error: cannot focus element?
As far as I can tell, I'm choosing the right element. Googling suggested to have a .click() on element before sending keys but that didn't help either.
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get("https://netbanking.hdfcbank.com/netbanking/")
login_wait = WebDriverWait(driver, 10)
assert "Welcome to HDFC Bank" in driver.title
frame = login_wait.until(EC.presence_of_element_located((By.NAME, 'login_page')))
driver.switch_to.frame(frame)
try:
elem = login_wait.until(EC.presence_of_element_located((By.CLASS_NAME, 'pwd_field')))
print("Page is ready!")
elem.send_keys("123456")
elem.send_keys(Keys.RETURN)
except TimeoutException:
print("Loading took too much time!")
driver.close()
This is because what element you've located by the pwd_field class name - you've actually got a span element matching the locator. Instead, you meant to get to the password input element:
elem = login_wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'input_password')))