I have an "onclick" element in a webpage whose HTML reads as follows:
Text 1
I am trying to use selenium to click on this element. I tried the following code (including an implicit wait)
import time
from selenium import webdriver
driver = webdriver.Chrome()
driver.get(webpage);
driver.implicitly_wait(5)
driver.find_element_by_xpath("//a[#onclick='blah.submit()']").click()
However, it returned a NoSuchElementException
NoSuchElementException: Message: no such element: Unable to locate element:
{"method":"xpath","selector":"//a[#onclick='blah.submit()']"}
I then tried using WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get(webpage);
button = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH,
"//a[#onclick='blah.submit()']")))
button.click()
However, this still didn't work and returned the following error:
TimeoutException: Message:
Does anyone know what's going wrong? Thanks in advance.
Related
Error: NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//td[#class='C($primaryColor) W(51%)']"}
(Session info: chrome=77.0.3865.120)
My Code is below:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome('/Users/ryanyee/Desktop/Python Code/Selenium/Launch
Chrome/chromedriver')
driver.get('https://finance.yahoo.com')
search_box = driver.find_element_by_id('yfin-usr-qry')
search_box.send_keys('GOOG')
search_box.submit()
name = driver.find_element_by_xpath("//td[#class='C($primaryColor) W(51%)']").text()
My problem is that it throws an error when I try to scrape the text for name.
Also, I have trouble trying to click the button "Historical Data"
This is the website I am trying to scrape from 'https://finance.yahoo.com/quote/GOOG?p=GOOG'
Please let me know what I am doing wrong! I have been stuck for days!
Try Adding wait statement after search_box.submit()
from selenium.webdriver.support.ui import WebDriverWait
WebDriverWait(driver, 10).until(
EC.presence_of_element_located(By.XPATH, "//td[#class='C($primaryColor) W(51%)']"))
You need wait after submit.
search_box.submit()
name = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//td[#class='C($primaryColor) W(51%)']"))).text
print(name)
#this is for click Historical Data
driver.find_element_by_link_text("Historical Data").click()
Following import:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I am pretty new in selenium and I was looking forward to execute a click() button that expands a table, and after that I was looking forward to scrape the content of the expanded table.
This is what I tried:
url = 'https://www.telegraph.co.uk/markets-hub/assets/shares/'
phantomjs_path = '/usr/local/bin/phantomjs'
driver = webdriver.PhantomJS(executable_path=phantomjs_path,service_args=['--load-images=no'])
driver.get(url)
element = driver.find_element_by_xpath("//div[#class='tmg-show-more']")
html_source = driver.page_source
element.click()
driver.quit()
soup = BeautifulSoup(html_source,"html5lib")
tables=soup.find('table',class_="table-static kurser-table")
link=pd.read_html(str(tables),flavor='html5lib',thousands='.',decimal=',',header=0)
print(link)
The current output is as it follows:
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: {"errorMessage":"Element is not currently visible and may not be manipulated",
"request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-
Type":"application/json;charset=UTF-8","Host":"127.0.0.1:58447","User-Agent":"Python http auth"},
"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1556893378248\", \"sessionId\": \"f223cb80-6dae-11e9-b00b-4bb158952171\"}",
"url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"",
"userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/f223cb80-6dae-11e9-b00b-4bb158952171/element/:wdc:1556893378248/click"}}
Screenshot: available via screen
You would need to scroll down to let the selenium web driver knows where the element is actually present.
Code :
driver.get("https://www.telegraph.co.uk/markets-hub/assets/shares/")
wait = WebDriverWait(driver, 10)
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'div.section-box.kurser-table-container')))
element = driver.find_element_by_xpath("//a[#ng-click='stocksShowMore()']")
actions = ActionChains(driver)
actions.move_to_element(element).perform()
element.click()
Imports :
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
You are getting ElementNotVisibleException means you are trying to click the element before it is visible.
Also, you are getting the page source before clicking the button. You need to get the source after clicking and loading the new data.
Try the following code before getting the page source.
wait = WebDriverWait(driver, 20)
element = wait.until(EC.element_to_be_clickable(driver.find_element_by_xpath("//div[#class='tmg-show-more']")))
element.click()
wait.until(EC.invisibility_of_element(element))
To use the WebDriverWait you need to import these
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
For automating downloading from a website, I am using this python script for automating click action. But the click does not work.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import time
browser = webdriver.Firefox()
browser.maximize_window()
browser.implicitly_wait(20)
browser.get('http://download.cnet.com/most-popular/windows/')
linkElem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "CCleaner")))
linkElem.click() # follows the "CCleaner" link
I first got this error:
selenium.common.exceptions.ElementNotInteractableException: Message: Element could not be scrolled into view
Then I add this line:
linkElem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "CCleaner")))
Now I get this error:
linkElem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "CCleaner")))
File "/usr/lib/python2.7/site-packages/selenium/webdriver/support/wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
I tried the things that people used to solve this problem but did not work.
Your code seems to work with Chrome, I only added the bottom two lines and it downloaded CCleaner.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import time
browser = webdriver.Chrome() # Firefox()
browser.maximize_window()
browser.implicitly_wait(20)
browser.get('http://download.cnet.com/most-popular/windows/')
linkElem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "CCleaner")))
linkElem.click() # follows the "CCleaner" link
download = browser.find_element_by_id('download-now-btn-text')
download.click()
selenium can't locate the element.
The error info:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"loadAllUpcomingPast"}
my code is:
url_base = 'http://www.christies.com/lotfinder/searchresults.aspx&searchtype=p&action=paging&searchFrom=header&lid=1&entry=&pg=all'
driver = webdriver.Chrome()
driver.get(url_base)
time.sleep(2)
driver.switch_to.frame("signupFrame")
driver.find_element_by_id("close_signup").click()
time.sleep(2)
driver.find_element_by_id("loadAllUpcomingPast").click()
the screenshot
Seems that the loadAllUpcomingPast gets loaded later on. You could try something like this:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException
url_base = 'http://www.christies.com/lotfinder/searchresults.aspx?&searchtype=p&action=paging&searchFrom=header&lid=1&entry=lama&pg=all'
driver = webdriver.Chrome()
driver.get(url_base)
wait = WebDriverWait(driver, 5)
try:
element = wait.until(EC.element_to_be_clickable((By.ID, 'loadAllUpcomingPast')))
print(f'Element found: {element}')
except TimeoutException:
print('could not find loadAllUpcomingPast')
In my console I see:
Element found: <selenium.webdriver.remote.webelement.WebElement (session="c9f1b83164f34001aa0a98581618d45b", element="0.6182317130585246-1")>
This will tell Selenium to wait until that element is clickable. You can adjust the wait time if needed within the WebDriverWait 2nd argument.
Why does this piece of code throw an exception selenium.common.exceptions.WebDriverException: Message: unknown error: cannot focus element?
As far as I can tell, I'm choosing the right element. Googling suggested to have a .click() on element before sending keys but that didn't help either.
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get("https://netbanking.hdfcbank.com/netbanking/")
login_wait = WebDriverWait(driver, 10)
assert "Welcome to HDFC Bank" in driver.title
frame = login_wait.until(EC.presence_of_element_located((By.NAME, 'login_page')))
driver.switch_to.frame(frame)
try:
elem = login_wait.until(EC.presence_of_element_located((By.CLASS_NAME, 'pwd_field')))
print("Page is ready!")
elem.send_keys("123456")
elem.send_keys(Keys.RETURN)
except TimeoutException:
print("Loading took too much time!")
driver.close()
This is because what element you've located by the pwd_field class name - you've actually got a span element matching the locator. Instead, you meant to get to the password input element:
elem = login_wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'input_password')))