I'm studying how Selenium works. There are some elements that are fully loaded, but can't be clickable. Here's the example of the case. Selenium took a time until it is fully loaded. But Still the button is not clickable. I think Selenium can't find the element. How can I resolve this issue?
FYI, a pop up is loaded when I click the button manually. Thank you very much for your assistance
url
https://cafe.naver.com/codeuniv
my python code
from selenium import webdriver
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get("https://cafe.naver.com/codeuniv")
time.sleep(1)
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="menuLink107"]'))).click()
time.sleep(1)
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="upperArticleList"]/table/tbody/tr[1]/td[2]/div/table/tbody/tr/td/a'))).click()
button
코뮤
ERROR
Traceback (most recent call last):
File "C:\project\naver.py", line 12, in <module>
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="upperArticleList"]/table/tbody/tr[1]/td[2]/div/table/tbody/tr/td/a'))).click()
File "C:\project\venv\lib\site-packages\selenium\webdriver\support\wait.py", line 89, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
You can use implicity wait that solve your problem or try to id id or other xpath Or try to find iframe
wait=WebDriverWait(driver,10)
driver.get("https://cafe.naver.com/codeuniv")
wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="menuLink107"]'))).click()
wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID,'cafe_main')))
wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="upperArticleList"]/table/tbody/tr[1]/td[2]/div/table/tbody/tr/td/a'))).click()
Your element is in an iframe please wait and switch to it.
<iframe name="cafe_main" id="cafe_main" title="카페 메인" src="//cafe.naver.com/MyCafeIntro.nhn?clubid=30026525" width="860" height="100%" frameborder="0" scrolling="no" marginwidth="0" marginheight="0" allowtransparency="true" allowfullscreen="" style="height: 2140px;" cd_frame_id_="3e06a515dca2ac262583e8ce3ea0e356"></iframe>
Imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Related
I am trying to find an element with xpath but it is not working.
import string
import random
import os
from random_word import RandomWords
import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
driver.get("https://yopmail.com/en/")
driver.find_element(By.NAME, 'login').send_keys('grudsimcobottest')
driver.find_element(By.NAME, 'login').submit()
try:
elem = WebDriverWait(driver, 60).until(
EC.presence_of_element_located((By.XPATH, '//div[text()="Confirm account deletion"]'))
)
finally:
a = 1
confirm_email = driver.find_element(By.XPATH, '//div[text()="Confirm account deletion"]')
confirm_email.click
try:
elem = WebDriverWait(driver, 60).until(
EC.presence_of_element_located((By.XPATH, '//a[text()="Delete"]'))
)
finally:
a = 1
delete_button_in_email = driver.find_element(By.XPATH, '//a[text()="Delete"]')
delete_button_in_email.click()
This is the code^
Traceback (most recent call last):
File "C:\Users\notgr\Desktop\Everything\SC ahks\python\yopmailmakendelete.py", line 28, in <module>
elem = WebDriverWait(driver, 60).until(
File "C:\Users\notgr\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\support\wait.py", line 90, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
This is the error^
the website
If you go to yopmail.com and type in grudsimcobottest you can see it live if you like. I'm not sure why the div highlighted isn't being selected and clicked, it times out when looking for it. Help much appreciated.
<iframe frameborder="0" width="100%" height="100%" src="" name="ifinbox" onload="onloadifinbox()" id="ifinbox" state="full"></iframe>
Your element is in an iframe switch to it prior to finding the element.
WebDriverWait(driver, 60).until(EC.frame_to_be_available_and_switch_to_it((By.NAME,"ifinbox")))
This is my first time using Python and Selenium. The first part of the code works but when it goes to the second page it can never find any of the elements. If I flip the code and make it go to the second site first, it works. What am I doing wrong here? I tried xpath, CSS_Selector, Class_Name seems like nothing is working. This is the error I get:
Traceback (most recent call last):
File "C:\Users\dresd\PycharmProjects\Test2\main.py", line 20, in
click_Register = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.CLASS_NAME, "q-text qu-ellipsis qu-whiteSpace--nowrap"))).click()
File "C:\Users\dresd\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\support\wait.py", line 89, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.action_chains import ActionChains
import time
driver = webdriver.Chrome ("C:/chromedriver.exe")
driver.get("https://10minutesemail.net/")
Copy_Email = WebDriverWait(driver, 20).until(
EC.presence_of_element_located((By.ID, "copyEmailAddress"))).click()
time.sleep(10)
driver.execute_script("window.open('https://quora.com/','_blank')")
click_Register = WebDriverWait(driver, 20).until(
EC.presence_of_element_located((By.CLASS_NAME, "q-text qu-ellipsis qu-whiteSpace--nowrap"))).click()
name = driver.find_element(By.NAME, "profile-name")
email = driver.find_element(By.ID, "email")
name.send_keys("Jackson Fuller")
Thanks in advance!
You have to switch the driver to the new opened tab.
Without that the focus will remain on the first browser window.
TimeoutException actually means that Selenium could not locate element by passed locator.
Also the locator you are using is bad.
Try this:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.action_chains import ActionChains
import time
driver = webdriver.Chrome ("C:/chromedriver.exe")
driver.get("https://10minutesemail.net/")
Copy_Email = WebDriverWait(driver, 20).until(
EC.presence_of_element_located((By.ID, "copyEmailAddress"))).click()
time.sleep(10)
driver.execute_script("window.open('https://quora.com/','_blank')")
driver.switch_to.window(driver.window_handles[1])
click_Register = WebDriverWait(driver, 20).until(
EC.presence_of_element_located((By.CLASS_NAME, "q-text qu-ellipsis qu-whiteSpace--nowrap"))).click()
name = driver.find_element(By.NAME, "profile-name")
email = driver.find_element(By.ID, "email")
name.send_keys("Jackson Fuller")
I am new to web scraping. For a little project i decided to try and log into instagram automatically. I am using chrome web driver to open the page and selenium to process it. i can get selenium to open Instagram but on instagram there is a cookie popup that i can't seem to automate a click for. I keep getting a error. what i want to do is just to pick the accept all.
my code is:
#Selenium imports here
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
#Other imports here
import os
#import wget
driver = webdriver.Chrome("/Users/kitchensink/Desktop/instab/chromedriver")
driver.get("https://www.instagram.com")
Accept_all = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(text(), 'Accept All)]"))).click()
There error that i get is:
Traceback (most recent call last):
File "/Users/kitchensink/PycharmProjects/instapost/main3.py", line 15, in <module>
Accept_all = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(text(), 'Accept All)]"))).click()
File "/Users/kitchensink/.local/share/virtualenvs/kitchensink-UJdjJQi/lib/python3.8/site-packages/selenium/webdriver/support/wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
See you have a issue with xpath here : (Your xpath does not contain ' at the end) you are using this
//button[contains(text(), 'Accept All)]
but one should use this instead :
//button[contains(text(), 'Accept All')]
I would do this as an xpath: /html[#class='js not-logged-in client-root js-focus-visible sDN5V']/body/div[#class='RnEpo Yx5HN _4Yzd2']/div[#class='pbNvD FrS-d gD9tr ']/div[#class='_1XyCr ']/button[#class='aOOlW bIiDR ']
So your code would look like:
#Selenium imports here
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
#Other imports here
import os
#import wget
driver = webdriver.Chrome("./chromedriver.exe")
driver.get("https://www.instagram.com")
Accept_all = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "/html[#class='js not-logged-in client-root js-focus-visible sDN5V']/body/div[#class='RnEpo Yx5HN _4Yzd2']/div[#class='pbNvD FrS-d gD9tr ']/div[#class='_1XyCr ']/button[#class='aOOlW bIiDR ']"))).click()
EDIT: I tried it and it works fine for me
First post on here and brand new to Python. I am trying to learn how to scrape data from a website. When you first load the website, a disclaimer window shows up and all I am trying to do is hit the accept button using the browser.find_element_by_id.
I am using the webdriverwait command to wait for the page to load prior to clicking the "Accept" button but I keep getting a Timeoutexception. Here is the code that I currently have:
from selenium import webdriver
#get the chrome webdriver path file
browser = webdriver.Chrome(executable_path=r"C:/Program Files (x86)/Google/Chrome/Application/chromedriver.exe")
browser.get('http://foreclosures.guilfordcountync.gov/')
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
#wait until element is loaded
wait = WebDriverWait(browser, 10)
wait.until(EC.presence_of_element_located((By.ID, "cmdAccept")))
element = browser.find_element_by_id("cmdAccept")
element.click()
Here is the error I keep getting:
Traceback (most recent call last):
File "C:/Users/Abbas/Desktop/Foreclosure_Scraping/Foreclosure_Scraping.py", line 33, in <module>
wait.until(EC.presence_of_element_located((By.ID, "cmdAccept")))
File "C:\Users\Abbas\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
I believe it has something to do with the calling out the ID of the button itself from the website but I honestly do not know. Any help is greatly appreciated.
Your attempts to locate the element are unsuccessful because of they are nested within an iframe. One must tell selenium to switch to the iframe that contains the desired element before attempting to click it or use it in any way. Try the following:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
#get the chrome webdriver path file
browser = webdriver.Chrome(executable_path=r"C:/Program Files (x86)/Google/Chrome/Application/chromedriver.exe")
browser.get('http://foreclosures.guilfordcountync.gov/')
browser.switch_to.frame(browser.find_element_by_name("ctl06"))
wait = WebDriverWait(browser, 10)
wait.until(EC.presence_of_element_located((By.ID, "cmdAccept")))
element = browser.find_element_by_id("cmdAccept")
element.click()
For automating downloading from a website, I am using this python script for automating click action. But the click does not work.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import time
browser = webdriver.Firefox()
browser.maximize_window()
browser.implicitly_wait(20)
browser.get('http://download.cnet.com/most-popular/windows/')
linkElem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "CCleaner")))
linkElem.click() # follows the "CCleaner" link
I first got this error:
selenium.common.exceptions.ElementNotInteractableException: Message: Element could not be scrolled into view
Then I add this line:
linkElem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "CCleaner")))
Now I get this error:
linkElem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "CCleaner")))
File "/usr/lib/python2.7/site-packages/selenium/webdriver/support/wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
I tried the things that people used to solve this problem but did not work.
Your code seems to work with Chrome, I only added the bottom two lines and it downloaded CCleaner.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import time
browser = webdriver.Chrome() # Firefox()
browser.maximize_window()
browser.implicitly_wait(20)
browser.get('http://download.cnet.com/most-popular/windows/')
linkElem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "CCleaner")))
linkElem.click() # follows the "CCleaner" link
download = browser.find_element_by_id('download-now-btn-text')
download.click()