When I run the code, it opens the browser, but I want to click the reject all button when the cookie banner comes from YouTube. I tried using class and a link text. It did not work.
I will appreciate any help.
from selenium import webdriver
from selenium.webdriver.common.by import By
url = 'https://www.youtube.com/'
driver = webdriver.Chrome(executable_path="C:\\Users\\donner\\Downloads\\chromedriver_win32\\chromedriver.exe")
driver.get(url)
driver.implicitly_wait(100)
continue_link = driver.find_element(By.LINK_TEXT, "Reject all")
continue_link.click()
driver.implicitly_wait(100)
content = driver.find_element(By.CLASS_NAME, '.style-scope.ytd-button-renderer.style-primary size-default')
content.click()
Try the below xpath with the given code
//*[normalize-space()='Reject all']
OR
//*[contains(text(),'Reject all')]
The Code:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "myXpath")))
element.click();
The above soultion work if there is no iframe
In case there is iframe, than you need to follow below
WebDriverWait(driver,20).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"iframe")))
driver.find_element(By.XPATH, "yourXpath").click()
Related
I am struggling to make selenium click on the cookie popup and access the website.
I have the following code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import pandas as pd
from datetime import datetime
import os
import sys
web = 'https://www.thesun.co.uk/sport/football/'
path = '/Users/cc/Documents/Documents/IT/1. Python/WebCrawler/chromedriver'
driver_service = Service(executable_path=path)
driver = webdriver.Chrome(service=driver_service)
driver.get(web)
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH,'//*.[#id="notice"]/div[4]/button[2]'))).click()
here is the webpage:
enter image description here
here is the htlm button element:
enter image description here
here is the vs code terminal after running:
enter image description here
The webpage open but the click command does not do the work...even when defining a wait until element is clickable.
I tried to use the click command with wait to be sure the element is clickable.
the idea is to click on the "Fine By Me!" button.
Thanks for your help.
You need to switch to the iframe and then click on the button. Also there is a browser pop-up for allowing/blocking notifications. You need to handle that in the driver setup.
Driver Setup can be done with params
chromeOpts = Options()
chromeOpts.add_argument('start-maximized')
chromeOpts.add_argument("--disable-notifications")
chromeOpts.add_argument("--disable-infobars")
chromeOpts.add_argument("--disable-extensions")
The code to switch to iframe and then clikcing on button is -
iframe = "//iframe[#title='SP Consent Message']"
ifr = driver.find_element(By.XPATH, iframe)
driver.switch_to.frame(ifr)
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH,'//*[#id="notice"]/div[4]/button[2]'))).click()
Here -
To click on the element Fine By Me! you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[title='Fine By Me!'][aria-label='Fine By Me!']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#title='Fine By Me!' and #aria-label='Fine By Me!']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
enter image description here
I want to make a web scraper for the news title on the news website, news.sky.com
The problem is, very often, the site will pop up a message to let me "accept all cookies" (see capture...)
I followed the guide on the similar question by using XPath. But, it feedbacks:
raise TimeoutException(message, screen, stacktrace)
Seemly, selenium can't find out the location based on XPath.
so, is it possible to click "Accept all"?
please help, thx.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
from selenium.webdriver.edge.service import Service
from selenium.webdriver.common.by import By
e_driver_path = r"F:/Download/Portable Test/msedgedriver.exe"
# establish the web driver
s = Service(e_driver_path)
driver = webdriver.Edge(service=s)
driver.get("https://news.sky.com/uk")
# search = driver.find_element_by_id(ContentPlaceHolder1_NotifyBtn)
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//*[#id='notice']/div[3]/button[1]"))).click()
print("yes")
The cookie buttons are in an iframe on that page. Here is one way of clicking that button:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
import time as t
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument('disable-notifications')
chrome_options.add_argument("window-size=1280,720")
webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
driver = webdriver.Chrome(service=webdriver_service, options=chrome_options)
actions = ActionChains(driver)
wait = WebDriverWait(driver, 20)
url = "https://news.sky.com/uk"
driver.get(url)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe[#title='SP Consent Message']")))
try:
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[title='Accept all']"))).click()
print('accepted cookies')
except Exception as e:
print('no cookie button!')
driver.switch_to.default_content()
print('back to main content')
This will access the iframe, click the button, and then exit the iframe back to main content, and also print in terminal:
accepted cookies
back to main content
Selenium setup is chrome/chromedriver/linux, you just need to observe the import and the code after defining the driver, to adapt it to your own setup. Selenium docs can be found here: https://www.selenium.dev/documentation/
I'm using selenium 4 with python to automate webpage.
Click() method is not working.
Able to click a button using submit() method only if the path has type=submit, but couldnt able to select a link using click method. Please help if anyone knows alternative option for click() method to click a link.
The following is the code:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
s = Service("C:/Program Files/webdriver/Chrome Driver/chromedriver.exe")
driver = webdriver.Chrome(service=s)
driver.get("https://www.sololearn.com/users/login")
time.sleep(2)
#driver.find_element(By.CLASS_NAME, "sl-login-login-form").send_keys(Keys.ENTER)
#---click here fails--------
driver.find_element(By.CLASS_NAME, "sl-login-login-form").click()
#---click here fails
#driver.find_element(By.CLASS_NAME, "sl-login-login-form").submit()
#login.click()
time.sleep(5)
driver.quit()
**Also i have tried with the below code but still it is not working:**
driver.find_element(By.CLASS_NAME, "sl-login-login-form").send_keys(Keys.ENTER)
driver.find_element(By.CLASS_NAME, "sl-login-login-form").submit()
login.click()
Try the below
element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CLASS_NAME, "CLASSNAME")))
element.click();
OR
element = driver.find_element(By.CLASS_NAME, "className")
driver.execute_script("arguments[0].click();", element)
Do not forget to import
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import org.openqa.selenium.JavascriptExecutor
I am trying to scrape some google data but I first want to click the 'I agree' button that google pops up. This is the script I use to do that:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
search_question = input("Ask a question: ")
driver = webdriver.Chrome("*Your Webdriver location*")
driver.wait = WebDriverWait(driver, 5)
driver.get("https://google.com")
time.sleep(1)
agree = driver.wait.until(EC.presence_of_element_located((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
# time.sleep(0.2)
search = driver.find_element_by_class_name("gLFyf")
search.send_keys(search_question)
search.send_keys(Keys.ENTER)
The problem is selenium doesn't seem to locate the button and therefore I get a timeout error. (I have tried also with find_element_by_xpath and still not working).
If you scroll up in the devtools inspector you'll notice that your element is within an iframe:
You need to switch to that frame first, click your button then switch back to the default content (the main page)
driver.get("https://google.com")
#active the iframe and click the agree button
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe")))
agree = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
#back to the main page
driver.switch_to_default_content()
That works for me.
FYI - There's only 1 iframe on the page, that's why the xpath //iframe works. If there were multiple you'd need to identify it with higher accuracy.
If you already have been agreed,then agree button would not appear. That why it is not be able to find given XPath.
Try this:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
search_question = input("Ask a question: ")
driver = webdriver.Chrome(".\chromedriver.exe")
driver.wait = WebDriverWait(driver, 5)
driver.get("https://google.com")
time.sleep(3)
# agree = driver.wait.until(EC.presence_of_element_located((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
# agree.click()
# time.sleep(0.2)
search = driver.find_element_by_class_name("gLFyf")
search.send_keys(search_question)
search.send_keys(Keys.ENTER)
Managed to get it working.
Seems like one of the few elements that are interactive in the popup when you first load the site is the language dropdown, which I found by class name.
Then the Agree button is detectable and you can find it by class too.
driver.get("https://www.google.com")
driver.maximize_window()
time.sleep(2)
#finds+clicks language dropdown, interaction unhides the rest of the popup html
driver.find_element_by_class_name('tHlp8d').click()
time.sleep(2)
#finds+clicks the agree button now that it has become visible
driver.find_element_by_id('L2AGLb').click()
you should now have the standard searchbar etc
i had some problems with clicking pop-ups and the problem turned out to be with that click().
im not sure that it might be the same issue with urs or not but try changing ur click to this :
agree = driver.find_element_by_xpath('//*[#id="introAgreeButton"]/span/span')
driver.execute_script("arguments[0].click();", agree)
The problem was that it didn't change frames here is the soulution code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('C:\\Users\\gassp\\OneDrive\\Namizje\\Python.projects\\chromedriver.exe')
url = 'https://www.google.com/maps/'
driver.get(url)
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, '//*[#id="consent-bump"]/div/div[1]/iframe')))
agree = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
#back to the main page
driver.switch_to_default_content()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="searchboxinput"]'))).send_keys('gostilne')
driver.submit()
I'm trying to close a popup iframe to do web scraping in aliexpress, but i can't access the image to click it
link: https://www.aliexpress.com/item/4000094534783.html?spm=a2g0o.productlist.0.0.4e024c1e5Hpt1O&algo_pvid=501da7fc-b3f5-46a9-bbbc-75c7bed32f81&algo_expid=501da7fc-b3f5-46a9-bbbc-75c7bed32f81-20&btsid=0b0a555716045883196526344e54cc&ws_ab_test=searchweb0_0,searchweb201602_,searchweb201603_
Where i want to click
This is my code:
import os
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
option = Options()
option.set_headless(True)
profile = webdriver.FirefoxProfile()
profile.set_preference('intl.accept_languages', 'fr')
def webscraping_aliexpress():
driver = webdriver.Firefox(executable_path=f"{os.getcwd()}\geckodriver.exe", firefox_profile=profile)
driver.get(str(input("Paste aliexpress url: ")))
name = driver.find_element_by_xpath("//h1[#class='product-title-text']").text
price = driver.find_element_by_xpath("//div[#class='product-price-current']").text
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe[#data-spm-anchor-id='a2g0o.detail.0.i4.4bc257b6555T2w'")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//img[#class='rax-image ']"))).click()
webscraping_aliexpress()
I can extract some data, like name and price before popup shows on, but he block other operations after.
You need switch content to iframe
iframe = driver.find_element_by_xpath('//iframe[contains(#src, "__poplayer")]')
driver.switch_to.frame(iframe)
# code of closing popup here
driver.switch_to_default_content()