We Keep Coding

Selenium, how to locate and click a particular button

I'm using selenium to try and scrape a listing of products in this website:
https://www.zonacriativa.com.br/harry-potter
However, I'm having trouble getting the full listing of products. the page list 116 products, yet only a few are shown at a time. If I want to see the other ones, I need to click on the "Carregar mais Produtos" (load more products) button at the bottom a few times to get the full listing.
I'm having trouble locating this button, as it doesn't have an id and its class is a huge string. I've tried several things, like the examples below, but they don't seem to work. Any suggestions?
driver.find_element("xpath", "//button[text()='Carregar mais Produtos']").click()
driver.find_element("css selector", ".vtex-button__label.flex.items-center.justify-center.h-100.ph5").click()
driver.find_element(By.CLASS_NAME, "vtex-button.bw1.ba.fw5.v-mid.relative.pa0.lh-solid.br2.min-h-small.t-action--small.bg-action-primary.b--action-primary.c-on-action-primary.hover-bg-action-primary.hover-b--action-primary.hover-c-on-action-primary.pointer").click()

The element you trying to click is initially out of the visible screen so you can't click it. Also this XPath at least for me doesn't locate that element.
What you need to do is to scroll the page down untill that button becomes visible and clickable and then click it.
The following code clicks that button 1 time:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 5)
url = "https://www.zonacriativa.com.br/harry-potter"
driver.get(url)
while True:
try:
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[contains(#class,'buttonShowMore')]//button"))).click()
break
except:
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
The above code can be simply modified to scroll and click that button until we reach the latest page where this button is not presented:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 5)
url = "https://www.zonacriativa.com.br/harry-potter"
driver.get(url)
while driver.find_elements(By.XPATH, "//div[contains(#class,'buttonShowMore')]//button"):
try:
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[contains(#class,'buttonShowMore')]//button"))).click()
except:
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

How can I use selenium to click accept all cookies on the website "sky news"

enter image description here
I want to make a web scraper for the news title on the news website, news.sky.com
The problem is, very often, the site will pop up a message to let me "accept all cookies" (see capture...)
I followed the guide on the similar question by using XPath. But, it feedbacks:
raise TimeoutException(message, screen, stacktrace)
Seemly, selenium can't find out the location based on XPath.
so, is it possible to click "Accept all"?
please help, thx.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
from selenium.webdriver.edge.service import Service
from selenium.webdriver.common.by import By
e_driver_path = r"F:/Download/Portable Test/msedgedriver.exe"
# establish the web driver
s = Service(e_driver_path)
driver = webdriver.Edge(service=s)
driver.get("https://news.sky.com/uk")
# search = driver.find_element_by_id(ContentPlaceHolder1_NotifyBtn)
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//*[#id='notice']/div[3]/button[1]"))).click()
print("yes")

The cookie buttons are in an iframe on that page. Here is one way of clicking that button:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
import time as t
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument('disable-notifications')
chrome_options.add_argument("window-size=1280,720")
webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
driver = webdriver.Chrome(service=webdriver_service, options=chrome_options)
actions = ActionChains(driver)
wait = WebDriverWait(driver, 20)
url = "https://news.sky.com/uk"
driver.get(url)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe[#title='SP Consent Message']")))
try:
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[title='Accept all']"))).click()
print('accepted cookies')
except Exception as e:
print('no cookie button!')
driver.switch_to.default_content()
print('back to main content')
This will access the iframe, click the button, and then exit the iframe back to main content, and also print in terminal:
accepted cookies
back to main content
Selenium setup is chrome/chromedriver/linux, you just need to observe the import and the code after defining the driver, to adapt it to your own setup. Selenium docs can be found here: https://www.selenium.dev/documentation/

How to make selenium click on a radio button

I want to use Selenium in Python 3.9 to click on a button.
https://www.sepaq.com/en/reservation/national-parks/annual-card
I want to click on Effective Date and click on July 2021 then click ok.
The first two was working, then I added the code to click the OK button and I can no longer click on July 2021. Interesting the click OK step also works.
Error I am getting this error:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
(Session info: chrome=90.0.4430.212)
I have tried:
sleep statement to make it wait for the button to become available
Web Driver wait
Action Chain to make it scroll into view
I am using Selenium IDE with chrome to give me the css tags.
from time import sleep
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.remote.webelement import WebElement
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
def print_hi():
chrome_options = Options()
chrome_options.add_experimental_option("detach", True)
chrome_options.add_argument("start-maximized")
chrome_options.add_argument("disable-infobars")
chrome_options.add_argument("--disable-extensions")
annual_pass = webdriver.Chrome(options=chrome_options)
annual_pass.get("https://www.sepaq.com/en/reservation/national-parks/annual-card")
type = annual_pass.find_element_by_css_selector('.form-list:nth-child(2) > li:nth-child(1) .form-label')
type.click()
annual_pass.get("https://www.sepaq.com/en/reservation/national-parks/annual-card")
open_eff_date_panel = annual_pass.find_element_by_link_text('Select')
open_eff_date_panel.click()
annual_pass.get("https://www.sepaq.com/en/reservation/national-parks/annual-card")
selectJuly = annual_pass.find_element_by_css_selector('li:nth-child(3) .form-label')
#selectJuly = annual_pass.find_element_by_xpath('//*[#id="date2"]')
selectJuly.click() ## error here##
annual_pass.get("https://www.sepaq.com/en/reservation/national-parks/annual-card")
ok_button = annual_pass.find_element_by_css_selector('#reserver-date .bouton')
ok_button.click()
sleep(10)
annual_pass.close()
if __name__ == '__main__':
print_hi()

Clicking toggles may be unstable with Selenium.
First, try this approach:
selectJuly = annual_pass.find_element_by_css_selector('#reserver .panel.is-active>.panel-middle li:nth-child(3) .form-label')
annual_pass.execute_script("arguments[0].click();", selectJuly)
ok_button = annual_pass.find_element_by_css_selector('#reserver-date .bouton')
ok_button.click()
It works, but I do not like that the window is not completely rolled out.
Full code with debugging info left:
import time
from time import sleep
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.remote.webelement import WebElement
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
def print_hi():
chrome_options = Options()
chrome_options.add_experimental_option("detach", True)
chrome_options.add_argument("start-maximized")
chrome_options.add_argument("disable-infobars")
chrome_options.add_argument("--disable-extensions")
annual_pass = webdriver.Chrome(executable_path='/snap/bin/chromium.chromedriver', options=chrome_options)
annual_pass.get("https://www.sepaq.com/en/reservation/national-parks/annual-card")
open_eff_date_panel = annual_pass.find_element_by_link_text('Select')
open_eff_date_panel.click()
# wait = WebDriverWait(open_eff_date_panel, 10)
# wait.until(EC.visibility_of_element_located(
# (By.CSS_SELECTOR, "//html[#class='js history es5array video opacity csspointerevents placeholder inlinesvg es5date supports es5function es5object strictmode es5string json es5syntax es5undefined es5 no-touchevents cssvhunit mediaqueries vibrate csscolumns csscolumns-width csscolumns-span csscolumns-fill csscolumns-gap csscolumns-rule csscolumns-rulecolor csscolumns-rulestyle csscolumns-rulewidth csscolumns-breakbefore csscolumns-breakafter csscolumns-breakinside flexbox csstransforms csstransforms3d no-ios panel-active']")))
selectJuly = annual_pass.find_element_by_css_selector(
'#reserver .panel.is-active>.panel-middle li:nth-child(3) .form-label')
annual_pass.execute_script("arguments[0].click();", selectJuly)
ok_button = annual_pass.find_element_by_css_selector('#reserver-date .bouton')
ok_button.click()
# annual_pass.close()
if __name__ == '__main__':
print_hi()
2 Also, you can try implementing your own function to wait till some css property of your form is changed (for example width). For this use Selenium's value_of_css_property.
3
If you wait for all classes to become available in xpath, it will also work, but does not look robust. You can change this locator into css and exclude classes one by one to make it shorter. Waiting just for panel-active class won't work:
wait = WebDriverWait(open_eff_date_panel, 10)
wait.until(EC.visibility_of_element_located(
(By.XPATH, "//html[#class='js history es5array video opacity csspointerevents placeholder inlinesvg es5date supports es5function es5object strictmode es5string json es5syntax es5undefined es5 no-touchevents cssvhunit mediaqueries vibrate csscolumns csscolumns-width csscolumns-span csscolumns-fill csscolumns-gap csscolumns-rule csscolumns-rulecolor csscolumns-rulestyle csscolumns-rulewidth csscolumns-breakbefore csscolumns-breakafter csscolumns-breakinside flexbox csstransforms csstransforms3d no-ios panel-active']")))
selectJuly = annual_pass.find_element_by_css_selector('#reserver .panel.is-active>.panel-middle li:nth-child(3) .form-label')
selectJuly.click()
I tried as well, wait.until(lambda driver: driver.execute_script("return jQuery.active == 0")) but it did not work for this case.
I had a similar question: Cannot click on toggle with selenium because of ElementNotInteractableException

Python Selenium click google "I agree" button

I am trying to scrape some google data but I first want to click the 'I agree' button that google pops up. This is the script I use to do that:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
search_question = input("Ask a question: ")
driver = webdriver.Chrome("*Your Webdriver location*")
driver.wait = WebDriverWait(driver, 5)
driver.get("https://google.com")
time.sleep(1)
agree = driver.wait.until(EC.presence_of_element_located((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
# time.sleep(0.2)
search = driver.find_element_by_class_name("gLFyf")
search.send_keys(search_question)
search.send_keys(Keys.ENTER)
The problem is selenium doesn't seem to locate the button and therefore I get a timeout error. (I have tried also with find_element_by_xpath and still not working).

If you scroll up in the devtools inspector you'll notice that your element is within an iframe:
You need to switch to that frame first, click your button then switch back to the default content (the main page)
driver.get("https://google.com")
#active the iframe and click the agree button
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe")))
agree = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
#back to the main page
driver.switch_to_default_content()
That works for me.
FYI - There's only 1 iframe on the page, that's why the xpath //iframe works. If there were multiple you'd need to identify it with higher accuracy.

If you already have been agreed,then agree button would not appear. That why it is not be able to find given XPath.
Try this:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
search_question = input("Ask a question: ")
driver = webdriver.Chrome(".\chromedriver.exe")
driver.wait = WebDriverWait(driver, 5)
driver.get("https://google.com")
time.sleep(3)
# agree = driver.wait.until(EC.presence_of_element_located((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
# agree.click()
# time.sleep(0.2)
search = driver.find_element_by_class_name("gLFyf")
search.send_keys(search_question)
search.send_keys(Keys.ENTER)

Managed to get it working.
Seems like one of the few elements that are interactive in the popup when you first load the site is the language dropdown, which I found by class name.
Then the Agree button is detectable and you can find it by class too.
driver.get("https://www.google.com")
driver.maximize_window()
time.sleep(2)
#finds+clicks language dropdown, interaction unhides the rest of the popup html
driver.find_element_by_class_name('tHlp8d').click()
time.sleep(2)
#finds+clicks the agree button now that it has become visible
driver.find_element_by_id('L2AGLb').click()
you should now have the standard searchbar etc

i had some problems with clicking pop-ups and the problem turned out to be with that click().
im not sure that it might be the same issue with urs or not but try changing ur click to this :
agree = driver.find_element_by_xpath('//*[#id="introAgreeButton"]/span/span')
driver.execute_script("arguments[0].click();", agree)

The problem was that it didn't change frames here is the soulution code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('C:\\Users\\gassp\\OneDrive\\Namizje\\Python.projects\\chromedriver.exe')
url = 'https://www.google.com/maps/'
driver.get(url)
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, '//*[#id="consent-bump"]/div/div[1]/iframe')))
agree = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
#back to the main page
driver.switch_to_default_content()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="searchboxinput"]'))).send_keys('gostilne')
driver.submit()

Click ''Not now" on Instagram notifications ask using Selenium and Python

I've written a script that successfully makes the login on Instagram.
When I should go on my account, at home, the website displays a popup that asks you if you want notifications.
At this point, I tried a lot of solutions, but I got nothing.
I just want that, when the pop-up is displayed, the script should click on "Not now".
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
ids = []
driver = webdriver.Chrome(executable_path = '/usr/local/bin/chromedriver')
driver.get("https://www.instagram.com/accounts/login/?source=auth_switcher")
usm = driver.find_element_by_name('username').send_keys("**")
pwd = driver.find_element_by_name('password').send_keys("**")
btnLog = driver.find_element_by_tag_name('form').submit()
acpt = driver.find_element_by_xpath("//*[contains(#class, 'aOOlW HoLwm ')]")
In the image, there's the line of the button highlighted that I want to click:

Try the following code for this:
from selenium.webdriver.support import ui
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
ui.WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".aOOlW.HoLwm"))).click()
PS: I have used 10-second wait for the element to be clickable before click on it.
Hope it helps you!

To click() on the element with text as Not now on Instagram popup notification you can use the following solution:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
options = Options()
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://www.instagram.com/accounts/login/?source=auth_switcher")
driver.find_element_by_name('username').send_keys("Giacomo")
driver.find_element_by_name('password').send_keys("Maraglino")
driver.find_element_by_tag_name('form').submit()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(.,'Non ora')]"))).click()

just add click to acpt, like this:
acpt = driver.find_element_by_xpath("//*[contains(#class, 'aOOlW HoLwm ')]")
acpt.click()

For some reason, the solution to this question didn't fully work for me. The script does click on "not now" then I'm redirected to the "Home" page only to find the 'activate notifications' pop-up waiting for me.
This is the solution that I came up with:
Go to https://instagram.com/
Wait for the page to load.
Find the "Not now" button on the button using the full XPATH.
Clicking on it.
Here's the code (inserted after identifying to my Instagram account):
driver.get("https://instagram.com/")
time.sleep(7)
acpt = browser.find_element(by=By.XPATH, value='/html/body/div[1]/div/div/div/div[2]/div/div/div[1]/div/div[2]/div/div/div/div/div/div/div/div[3]/button[2]')
acpt.click()