We Keep Coding

How to click on cookie popup using webdriver chrome to access the actual webpage

I am struggling to make selenium click on the cookie popup and access the website.
I have the following code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import pandas as pd
from datetime import datetime
import os
import sys
web = 'https://www.thesun.co.uk/sport/football/'
path = '/Users/cc/Documents/Documents/IT/1. Python/WebCrawler/chromedriver'
driver_service = Service(executable_path=path)
driver = webdriver.Chrome(service=driver_service)
driver.get(web)
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH,'//*.[#id="notice"]/div[4]/button[2]'))).click()
here is the webpage:
enter image description here
here is the htlm button element:
enter image description here
here is the vs code terminal after running:
enter image description here
The webpage open but the click command does not do the work...even when defining a wait until element is clickable.
I tried to use the click command with wait to be sure the element is clickable.
the idea is to click on the "Fine By Me!" button.
Thanks for your help.

You need to switch to the iframe and then click on the button. Also there is a browser pop-up for allowing/blocking notifications. You need to handle that in the driver setup.
Driver Setup can be done with params
chromeOpts = Options()
chromeOpts.add_argument('start-maximized')
chromeOpts.add_argument("--disable-notifications")
chromeOpts.add_argument("--disable-infobars")
chromeOpts.add_argument("--disable-extensions")
The code to switch to iframe and then clikcing on button is -
iframe = "//iframe[#title='SP Consent Message']"
ifr = driver.find_element(By.XPATH, iframe)
driver.switch_to.frame(ifr)
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH,'//*[#id="notice"]/div[4]/button[2]'))).click()
Here -

To click on the element Fine By Me! you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[title='Fine By Me!'][aria-label='Fine By Me!']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#title='Fine By Me!' and #aria-label='Fine By Me!']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Selenium web scraping iframe

i want to read the toner values ​​on the web pages of the various printers in my office.
The problem is that the page is made up of several frames, and the one in which there is the remaining toner, is written in js and I can't read it even with selenium
This is my code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.expected_conditions import (
presence_of_element_located)
from selenium.webdriver.support.wait import WebDriverWait
def get_comment_count(driver, url):
driver.get(url)
wait = WebDriverWait(driver, 3)
e = driver.find_elements_by_xpath("/html/frameset/frame")
driver.switch_to_frame(e[0])
toner_iframe = driver.find_elements_by_xpath('//*[#id="contain"]')
# iframe_url = toner_iframe.get_attribute('src')
#driver.switch_to_frame(toner_iframe)
driver.switch_to.frame(toner_iframe)
print(toner_iframe)
url = "https://pritner_web_page"
options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
driver = webdriver.Chrome(options=options)
get_comment_count(driver,url)
I tried also...
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
driver = webdriver.Chrome(options=options)
driver.get("http://printer_web_page")
WebDriverWait(driver,5).until(EC.frame_to_be_available_and_switch_to_it((By.ID,'wlmframe')))
WebDriverWait(driver,5).until(EC.frame_to_be_available_and_switch_to_it((By.ID,'toner')))
page_source=driver.page_source
print(page_source)
This is DOM Inspector of page. The various frames are dynamic and written in js as follows:
The code I wrote is just one of several different attempts to get to the frame, but to no avail

The element is within nested <frame> / <iframe> elements so you have to:
Induce WebDriverWait for the parent frame to be available and switch to it.
Induce WebDriverWait for the child frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("http://printer_web_page")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"frame[name='wlmframe']")))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#toner[name='toner']")))
Using XPATH:
driver.get("http://printer_web_page")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//frame[#name='wlmframe']")))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='toner' and #name='toner']")))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
How to write appropriate Xpath to locate text value
How To sign in to Applemusic With Python Using Chrome Driver With Selenium

Python selenium find_element_by_xpath not finding existing element in page

I am using Python 3.6+ and Selenium 3.141
I am trying to get an element from a page, and although I'm using the correct Xpath expression (confirmed in the brower console), the same Xpath expression raises a 'NotFound' error in selenium chrome driver.
myscript.py
from selenium import webdriver
url = 'https://live.euronext.com/en/product/stock-options/AH1-DPAR'
browser = webdriver.Chrome(executable_path='./chromedriver')
browser.get(url)
try:
checkbox = browser.find_element_by_xpath('//*[#id="form-options-index"]/div/div[2]')
except:
pass
The script throws an exception where the find_element_by_xpath() method is invoked - even though when using a browser, the same Xpath expression will result in the element being identified/selected correctly.
Why is the Xpath expression not working with selenium? How do I fix this?

Required content absent in page source - it's loaded dynamically, so you need to wait until it appear in DOM:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = 'https://live.euronext.com/en/product/stock-options/AH1-DPAR'
browser = webdriver.Chrome(executable_path='./chromedriver')
browser.get(url)
checkbox = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.XPATH, '//*[#id="form-options-index"]/div/div[2]')))

To click on Select all CheckBox. Use WebDriverWait() and wait for element_to_be_clickable() and following Xpath.
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,'//*[#id="form-options-index"]//label[#for="select_all_dates"]'))).click()
You need to import below libraries.
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

Clicking button via python selenium chrome

I would like to click button "ja ik ga akkoord" on url anwb.nl with python selenium chrome. I have copied the relative xpath but when is use it i keep getting NoSuchElementException. Also id, name, etc no luck
I start with:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
driver = webdriver.Chrome(options=options)
driver.get('https://anwb.nl')
When i inspect the page, xpath of the button gives me:
//*[#id="accept default level"]
When i use this with ...by_xpath i get NoSuchElementException
The code of the button is:
<button class="btn-decide_link-internal" type="button"
name="save"
id="accept default level"> ==$0
Ja, ik ga akkoord</button>
I tried id (accept def...), name (save), but all nosuchelement
In general i would really like to understand how to interpret the web code in general can solve future problems.

The element with text as Ja, ik ga akkoord is within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("https://www.anwb.nl/");
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src*='anwb']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.btn-decide_link-internal"))).click()
Using XPATH:
driver.get("https://www.anwb.nl/");
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[contains(#src, 'anwb')]")))
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//button[#id='accept default level']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser snapshot:
Here you can find a relevant discussion on Ways to deal with #document under iframe

There is an iframe.Induce WebDriverWait and switch to frame first and then click on the button.
EC.frame_to_be_available_and_switch_to_it()
EC.element_to_be_clickable()
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
driver = webdriver.Chrome(options=options)
driver.get('https://anwb.nl')
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.TAG_NAME,"iframe")))
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.ID,"accept default level"))).click()

Selenium can't find element, but element is on the https://login.aliexpress.com/ webpage

On the website the selenium script cannot find the login and password fields. I tried to search by xpath, css selector, name and class name. But nothing worked.
from selenium import webdriver
from time import sleep
driver = webdriver.Firefox()
driver.get("https://login.aliexpress.com/")
driver.find_element_by_id("fm-login-id").send_keys("test_id")
driver.find_element_by_id("fm-login-password").clear()
driver.find_element_by_id("fm-login-password").send_keys("test_pass")
driver.find_element_by_id("fm-login-submit").click()`
I tried to do this with the help of Selenium IDE, and everything worked in the GUI. But after I exported the code to python and ran it, the program gave an error that it could not find the element.

The login form is inside of a frame, you need to switch to it first.
from selenium import webdriver
from time import sleep
driver = webdriver.Firefox()
driver.get("https://login.aliexpress.com/")
frame = driver.find_element_by_id("alibaba-login-box")
driver.switch_to.frame(frame)
driver.find_element_by_id("fm-login-id").send_keys("test_id")
driver.find_element_by_id("fm-login-password").clear()
driver.find_element_by_id("fm-login-password").send_keys("test_pass")
driver.find_element_by_id("fm-login-submit").click()

However as the the desired elements are within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired elements to be clickable.
You can use the following solution:
Using CSS_SELECTOR:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get("https://login.aliexpress.com/")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#alibaba-login-box[src^='https://passport.aliexpress.com/mini_login.htm?']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.fm-text#fm-login-id"))).send_keys("test_id")
driver.find_element_by_css_selector("input.fm-text#fm-login-password").send_keys("test_pass")
driver.find_element_by_css_selector("input.fm-button#fm-login-submit").click()
Interim Broswer Snapshot:
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a relevant discussion in
Ways to deal with #document under iframe