We Keep Coding

How to click on cookie popup using webdriver chrome to access the actual webpage

I am struggling to make selenium click on the cookie popup and access the website.
I have the following code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import pandas as pd
from datetime import datetime
import os
import sys
web = 'https://www.thesun.co.uk/sport/football/'
path = '/Users/cc/Documents/Documents/IT/1. Python/WebCrawler/chromedriver'
driver_service = Service(executable_path=path)
driver = webdriver.Chrome(service=driver_service)
driver.get(web)
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH,'//*.[#id="notice"]/div[4]/button[2]'))).click()
here is the webpage:
enter image description here
here is the htlm button element:
enter image description here
here is the vs code terminal after running:
enter image description here
The webpage open but the click command does not do the work...even when defining a wait until element is clickable.
I tried to use the click command with wait to be sure the element is clickable.
the idea is to click on the "Fine By Me!" button.
Thanks for your help.

You need to switch to the iframe and then click on the button. Also there is a browser pop-up for allowing/blocking notifications. You need to handle that in the driver setup.
Driver Setup can be done with params
chromeOpts = Options()
chromeOpts.add_argument('start-maximized')
chromeOpts.add_argument("--disable-notifications")
chromeOpts.add_argument("--disable-infobars")
chromeOpts.add_argument("--disable-extensions")
The code to switch to iframe and then clikcing on button is -
iframe = "//iframe[#title='SP Consent Message']"
ifr = driver.find_element(By.XPATH, iframe)
driver.switch_to.frame(ifr)
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH,'//*[#id="notice"]/div[4]/button[2]'))).click()
Here -

To click on the element Fine By Me! you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[title='Fine By Me!'][aria-label='Fine By Me!']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#title='Fine By Me!' and #aria-label='Fine By Me!']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Unable to find input element with Selenium and Python

I am new in selenium and python.
I trying to find element using selenium but no matter what I tried (xpath, CSS) I always got the following message:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element
Here the code I wrote :
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from time import sleep
from requests_html import HTML
options = Options()
# options.add_argument("--headless")
options.add_argument('disable-notifications')
options.add_argument('--disable-infobars')
options.add_argument('--disable-blink-features=AutomationControlled')
driver = webdriver.Chrome(executable_path=r'C:/Users/XXX/PycharmProjects/chromedriver.exe', options=options)
first_url = "https://www.micromania.fr/jeu-concours-summer-show.html"
driver.get(first_url)
# For the following, it works
refuse = driver.find_element_by_xpath('//*[#id="truste-consent-required"]').click()
time.sleep(2)
But when I tried to do a driver.find_element_by -CSS, Xpath, to find for the mail field to be able to fill it, I met the error message.
HTML of the mail field I tried to reach is there:
<input data-v-0f8f86ae="" type="email" placeholder="Email*" class="input">
UPDATE

You need to first click on the account icon, having class sidebar-login header-link-item icon-account no-decoration color-white. That click will do a XHR network call, which will bring some new elements into page, including email address field. Also, it's wise to wait for elements to load in page before searching them, instead of just searching for them. Also, it's wise to wrap the cookie button dismissal in a try/except block, just in case it's not popping up all the time.
Adapting the following snippet to your code will get you what you're after:
url='https://www.micromania.fr/jeu-concours-summer-show.html'
browser.get(url)
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".sidebar-login.header-link-item.icon-account.no-decoration.color-white"))).click()
email_field = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#title='Adresse email']")))
email_field.click()
email_field.send_keys('my amazing email address')
print('done')
EDIT: for the email field I'm not able to see (restricted to French IPs only):
email_2_field = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='home']/input[#type='email']")))
email_2_field.click()

Given the HTML:
<input data-v-0f8f86ae="" type="email" placeholder="Email*" class="input">
As the desired element is a dynamic element, to send a character sequence within the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.input[type='email'][placeholder^='Email']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#class='input' and starts-with(#placeholder, 'Email')][#type='email']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Selenium web scraping iframe

i want to read the toner values ​​on the web pages of the various printers in my office.
The problem is that the page is made up of several frames, and the one in which there is the remaining toner, is written in js and I can't read it even with selenium
This is my code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.expected_conditions import (
presence_of_element_located)
from selenium.webdriver.support.wait import WebDriverWait
def get_comment_count(driver, url):
driver.get(url)
wait = WebDriverWait(driver, 3)
e = driver.find_elements_by_xpath("/html/frameset/frame")
driver.switch_to_frame(e[0])
toner_iframe = driver.find_elements_by_xpath('//*[#id="contain"]')
# iframe_url = toner_iframe.get_attribute('src')
#driver.switch_to_frame(toner_iframe)
driver.switch_to.frame(toner_iframe)
print(toner_iframe)
url = "https://pritner_web_page"
options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
driver = webdriver.Chrome(options=options)
get_comment_count(driver,url)
I tried also...
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
driver = webdriver.Chrome(options=options)
driver.get("http://printer_web_page")
WebDriverWait(driver,5).until(EC.frame_to_be_available_and_switch_to_it((By.ID,'wlmframe')))
WebDriverWait(driver,5).until(EC.frame_to_be_available_and_switch_to_it((By.ID,'toner')))
page_source=driver.page_source
print(page_source)
This is DOM Inspector of page. The various frames are dynamic and written in js as follows:
The code I wrote is just one of several different attempts to get to the frame, but to no avail

The element is within nested <frame> / <iframe> elements so you have to:
Induce WebDriverWait for the parent frame to be available and switch to it.
Induce WebDriverWait for the child frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("http://printer_web_page")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"frame[name='wlmframe']")))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#toner[name='toner']")))
Using XPATH:
driver.get("http://printer_web_page")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//frame[#name='wlmframe']")))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='toner' and #name='toner']")))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
How to write appropriate Xpath to locate text value
How To sign in to Applemusic With Python Using Chrome Driver With Selenium

How to click the follow button on Instagram?

I am trying to click the follow button on Instagram using Python Selenium
https://www.instagram.com/luvly_zuby/?hl=en
I've tried the bellow code but it's not working.
#click follow
follow = driver.find_element_by_partial_link_text("Follo")
ActionChains(driver).move_to_element(follow).click().perform()

You can click on the element by using simple selenium click by finding the element using its text in the xpath and then using explicit wait on the element.
You can do it like:
follow_button = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//button[text()='Follow']")))
follow_button.click()
You need to add the following imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

This should work for you. Pass the WebElement to click(element) method.
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('testUrl')
follow = driver.find_element(By.XPATH, '//button[contains(text(),'Follow')]')
webdriver.ActionChains(driver).move_to_element(follow).click(follow).perform()

Try to find the element using this css selector a.BY3EC > button and wait using .element_to_be_clickable:
follow = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'a.BY3EC > button')))
follow.click()
Following import:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Selenium can't find element, but element is on the https://login.aliexpress.com/ webpage

On the website the selenium script cannot find the login and password fields. I tried to search by xpath, css selector, name and class name. But nothing worked.
from selenium import webdriver
from time import sleep
driver = webdriver.Firefox()
driver.get("https://login.aliexpress.com/")
driver.find_element_by_id("fm-login-id").send_keys("test_id")
driver.find_element_by_id("fm-login-password").clear()
driver.find_element_by_id("fm-login-password").send_keys("test_pass")
driver.find_element_by_id("fm-login-submit").click()`
I tried to do this with the help of Selenium IDE, and everything worked in the GUI. But after I exported the code to python and ran it, the program gave an error that it could not find the element.

The login form is inside of a frame, you need to switch to it first.
from selenium import webdriver
from time import sleep
driver = webdriver.Firefox()
driver.get("https://login.aliexpress.com/")
frame = driver.find_element_by_id("alibaba-login-box")
driver.switch_to.frame(frame)
driver.find_element_by_id("fm-login-id").send_keys("test_id")
driver.find_element_by_id("fm-login-password").clear()
driver.find_element_by_id("fm-login-password").send_keys("test_pass")
driver.find_element_by_id("fm-login-submit").click()

However as the the desired elements are within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired elements to be clickable.
You can use the following solution:
Using CSS_SELECTOR:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get("https://login.aliexpress.com/")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#alibaba-login-box[src^='https://passport.aliexpress.com/mini_login.htm?']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.fm-text#fm-login-id"))).send_keys("test_id")
driver.find_element_by_css_selector("input.fm-text#fm-login-password").send_keys("test_pass")
driver.find_element_by_css_selector("input.fm-button#fm-login-submit").click()
Interim Broswer Snapshot:
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a relevant discussion in
Ways to deal with #document under iframe