We Keep Coding

Use Python Selenium to extract span text

Hi i'm new at selenium and webscraping and i need some help.
i try to scrape one site and i need and i dont know how to get span class.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
PATCH = "/Users/bobo/Downloads/chromedriver"
driver = webdriver.Chrome(PATCH)
driver.get("https://neonet.pl")
print(driver.title)
search = driver.find_element_by_class_name("inputCss-input__label-263")
search.send_keys(Keys.RETURN)
time.sleep(5)
i try to extract this span
<span class="inputCss-input__label-263">Szukaj produktu</span>

I can see that you are trying to search something in the search bar.
First I recommend you to use the xpath instead of the class name, here is a simple technique to get the xpath of every element on a webpage:
right-click/ inspect element/ select the mouse in a box element on the upper left/ click on the element on the webpage/ it will directly show you the corresponding html/ then right click on the selected html/ copy options and then xpath.
Here is a code example that searches an element on the webpage, I also included the 'Webdriver-wait' option because sometimes the code runs to fast and can't find the next element so this function make the code wait till the element is visible:
from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(executable_path="/Users/bobo/Downloads/chromedriver")
driver.get("https://neonet.pl") #loading page
wait = WebDriverWait(driver, 20) #defining webdriver wait
search_word = 'iphone\n' # \n is going to act as an enter key
wait.until(EC.visibility_of_element_located((By.XPATH, '//*[#id="root"]/main/div[1]/div[4]/div/div/div[2]/div/button[1]'))).click() #clicking on cookies popup
wait.until(EC.visibility_of_element_located((By.XPATH, '//*[#id="root"]/main/header/div[2]/button'))).click() #clicking on search button
wait.until(EC.visibility_of_element_located((By.XPATH, '//*[#id="root"]/aside[2]/section/form/label/input'))).send_keys(search_word) #searching on input button
print('done!')
sleep(10)
Hope this helped you!

wait=WebDriverWait(driver,10)
driver.get('https://neonet.pl')
elem=wait.until(EC.visibility_of_element_located((By.XPATH, "//span[contains(#class,'inputCss-input')]"))).text
print(elem)
To output the value of the search bar you use .text on the Selenium Webelement.
Imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

I'm trying to automate a process on a website. How do I close a pop up window with selenium, when it is not clear when the popup window will appear?

I am trying to close a pop up window with selenium in python, which is not allowing my code to execute further. But I am unable to do that. There is a a pop up window which asks me if i want to sign up but it keeps popping up at inconsistent times. Is there a method to check wether or not the pop-up window is active?
My code so far:
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
PATH = "C:\Program Files (x86)\chromedriver.exe";
driver = webdriver.Chrome(PATH);
driver.get("https://www.investing.com/news/")
time.sleep(3)
accept_cookies = driver.find_element_by_xpath('//*[#id="onetrust-accept-btn-handler"]');
accept_cookies.click();

So your problem is that you don't want the signup popup to display.
I was just poking the site's script, I found a really nice way for you to work around it: set user agent to gene. The script to display popup will be disabled if user agent matches some mobile browsers.
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
opts = Options()
opts.add_argument("user-agent=gene")
driver = webdriver.Chrome(options=opts)
# Adds the cookie into current browser context
driver.get("https://www.investing.com/news/")
time.sleep(60) # wait for a minute to see if the popup happens
driver.quit()
Run this code, observe the page, it will not display the signup popup.
Alternative work around:
Use a Chrome profile that already turned off the Signup popup, will also not display the popup
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Path") #Path to your chrome profile
If you really need to close the popup without these methods, define a check function with try/except. Before doing any actions, call this function to check if there's popup, then close the popup, save the state to some variable (so you don't check it anymore). Since the function has try/except, it will not raise Exception and your code will run.

You can do this with 2 quick methods that I can think of immediately.
1:
You will use this often
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
then when you want to get an element that needs time to load and you want to do actions with it, you implement it with WebDriverWait :
wait = WebDriverWait(driver, 10)
try:
accept_cookies = wait.until(EC.presence_of_element_located((By.XPATH, "'//*
[#id=\"onetrust-accept-btn-handler\"]'")))
catch:
# probably you have a bad locator and the element doesn't exist or it needs more than 10 sec to load.
else:
# logic here is processed after successfully obtaining the pop up
accept_cookies.click()
I would recommend using ActionChains
from selenium.webdriver.common.action_chains import ActionChains
then after you obtain the pop up element make the click like this:
actions = ActionChains(driver)
actions.move_to_element(accept_cookies).click().perform()
This is the equivalent of moving the mouse pointer to the middle of the pop up close btn and clicking it

How to Click the "OK" Button within an Alert using Python + Selenium

I want to click the "OK" button in this pop up dialog
I tried:
driver.switchTo().alert().accept();
but it doesn't work

To click on the OK button within the alert you need to induce WebDriverWait for the desired alert_is_present() and you can use the following solution:
WebDriverWait(driver, 10).until(EC.alert_is_present())
driver.switch_to.alert.accept()
Note : You have to add the following imports :
from selenium.webdriver.common.alert import Alert
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a couple of relevant discussions in:
Python click button on alert
How to read the text from the alert box using Python + Selenium
Why switching to alert through selenium is not stable?
Would like to understand why switch_to_alert() is receiving a strikethrough and how to fix

Using Selenium to remove PopUp

I am using Selenium to remove a PopUp from investing.com but i am not able to recognise the PopUp correctly. The code I am using is ;
I need to click the button "Got it"...
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from time import sleep
browser = webdriver.Chrome(executable_path="C:\\…….\chromedriver.exe")
browser.get("https://uk.investing.com/indices/mining-historical-data/")
browser.switchTo().frame(browser.findElement(By.xpath("//iframe[contains(#src,'https://prefmgr- cookie.truste-svc.net/cookie_js/cookie_iframe.html?parent=https://consent-pref.trustarc.com/?? layout=gdpr&type=investingiab2&site=investing-iab.com&action=notice&country=gb&locale=en&behavior=expressed&uid=dc7cd041-d8c4-4102-9522-1025da9b6e09&privacypolicylink=https://uk.investing.com/about-us/privacy-policy&iab=true&irm=undefined&from=https://consent.trustarc.com/')]"")));
sleep(3)
browser.findElement(By.xpath("//input[#name='Got it']")).click();

The popup is inside an iframe and elements ids change so it is a bit challenging.
Sometimes a second popup appears, make sure run this code before it.
This code is for closing the first popup.
from selenium import webdriver
from time import sleep
browser = webdriver.Chrome(executable_path="C:\\…….\chromedriver.exe")
browser.get("https://uk.investing.com/indices/mining-historical-data/")
sleep (3)
# get iframe element
title = "TrustArc Cookie Consent Manager"
frameElement = browser.find_element_by_xpath("""//*[#title="TrustArc Cookie Consent Manager"]""")
# get id iframe
id = frameElement.get_property("id")
print(id)
# switch to iframe element
browser.switch_to.frame(id)
sleep (1)
# get element to click
elemet = browser.find_element_by_xpath("""//*[#class='call']""")
print(elemet.get_attribute('innerHTML'))
# click element
elemet.click()
# switch the control back to the parent frame
browser.switch_to.default_content()

Python Selenium closing a pop-up window

I'd like to close this window opened up on Amazon web site by using Selenium with Python. I've tried find_element_by_xpath, but it doesn't work. Here's the snippet of the code;
close_to_list = browser.get("/html/body/div[4]/div/div/div[2]/div[2]/div[2]/div[1]/div[2]/div/div/table/tbody/tr[2]")
I get the xpath of that 'X' button but I guess I need to close it as switch_to_alert, but I'm new to this era so I couldn't write it properly.
Here's the image view;
enter image description here

You don't need to use switch_to_alert() as well as get() to close modal window. Just try to close it with below code:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button[aria-label="Close"]'))).click()
This allows you to wait for button appearance and click the button