I'm trying to automate email printing with Selenium & Chrome Driver; for some reason, the snippet below works fine when I exclude the options=options argument (i.e. not in headless mode). When I switch headless on, even specifying window size it still is unable to find the element by ID.
Config
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from time import sleep
#-----------------DEFAULT CONFIG
options = Options()
options.add_argument('--headless')
options.add_argument('--window-size=1920,1080')
options.add_argument("--allow-insecure-localhost")
options.add_argument("--allow-insecure-content")
options.add_argument("--no-sandbox")
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
PATH = '/Users/*********/2021/selenium/chromedriver'
driver = webdriver.Chrome(PATH,options=options)
Failing Block (last line)
google_login = 'https://accounts.google.com/Login#identifier'
driver.get(google_login)
sleep(2)
html = driver.page_source.strip()
# =============EMAIL BOX
user_name = driver.find_element_by_id('identifierId')
Error Traceback: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="identifierId"]"}
I've tried using WebDriverWait in a try/except which also doesn't help.
Screenshot captured in headless mode attached (line prior to the failing step)
see headless mode is bit fast so I would suggest you to use it with Explicit waits :
wait = WebDriverWait(driver, 20)
user_name = wait.until(EC.element_to_be_clickable((By.ID, "identifierId")))
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Refer here for more
Instead of
options.add_argument("--window-size=1920,1080")
Please try this.
options.add_argument('--window-size=1920x1080')
Related
I am using Python 3.10 and I am trying to log in on Twitter using Selenium. I am not able to get a hold of the input button to enter my user. Looks like the selenium documentation has changed. I could not find a good example.
This is the code I try:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys # not sure why this is greyed out
Chrome_driver_path = "C:/Users/osman/OneDrive/Desktop/chromedriver.exe"
driver = webdriver.Chrome(Chrome_driver_path)
driver.get("https://www.twitter.com/login")
First_Name = driver.find_element(By.XPATH, '//*[#id="react-root"]/div/div/div/main/div/div/div/div[2]/div[2]/div/div[5]/label/div/div[2]/div/input')
First_Name.send_keys('osman#yahoo.com')
And this is the error I get:
executable_path has been deprecated, please pass in a Service object
driver = webdriver.Chrome(Chrome_driver_path)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="react-root"]/div/div/div/main/div/div/div/div[2]/div[2]/div/div[5]/label/div/div[2]/div/input"}
(Session info: chrome=106.0.5249.119)
Stacktrace:
Backtrace:
Ordinal0 [0x00841ED3+2236115]
Process finished with exit code 1
There are several problems here:
As mentioned, executable_path has been deprecated. You should use Service object instead as following.
Very long absolute XPath locators are very breakable. You have to use relative locators.
You also have to use WebDriverWait expected_conditions to wait for elements to become clickable before iterating them.
And finally from selenium.webdriver.common.keys import Keys is greyed out since you never used this import in your code. So, it's OK, but this line could be removed to make your code clean.
The following code works:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(service=webdriver_service, options=options)
url = 'https://www.twitter.com/login'
driver.get(url)
wait = WebDriverWait(driver, 20)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '[autocomplete="username"]'))).send_keys("osman#yahoo.com")
This is what happens on www.twitter.com/login after the above code run, screenshot:
I am trying to write some python code which can click on 'Alles accepteren'.
The website is called: www.Bol.com
Because of my lack of knowledge, i don't know how to find the frame python should focus on.
I know that i should use:
driver.switch_to.frame()
Anyone who can help me??
You have to just accept the cookies and ever more reliable is to use load time locator strategy which is WebDriverWait
Full working code as an example:
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from webdriver_manager.chrome import ChromeDriverManager
options = webdriver.ChromeOptions()
options.add_argument("--no-sandbox")
options.add_argument('--disable-blink-features=AutomationControlled')
options.add_argument("start-maximized")
options.add_experimental_option("detach", True)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),options=options)
URL ='https://www.bol.com/nl/nl/'
driver.get(URL)
#To accept cookie
WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#js-first-screen-accept-all-button'))).click()
Actually, there are no frames on this page. So no need to switch.
element = driver.find_element(By.XPATH, "//button[#id='js-first-screen-accept-all-button']")
element.click()
There is no iframe, you can just use ID:
driver.find_element(By.ID, "js-first-screen-accept-all-button").click()
I am trying to click on a place on a video. I tried it with xpath already, but without success.
For example on this tiktok video: https://www.tiktok.com/#willsmith/video/7125844820328926510?is_from_webapp=v1&item_id=7125844820328926510&web_id=7139992072584676869
I'm trying to click on the heart with selenium (python).
That's my code:
if driver.find_element_by_xpath("/html/body/div[2]/div[2]/div[2]/div[1]/div[3]/div[1]/div[1]/div[3]/button[1]/span/div/svg/g/path") :
driver.find_element_by_xpath("/html/body/div[2]/div[2]/div[2]/div[1]/div[3]/div[1]/div[1]/div[3]/button[1]/span/div/svg/g/path").click()
It says that it's "Unable to locate element". I don't know why. I even added some sleep to the code because I thought that the website didn't load up fully or even tried with a different xpath.
I also tried to do it with the ID of the "heart-location" but the ID is very hard to understand if I inspect element.
Could someone please help me out? Thanks in advance!
You need to use the correct locator
And to wait for the element to be clickable.
For the former WebDriverWait Expected Conditions explicit wait should be used.
The below code works:
(In case you are already logged in)
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(service=webdriver_service, options=options)
url = "https://www.tiktok.com/#willsmith/video/7125844820328926510?is_from_webapp=v1&item_id=7125844820328926510&web_id=7139992072584676869"
driver.get(url)
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "span[data-e2e='like-icon']"))).click()
In case you want to use XPath instead of CSS Selector just change the line above with
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[#data-e2e='like-icon']"))).click()
I have tried multiple times with other instruction codes with space from the tutorial which worked fine. However, when I just changed the URL and the following class, it would give the error saying
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified
(Session info: chrome=100.0.4896.88)
Everything worked when I used the tutorial code.
Here is my code (I have solved a few chrome driver problems from the internet)
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-logging"])
driver = webdriver.Chrome(options=options)
driver.get("https://raritysniper.com/nft-drops-calendar")
time.sleep(1)
link = driver.find_element_by_css_selector(".w-full.h-full.align-middle.object-cover.dark:brightness-80.dark:contrast-103.svelte-f3nlpp").get_attribute("alt")
print(link)
I am trying to get attributes of each projects and make them into csv.
(Please refer to the screenshot)
Screen shot of HTML that I am trying to extract
it would be wonderful if anyone could depict the problem I got with the code.
Thank you!
The CSS_SELECTOR that you are using
.w-full.h-full.align-middle.object-cover.dark:brightness-80.dark:contrast-103.svelte-f3nlpp
does not really match any element in the HTML.
Instead, you should use this CSS_SELECTOR:
div.w-full.h-full.align-middle img:not(.placeholder)
In code:
driver.maximize_window()
wait = WebDriverWait(driver, 30)
driver.get("https://raritysniper.com/nft-drops-calendar")
#time.sleep(1)
first_element = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "div.w-full.h-full.align-middle img:not(.placeholder)")))
print(first_element.get_attribute('alt'))
print(first_element.get_attribute('src'))
Import:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Output:
NEON PLEXUS
https://media.raritysniper.com/featured/neon-plexus_1648840196045_3.webp
Process finished with exit code 0
So I use this script:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
import time
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
driver = webdriver.Chrome('C:/Users/N-Thalpy/Documents/NT/py/chromedriver.exe', chrome_options=chrome_options)
driver.get("https://web.whatsapp.com/")
wait = WebDriverWait(driver, 600)
target = '"Batman"'
string = "Python 3.8"
x_arg = '//span[contains(#title,' + target + ')]'
group_title = wait.until(EC.presence_of_element_located((
By.XPATH, x_arg)))
group_title.click()
inp_xpath = '//div[#class="_3u328"][#dir="ltr"][#data-tab="1"]'
input_box = wait.until(EC.presence_of_element_located((
By.XPATH, inp_xpath)))
for i in range(100):
input_box.send_keys(string + Keys.ENTER)
time.sleep(1)
And Chrome opens in web.whatsapp.com as expected - then it opens "Batman's" chat. However, it doesn't write anything. I checked the attributes and even tried to be more specific about them, but nothing seems to work. I tried it without sandbox option and still nothing.
Edit: the only message I get from the py.exe console is this:
"[13920:4684:0130/154024.795:ERROR:wmi_refresher.cc(129)] Unable to add the Win32_PerfRawData_PerfDisk_PhysicalDisk enum."
I managed to solve it by:
Downloading the correct chromedriver version as I was using Chrome 79 and using the driver for Chrome 80.
Searching for ALL the attributes of the div, includin spellcheck, contenteditable and dir (I don't how relevant this actually is, but just in case).