I am using Python and Selenium to open different pages on PredictIt. So far I have this:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--disable-notifications")
browser = webdriver.Chrome(options = chrome_options)
browser.get('https://www.predictit.org/account/')
#Input username into PredictIt
username = browser.find_element_by_id("username")
username.clear()
username.send_keys([MY USERNAME])
#Input password into PredictIt
password = browser.find_element_by_id("password")
password.clear()
password.send_keys([MY PASSWORD])
#Click login button
browser.find_element_by_xpath('//*[#id="app"]/div[3]/div/div/div/div[2]/div/form/div/div[6]/button[#type="submit"]').click()
browser.get_cookies() #get_cookies() allows it to stay logged in
browser.find_element_by_xpath('//*[#id="nav-markets"]').click()
The last line of code routes it to the "Markets" page. The issue I have is that this sometimes works and sometimes doesn't. I would say about 60% of the time, it clicks through and brings me to the page. The other 40% of the time, it stays on the current page after login and produces the error:
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element ... is not clickable at point (270, 24). Other element would receive the click: <div class="modal__wrapper">...</div>
(Session info: chrome=86.0.4240.198)
Would anyone have a solution to why this might be?
I would guess that's because there are no delay in your code so everything appends at the same time. You may need to add a bit of delay after submitting the form so that the page can load.
As a rule of thumb, I always add a bit of delay before actions such as submitting a form or loading a page using time.sleep(), 0.5s should be enough.
Import the time module and try adding time.sleep(0.5) before the last line.
After you sign in I would induce a webdriver wait for the element to popup since the element might not be loaded in or currently interactable.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[#id='nav-markets']"))).click()
Related
Goal:
Write a review of a location on google maps with python and selenium
Stages:
Login to google account (already successful)
Go to the review page (already successful)
Giving 5 stars (not yet successful)
Give a review (not yet successful)
Submit a review (unsuccessful)
Problem:
An error occurred: selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: invalid locator (Session info: chrome=108.0.5359.125)
Stuck on the page in the following image
I've checked inspect to make sure
Code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
chrome_options = webdriver.ChromeOptions()
driver = webdriver.Chrome(executable_path=r'C:\Users\user\OneDrive\Desktop\selenium\chromedriver.exe', options=chrome_options)
# go to login page
driver.get("https://accounts.google.com/login?hl=id")
# write the email address
email = driver.find_element("name", "identifier")
email.send_keys("email#gmail.com")
sleep(2)
email.send_keys(Keys.RETURN)
sleep(5)
# write the password
password = driver.find_element("name", "Passwd")
password.send_keys("Password")
password.send_keys(Keys.RETURN)
sleep(5)
# go to review page
driver.get("https://search.google.com/local/writereview?placeid=ChIJDf8ln6V5ei4REzngJl6HEEc&pli=1")
sleep(8)
driver.find_element("aria-label", "Lima bintang").click()
How to solve this problem? At least to click the 5 stars. Thanks
That elements block is inside an iframe. So, you need to switch into the iframe to access those elements. So, you code can be as following:
driver.get("https://search.google.com/local/writereview?placeid=ChIJDf8ln6V5ei4REzngJl6HEEc&pli=1")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it(driver.find_element(By.XPATH, "//div[not(#aria-hidden)]/iframe[#name='goog-reviews-write-widget']")))
driver.find_element("aria-label", "Lima bintang").click()
When finished working inside that iframe do not forget to get out to the default content with
driver.switch_to.default_content()
if you want to continue your test flow.
Also, instead of all these hardcoded pauses you should use WebDriverWait expected_conditions explicit waits.
I am trying to automate downloads from a webpage using selenium.
So far my strategy is to instantiate a firefox driver that loads the page and clicks the download button. However, to be clickable, the button needs to be visible, i.e. not covered by any banners on the page (at least in my understanding). Therefore I need to scroll down (I use scrollIntoView()). If I run button.click() immediately after scrolling down, the download doesn't start, if I hard code a sufficient timeout in between it works out fine. Can somebody help me to set a timeout conditioned on the scroll down?
Here is my code:
from selenium import webdriver
profilePath = '/path/to/my/firefox/profile'
profile = webdriver.FirefoxProfile(profilePath)
driver = webdriver.Firefox(firefox_profile=profile)
driver.get("https://www.happyscribe.com/public/lex-fridman-podcast-artificial-intelligence-ai/164-andrew-huberman-sleep-dreams-creativity-the-limits-of-the-human-mind")
button = driver.find_element_by_id('btn-download')
target=driver.execute_script("arguments[0].scrollIntoView({block: 'center'});", button)
time.sleep(10)
button.click()
and this is the html code of the button:
<button class="hs-btn-secondary small" id="btn-download" type="button">Download</button>
I would be very thankful for any direct help or suggestions on how to tackle the problem from a different angle.
It sounds like you may need to induce WebDriverWait for the element to by clickable. You could do so like this. It will wait for the element to be both visible and enabled before clicking.
from selenium import webdriver
import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
profilePath = '/path/to/my/firefox/profile'
profile = webdriver.FirefoxProfile(profilePath)
driver = webdriver.Firefox(firefox_profile=profile)
driver.get("https://www.happyscribe.com/public/lex-fridman-podcast-artificial-intelligence-ai/164-andrew-huberman-sleep-dreams-creativity-the-limits-of-the-human-mind")
button = driver.find_element_by_id('btn-download')
target=driver.execute_script("arguments[0].scrollIntoView({block: 'center'});", button)
WebDriverWait(driver, 10).until(EC.element_to_be_clickable(button))
button.click()
I'm trying to learn how to use Selenium to log into a site:Ingram-Micro. I made a script and it worked on a different page: https://news.ycombinator.com/login.
Now I'm trying to apply the same thing to Ingram-Micro and I'm stuck and I don't know what else to try. The problem I'm having is a error/message that says the submit element is not clickable, there is a accept cookies button on the bottom of the page which seems to be causing the problem.
I've tried to account for it but I always get error saying that element doesn't exist. Yet if I don't try to click on the accept cookies element I get the original error saying the submit button isn't clickable. Here is my code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.common.exceptions import NoSuchElementException
import time
chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
url = "https://usa.ingrammicro.com/_layouts/CommerceServer/IM/Login.aspx?
returnurl=//usa.ingrammicro.com/"
driver = webdriver.Chrome(options=chrome_options)
driver.get(url)
def login():
USERNAME = 'email'
PASSWORD = 'password'
element = driver.find_element_by_link_text('I ACCEPT')
if element.is_displayed():
print("Element found")
element.click()
else:
print("Element not found")
driver.find_element_by_id('okta-signin-username').send_keys(USERNAME)
driver.find_element_by_id('okta-signin-password').send_keys(PASSWORD)
driver.find_element_by_id('okta-signin-submit').click()
login()
try:
me = driver.find_element_by_id("login_help-about")
print(f"{me.text} Element found")
except NoSuchElementException:
print('Not found')
driver.quit()
Here are the errors I get:
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <input class="button button-primary" type="submit" value="Log in" id="okta-signin-submit" data-type="save"> is not clickable at point (365, 560). Other element would receive the click: <p class="cc_message">...</p>
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate
element: {"method":"link text","selector":"I ACCEPT"}
(Session info: headless chrome=84.0.4147.125)
The challenge you face is synchronisation around scripts.
The chain of events on this site is 1) the page is loaded, 2) it kicks off it's javascript, 3) that slides the cookie window into view...
However, after the page is loaded, selenium doesn't know about the scripts so it thinks it is good to go. It's trying to click the button before it's there and gets upset that it can't find it. (NoSuchElementException)
There are different sync strategies - What works here is a webdriverwait to tell selenium to wait (without error) until that your object reached the specified expected conditions.
You can read more about waits and expected conditions here
Try this code.
For the cookie "I ACCEPT" button, I changed the identifier to xpath (since i like xpaths) and wrapped it in webdriverwait, waiting for the object to be clickable...
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.common.exceptions import NoSuchElementException
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
chrome_options = Options()
#chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
url = "https://usa.ingrammicro.com/_layouts/CommerceServer/IM/Login.aspx?returnurl=//usa.ingrammicro.com/"
driver = webdriver.Chrome(options=chrome_options)
driver.get(url)
def login():
USERNAME = 'email'
PASSWORD = 'password'
element = WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH, '//a[text()="I ACCEPT"]')))
if element.is_displayed():
print("Element found")
element.click()
else:
print("Element not found")
driver.find_element_by_id('okta-signin-username').send_keys(USERNAME)
driver.find_element_by_id('okta-signin-password').send_keys(PASSWORD)
driver.find_element_by_id('okta-signin-submit').click()
login()
Note that i had to remove headless to check it worked and there are 3 additional imports at the top.
Webdriverwait is great when you don't have lots of complicated objects, or have ojects with different wait conditions.
An alternative sync and (Easier in my opionin) is to set an implicit wait ONCE at the start of your script - and this configures the driver objecct.
driver.implicitly_wait(10)
As that link earlier says:
An implicit wait tells WebDriver to poll the DOM for a certain amount
of time when trying to find any element (or elements) not immediately
available. The default setting is 0. Once set, the implicit wait is
set for the life of the WebDriver object.
You can use it like this .. not doing all the code, just add this one line added after you create your driver and your code worked:
.....
url = "https://usa.ingrammicro.com/_layouts/CommerceServer/IM/Login.aspx?returnurl=//usa.ingrammicro.com/"
driver = webdriver.Chrome(options=chrome_options)
driver.get(url)
driver.implicitly_wait(10) # seconds
def login():
USERNAME = 'email'
PASSWORD = 'password'
element = driver.find_element_by_link_text('I ACCEPT')
if element.is_displayed():
print("Element found")
element.click()
else:
print("Element not found")
........
You probably need to click on the div above the input. try somethin like this:
child = driver.find_element_by_id('okta-signin-submit')
parent = child.find_element_by_xpath('..') # get the parent
parent.click() # click parent element
UPDATE: This worked great on geckodrive without headless, but not with chromedrive. so instead i've tried something else. Instead of clicking the button, lets just hit enter in the form and submit it this way:
from selenium.webdriver.common.keys import Keys
...
driver.find_element_by_id('okta-signin-username').send_keys(USERNAME)
password_field = driver.find_element_by_id('okta-signin-password')
password_field.send_keys(PASSWORD)
password_field.send_keys(Keys.RETURN)
I log into a private site and then wait for a link text to appear and then open the link with browser.get(). The code below throws a timeout exception almost every time while waiting for the link text "OTD" to appear. It is strange that it works once out of many tries. Even stranger is that if I use a sleep timer instead of the wait for expected condition, it works, but that is not what I want to use.
Here is the html:
<a target="_blank" href="/privateurl">OTD</a>
And here is the code that uses chromedriver:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import webbrowser
chrome_options = webdriver.ChromeOptions()
prefs = {'download.default_directory' : 'E:\\folder'}
chrome_options.add_experimental_option('prefs', prefs)
browser = webdriver.Chrome(chrome_options=chrome_options)
#browser.get('private url login page')
#enter login information and submit
wait = WebDriverWait(browser, 10)
wait.until(
EC.presence_of_element_located((By.LINK_TEXT, "OTD"))
)
browser.get('private url')
I have the same code, but for the Firefox driver and it works perfectly every time. I just need to use Chrome browser instead.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import webbrowser
profile = webdriver.FirefoxProfile()
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.download.dir", 'E:\\folder')
profile.set_preference("browser.download.useDownloadDir", True)
browser=webdriver.Firefox(profile)
#browser.get('private url login page')
#enter login information and submit
wait = WebDriverWait(browser, 10)
wait.until(
EC.presence_of_element_located((By.LINK_TEXT, "OTD"))
)
browser.get('private url')
What am I doing wrong in the code using the chromedriver that is making it throw a timeout exception?
There are a couple of things to address :
Finally as you are invoking click() on the WebElement so instead of the expected_conditions clause as presence_of_element_located() you should use the clause element_to_be_clickable(locator).
When you use expected_conditions clause as element_to_be_clickable(locator) the WebElement is returned back and you can directly invoke click() method on it.
Your optimized line of code will be :
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.LINK_TEXT, "OTD"))).click()
Update A
As per your comment update you are seeing the error :
'element_to_be_clickable' object has no attribute 'click'
An alternative would be to extract the href attribute and invoke get() as follows :
element = WebDriverWait(browser, 10).until(EC.visibility_of_element_located((By.LINK_TEXT, "OTD")))
attr_href = element.get_attribute("href")
driver.get(attr_href)
Update B
As per your comment update the idea of Selenium does interact with the element ... even though it is unable to attain visibility_of_element is not a full proof solution as :
The expected_conditions clause presence_of_element_located(locator) mentions :
class selenium.webdriver.support.expected_conditions.presence_of_element_located(locator)
An expectation for checking that an element is present on the DOM of a page. This does not necessarily mean that the element is visible.
locator - used to find the element returns the WebElement once it is located.
Where as the expected_conditions clause visibility_of_element_located(locator) mentions :
class selenium.webdriver.support.expected_conditions.visibility_of_element_located(locator)
An expectation for checking that an element is present on the DOM of a page and visible. Visibility means that the element is not only displayed but also has a height and width that is greater than 0.
locator - used to find the element returns the WebElement once it is located and visible.
From Selenium perspective if an element is not displayed Selenium won't be interact with the element e.g. invoking click()
I am getting very frustrated trying to login to costar.com with python and selenium. I have tried it on the chrome browser and firefox browser, but can't figure out the correct code.
I have logged into other websites, but cannot figure out how to input text into the login boxes for this site.
Here's what I have so far:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome(executable_path="C:/Users/Gus Gabel/Anaconda/chromedriver.exe")
browser.get("http://www.costar.com")
browser.maximize_window()
#the username and password boxes are hidden until you press the login button on the home page
login = browser.find_element_by_id("loginLink")
login.click()
#now that the username and password boxes are available, I've tried finding the elements by class_name, xpath, (BY.NAME), id, etc and nothing has worked
#here are a few codes that haven't worked and the errors associated
user = browser.find_element_by_id('username')
NoSuchElementException: Message: no such element
user = browser.find_element_by_class_name('usernameNew')
NoSuchElementException: Message: no such element
#when i try to use the above code with "elements" instead of "element", no error message pops up.
user = browser.find_elements_by_class_name('usernameNew')
#but then whey i try to choose which element by doing this
user = browser.find_element_by_class_name('usernameNew')[0]
NoSuchElementException: Message: no such element
How is it possible that there can be a list of elements, but yet not have an initial element?
If anyone can figure out how to input text into the username and password text boxes of costar.com, I will be greatly appreciative. I can't figure this out for the life of me!
To enter text into an input box with Selenium (e.g, your user name into the username field), use the send_keys method of the element:
input_element.send_keys('some_text_string')
You can also try Selenium's action_chains module, as in the following code (untested), to get past the roadblock from #dm295's answer:
from selenium import webdriver
from selenium.webdriver.common import action_chains
driver = webdriver.Firefox()
driver.maximize_window()
driver.get("http://www.costar.com")
action = action_chains.ActionChains(driver)
login = driver.find_element_by_class_name("login-icon")
login.click()
driver.switch_to.frame(driver.find_element_by_id('custlogin'))
username = driver.find_element_by_id('username')
action.move_to_element(username).perform()
username.send_keys('Gus Gabel')
A couple of things:
(a) you have to wait until the page is loaded (or at least the part of the page that you are interested in)
(b) only maximized browser window works for me (depends on device/resolution)
(c) you are trying to click the wrong element
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.maximize_window()
driver.get("http://www.costar.com")
try:
# wait 15 seconds till the login link is present
login = WebDriverWait(driver, 15).until(
EC.presence_of_element_located((By.CLASS_NAME, "login-icon"))
)
login.click()
# just sleep 5 seconds to show that the login link was clicked
time.sleep(5)
# do other stuff (probably fill in username and password) ...
finally:
driver.quit()
Read the documentation regards waiting in selenium.
EDIT
Manged to get one step further (switch to iframe), but the following code raises
selenium.common.exceptions.ElementNotVisibleException: Message:
Element is not currently visible and so may not be interacted with
from selenium import webdriver
driver = webdriver.Firefox()
driver.maximize_window()
driver.get("http://www.costar.com")
try:
login = driver.find_element_by_class_name("login-icon")
login.click()
# switch to the custlogin iframe
driver.switch_to.frame(driver.find_element_by_id('custlogin'))
username = driver.find_element_by_id('username')
# this will raise:
# selenium.common.exceptions.ElementNotVisibleException: Message:
# Element is not currently visible and so may not be interacted with
username.send_keys('username')
finally:
driver.quit()