I'm trying to fill out a form on a website that has different versions depending on language, location, etc. I am using the same firefox profile for every request, and the previously selected information is stored locally in cookies, so after choosing a setting the first time, the modal doesn't appear for a while. However: it seems to inconsistently appear, taking focus away from the form and causing an ElementNotInteractableException.
To add more difficulty, the modal will often appear some time after the page has loaded. For example, the first field will already be filled out, and then it appears.
My question is, what is the best way to handle this modal? Can I catch the exception cause by its appearance, check for the presence of the modal, and then continue populating the form fields? Or is there a better solution?
Thanks for any help.
The code I have tried so far:
url = "https://www.aircanada.com/ca/en/aco/home.html"
control_profile = webdriver.FirefoxProfile('/path/to/my/profile')
browser_control = webdriver.Firefox(control_profile)
browser_control.get(url)
# To deal with the modal, but obviously fails when it is not present
browser_control.find_element_by_id('enCAEdition').click()
# two text fields I tried to fill out, as a sanity check
departure = browser_control.find_element_by_id('origin_focus_0')
departure.send_keys("my departure location")
departure.send_keys(Keys.RETURN)
destination = browser_control.find_element_by_id('destination_label_0')
destination.send_keys("my destination")
destination.send_keys(Keys.RETURN)
You can wait some time until modal appears, close it and handle form:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
url = "https://www.aircanada.com/ca/en/aco/home.html"
control_profile = webdriver.FirefoxProfile('/path/to/my/profile')
browser_control = webdriver.Firefox(control_profile)
browser_control.get(url)
# Wait up to 10 seconds until modal appears to close it
try:
wait(browser_control, 10).until(EC.element_to_be_clickable(('xpath', '//button[text()="Confirm | Confirmer"]'))).click()
# If modal didn't appear- just continue
except TimeoutException:
pass
departure = browser_control.find_element_by_xpath('//input[#placeholder="FROM"]')
browser_control.execute_script('arguments[0].setAttribute("class","glyph-input glyph-left-input form-control ng-pristine ng-valid ng-touched");', departure)
departure.send_keys("Berlin")
wait(browser_control, 5).until(EC.visibility_of_element_located(("xpath", "(//div[#class='location-primary']/span)[1]")))
departure.send_keys(Keys.RETURN)
destination = browser_control.find_element_by_xpath('//input[#placeholder="TO"]')
browser_control.execute_script('arguments[0].setAttribute("class","glyph-input glyph-left-input form-control ng-pristine ng-valid ng-touched");', destination)
destination.send_keys("Oslo")
wait(browser_control, 5).until(EC.visibility_of_element_located(("xpath", "(//div[#class='location-primary']/span)[4]")))
destination.send_keys(Keys.RETURN)
Related
I'm trying to scrape data by python from this e-commerce site
Because it requires to select the shipping location first to access the data and the 3 selects have the same xpath so I use the code below
city = browser.find_element(By.XPATH,"(//select[not(#id) and not(#class)])[1]")
citydd = Select(city)
citydd.select_by_value('01') # Hanoi
time.sleep(1)
district = browser.find_element(By.XPATH,"(//select[not(#id) and not (#class)])[2]")
districtdd = Select(district)
districtdd.select_by_value('0101') # Ba Dinh
time.sleep(1)
ward = browser.find_element(By.XPATH,"(//select[not(#id) and not (#class)])[3]")
warddd = Select(ward)
warddd.select_by_value('010104') # Cong Vi
browser.find_element(By.XPATH,"//div[text()='Xác nhận']").click() # Xac nhan
It returns me this error
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"(//select[not(#id) and not(#class)])[1]"}
May I know how to bypass this situation?
There is the ability to select better xpaths. You can use a relative xpaths using the label of associated select
//label[contains(text(),'Tỉnh/Thành phố')]/following-sibling::div/select
//label[contains(text(),'Quận/Huyện')]/following-sibling::div/select
//label[contains(text(),'Phường/Xã')]/following-sibling::div/select
This is the middle one identified as unique using the above:
If you're still getting no such error with these xpaths - please ensure you include explicit or implicit waits
Selenium's default wait strategy is the "the page has loaded". Most often in modern pages, the page loads, THEN scripts run which get more data or display a modal (like the popup on the image). Those async calls cause fails as nosuchelements in selenium.
Let me know if you need more information on sycnhronisation.
This is what i have tried -
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.wait import WebDriverWait
from time import sleep
from selenium import webdriver
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get('https://vinmart.com/')
FirstDropDown = Select(driver.find_element_by_xpath("(//select)[1]"))
FirstDropDown.select_by_index(1)
sleep(2)
SecondDropDown = Select(driver.find_element_by_xpath("(//select)[2]"))
SecondDropDown.select_by_index(1)
sleep(2)
ThirdDropDown = Select(driver.find_element_by_xpath("(//select)[3]"))
ThirdDropDown.select_by_index(1)
I have used sleep() because it will take time to populated data in the dropdown as per pervious dropdown selection.
Please mark it as answer if it resolves your problem.
I have tried to make my script click the Purchase/Buy Family button on the Spotify Checkout Page. No matter what class, CSS, XPath, ID, or whatever I put in, it's just saying it could not find the object.
This is the button. It's not in an iframe:
<div class="sc-fzXfOu cvoJMt">
<button id="checkout_submit" class="Button-oyfj48-0 kaOWUo sc-fzXfOv tSdMK">
Buy Premium Family
</button>
</div>
My code:
time.sleep(3)
buy = driver.find_element_by_xpath("/html/body/div[3]/div/div/div/div/div/div/div[3]/div/div/div[2]/form/div[2]/button").click()
I am able to click the button,by a different xpath
driver.findElement(By.xpath("//button[#id='checkout_submit']")).click();
Edit -
Your xpath also works for me only when i load the page initially and there is no change in the dom - /html/body/div[3]/div/div/div/div/div/div/div[3]/div/div/div[2]/form/div[2]/button
It doesn't work when some new event or error is displayed and the dom structure changes.
Why use such relative XPath when the element has distinguishable attributes?
The problem here is the form is not static you have to wait loading of all elements.
And page loads with pane to accept cookies who can mask elements you want to run action on.
The best way in that case is first accept cookies and then run all actions you need.
Try to adapt this code for your needs, that run with success on my test.
In case you run this code you have to brake execution to login first, when the driver get the page.
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.remote.webelement import WebElement
# change this line
path_driver = "your_path_to_chrome_driver"
by, buy_selector, cookies_selector = By.CSS_SELECTOR, 'button#checkout_submit', "button#onetrust-accept-btn-handler"
driver = webdriver.Chrome(path_driver)
driver.maximize_window()
actions = ActionChains(driver)
driver.get("https://www.spotify.com/us/purchase/offer/premium-family/?country=US")
# wait for loading buy button
sls = wait.until(EC.presence_of_all_elements_located((by, buy_selector)))
if sls:
# get accept cookies button element and click
cookies_accept = driver.find_element_by_css_selector(cookies_selector)
if isinstance(cookies_accept, WebElement):
cookies_accept.click()
# get buy button element, move to element and click
buy = driver.find_element_by_css_selector(buy_selector)
if isinstance(buy, WebElement) and buy.is_displayed() and buy.is_enabled():
actions.move_to_element(buy).click(buy).perform()
I Got this code:
from selenium import webdriver
from selenium.webdriver.support.ui import Select
myurl = "https://foobar.pl"
driver = webdriver.Chrome()
driver.get(myurl)
select = Select(driver.find_element_by_xpath('/html/body/div/select'))
select.select_by_visible_text('foobar')
time.sleep(5)
after selecting "foobar" an input field appears.
but after I try:
driver.find_element_by_xpath('/html/body/div/div[2]/input').click()
I get
ElementNotVisibleException: element not visible
How can I update driver, so it would see input, without refreshing page (I would loose my selection)?
Can you please try the code snippet below. You might be needing to wait between clicking and getting the field that you would like to retrieve.
select = Select(driver.find_element_by_xpath('/html/body/div/select'))
select.select_by_visible_text('foobar')
time.sleep(5)
driver.find_element_by_xpath('/html/body/div/div[2]/input').click()
You first need to click to the field, then wait, then click again.
This commonly occurs when JavaScript modifies the page after you interact with it. The solution is to use WebDriverWait:
from selenium.webdriver.support import ExpectedConditions as EC
wait = WebDriverWait(driver, 5)
input = wait.until(EC.element_to_be_clickable(By.xpath('/html/body/div/div[2]/input'))
input.click()
I want to create 10 emails on ProtonMail.
I already automated half of the part, using PyAutoGui and Selenium but I want to make it like a checker because sometimes usernames are taken.
Now what I wanna do is this:
Generate Random Usernames
Check the usernames either by Selenium or another package(suggestions if you know any)
If the username is valid, a notepad will be created to save the valid usernames.
This is the concept of what I am trying to build now. I am truly sorry if I sound garbage but I legit started using Python a few days ago and it hasn't yet been a week so... I'm learning :P
I have automated the part where Selenium fills the form of ProtonMail for Sign In / Sign Up, but sometimes I get the error that the Username is already taken and I want the script to check if that error message pops up, and if it does, a "reserved code" line will be executed to fix the issue. Then, the code can continue. But, I want the script not to interfere with the "reserved code" if the element doesn't pop up.
If anyone is here just to get the code ready, here you go:
import selenium
import pyautogui
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
#Variables
protonmail = "https://mail.protonmail.com/create/new?language=en"
username = input("Please enter your desired username for the email:")
password = input("Enter your password:")
driver = webdriver.Firefox()
time.sleep(4)
driver.get(protonmail)
time.sleep(7)
pyautogui.click(535, 501)
time.sleep(1)
pyautogui.typewrite(username)
time.sleep(2)
driver.find_element_by_xpath(
"/html[1]/body[1]/div[2]/div[1]/div[1]/div[1]/div[1]/form[1]/div[1]/div[2]/div[1]/div[1]/input[1]").send_keys(
password)
time.sleep(2)
driver.find_element_by_xpath(
"/html[1]/body[1]/div[2]/div[1]/div[1]/div[1]/div[1]/form[1]/div[1]/div[2]/div[2]/div[1]/input[1]").send_keys(
password)
time.sleep(2)
pyautogui.click(1284, 916)
time.sleep(2)
pyautogui.click(655, 762)
time.sleep(3)
You can use the request library to check if the username is valid:
import requests
URL = "https://mail.protonmail.com/api/users/available"
PARAMS = {"Name": "UsernameToCheck"}
# idk what these are but it seems like they are needed
HEADERS = {"x-pm-appversion": "Web_3.16.17",
"x-pm-apiversion": "3"}
r = requests.get(url=URL, params=PARAMS, headers=HEADERS)
if int(r.json()["Code"]) == 1000:
print("valid username")
else:
print("invalid username")
I've just checked the ProtonMail sign up page to locate the error message you are talking about. Based on your problem description, it seems like you want to proceed differently in your code based on the presence of this error message. Your code sample is a bit hard to read, because your clicks are all on absolute coordinates, rather than actual WebElements, so I'm not entirely sure what is being clicked throughout your example. This might be a good starting point for you:
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# Fill in all details on form
# Click Create Account - I assume you have already done these two steps
# Check for error message -- this is in an iframe
# switch to first iframe on the page which will contain the error message
iframe = driver.find_elements_by_xpath("//iframe[#title='Registration form']")[0]
# attempt to find the error message, catch the exception if it does not exist
try:
# Handle scenario where error message exists -- username is taken
error_message = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//p[text()='Username already used']")))
# better to write a handler method, instead of stuffing code into the try block
call_some_handler_method_here()
except TimeoutException:
# Handle scenario where error message does NOT exist -- meaning, username is not taken
call_some_other_handler_method_here()
This code will switch to the iframe which contains the error message for username already taken -- this is necessary for driver to locate the element. After that, induce WebDriverWait in a try / except block to check if the error message exists or not.
If the error message is present, you will end up inside the try block, where you can call a method to proceed accordingly.
If the error message is NOT present, meaning the username is not taken, you will end up in the except block, where you can call a different method to proceed accordingly (such as saving the attempted username to a file).
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
import os
Game_Pin = input('Enter your PIN: ')
NickNAME = input('Enter your nickname: ')
driver = webdriver.Chrome(executable_path=r"C:\WebDriver\bin\chromedriver.exe")
def Enter_Press(driver):
driver.find_element_by_xpath("//*[contains(text(), 'Enter')]").click()
def OK_GO(driver):
driver.find_element_by_xpath("//*[#class='btn btn-greyscale join ng-binding']").click()
def Kahoot_Spammer(Game_Pin, NickNAME, driver):
driver.get('https://kahoot.it/')
driver.maximize_window() #For maximizing window
driver.implicitly_wait(2) #gives an implicit wait for 2 seconds
game_pin = driver.find_element_by_xpath("//*[#id='inputSession']")
game_pin.send_keys(Game_Pin)
Enter_Press(driver)
driver.implicitly_wait(2)
Name = driver.find_element_by_xpath("//*[#id='username']")
Name.send_keys(NickNAME)
OK_GO(driver)
Kahoot_Spammer(Game_Pin, NickNAME, driver)
The program works fine until you get to a certain point. Then selenium cannot click a button. It gives me this error.
selenium.common.exceptions.WebDriverException: Message: unknown error: Element <button type="submit" class="btn btn-greyscale join ng-binding" blocking="" data-functional-selector="join-button-username">...</button> is not clickable at point (1279, 741). Other element would receive the click: <div id="waitOverlay" class="alert-fullscreen valignwrapper" data-functional-selector="wait-overlay" style="opacity: 0.7;">...</div>
Here is the code for the button I am trying to click.
<button class="btn btn-greyscale join ng-binding" type="submit" data-functional-selector="join-button-username" blocking=""> OK, go! </button>
You can use action class to resolve this exception,
action=ActionChains(driver)
action.move_to_element("Your element").click().build().perform()
You use implicitly_wait as a wrong way. In selenium implicitly wait is for setting the find duration of all selenium find API, like find_element_by_xxx. Rather than used for waiting page refresh/load complete.
From the error message we can know, when you click on the Go button, but another element (A layer display loading which at top of the GO button ) received the click event.
So you need to wait the loading layer disappear before click anything covered by it.
Enter_Press(driver)
// driver.implicitly_wait(2)
// remove above 'implicitly_wait' line,
// you set the find duration to 2 seconds, the default value is 10 seconds.
// we don't recommend to change this default value, unless you are doing
// performance testing to expect the element comes out on page in short time.
// Selenium will return the find result once it found before 10 seconds,
// otherwise throw `NoSuchElementException` if exceed 10 seconds.
diver.wait(xxx) // or simply use sleep() to wait the `loading` disappear.
Name = driver.find_element_by_xpath("//*[#id='username']")
Name.send_keys(NickNAME)
OK_GO(driver)