I'm using Selenium on http://seattle.bestparking.com/ to click through the parking garage markers (red, grey, blue), with the intention of opening the pop-up info window, so I can then scrap the info (on the "rate" page in the pop-up window). See my code at the end.
However, it appears that each of the garage marker's inner HTML changes (!!), after you click any other garage marker of the same color, then closing it (due to content reloading).
E.g.: when running my code, "Motif Seattle" garage shows up as div id = daily_colored_marker_577. But after any other marker of the same color is clicked on, "Motif Seattle" shows up as div id = daily_colored_marker_1042 ... and keeps on changing.
This makes it seemingly impossible to iterate through all the markers that I've selected (result of find_elements_by_class), since it will throw the following type or error when reaching the 2nd element in the selected list:
WebDriverException: unknown error: Element <div
class="daily_colored_marker_n_a" id="daily_colored_marker_344" onmouseover="show_hide_balloonWindowMonthly('', 'IMPARK', 'IMPARK M Street
Garage', '344', 400, 'n_a', offsetPosition('daily_colored_marker_344')...
</div> is not clickable at point (708, 543). Other element would receive the
click: <div class="daily_colored_marker_grey"...
My code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Chrome()
driver.get('http://seattle.bestparking.com/')
driver.maximize_window()
'''Tells driver to wait until the "I understand" button to be clickable'''
wait = WebDriverWait(driver, 30)
element = wait.until(EC.element_to_be_clickable((By.XPATH, "//*
[#id='calendar_navigation_hint']/button")))
'''makes the disclaimer pop-up and the left menu go away, to reveal more of
the map'''
driver.find_element_by_xpath("//*#id='calendar_navigation_hint']
/button").click()
time.sleep(2)
driver.find_element_by_xpath("//*
[#id='map_left_panel_min_max_btn_div_oa']").click()
time.sleep(2)
'''zooms out one level'''
driver.find_element_by_xpath("//*
[#id='google_maps_zoom_control_minus_id']").click()
time.sleep(5)
'''find all the red markers and click on them'''
all_red = driver.find_elements_by_css_selector("div.daily_colored_marker_n_a")
time.sleep(3)
for x in range(0,len(all_red)):
all_red[x].click()
time.sleep(2)
driver.find_element_by_xpath("//*[#id='marker_window_close_text']").click()
time.sleep(3)
I don't think that error means what you think it means. If you look at it, it's stating that another element would receive the click. Selenium clicks on the center of an element. If you look at the map, there are some pins that cover each other. It's possible that it's trying to click the next element and it's partially covered by a different element so it throws the error.
Since I'm assuming you aren't trying to perform some user scenario, you can use JavascriptExecutor to execute the click and click the precise element.
You can see an example of how to do this in this question.
Related
I am clicking on a element that brings up a popup element, then I want to scrape data from this popup element. But I am not able to find the data or close the popup once I open it.
Looking at the element hierarchy, the popup doesn't seem to be in a different frame, but all my searching suggested that's what the problem is, so I tried below code
When I try to switch frames by element I am getting the same unable to locate element and the html data doesnt seem to show a frame related to this popup so I dont know any attributes of the frame to switch to it (I selected all frames and none of them highlighted the popup)
Here is the [webpage]
Clicking on "view details" under the picture shows the popup. How do I select elements on that popup?
data = []
e = driver.find_element_by_xpath(f'/html/body/div[1]/div[2]/div/div/div[5]/div[1]/div/div[4]/div/form[1]/div/div[2]/div[1]/a')
e.click()
e = driver.find_element_by_xpath('/html/body/div[12]/div[2]/div[4]/div/div/div/div/div[1]/div/button') ## Shows unable to locate element error
driver.switch_to.frame(e)
for x in range (1,5):
element = driver.find_element_by_xpath(f'/html/body/div[13]/div[2]/div[4]/div/div/div/div/div[3]/div/div[2]/div/div[1]/ul/li[{x}]')
data.append(element.text) ## Doesnt have the text inside the list
e.click()
driver.switch_to.default_content()
The elements are not in an iframe. So no need to switch to frame. The details appear in a pop-up.
Instead of Absolute xpaths opt for Relative xpaths. Link to refer
Try like below and confirm:
# Imports required for Explicit wait:
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver.get("https://koa.com/campgrounds/austin-east/reserve/step2/?res=kI2O68dJS8X6si949wGzFa68NxYqwyCBul7NzF70cO3rQJLWCoDqp38g67l9SA4BcZqoRAxVFUdFtWBLcjbhl55toq-KCG20N36F-ytOHhr3EKHiJ3_qxDvkUMId1flJiRSh-9X6BDSw3EJ_IVCTNjo0ax44UscxXmfNXAVc8WybZ2xp6tu6H9lvR_igbYAXMf2t2dyN_NM-8JDDtJEe1ofkmQmsl1QJPJsZqfNq-cMnDD2mTdNCJxsfUQwEUYrUY6QEsdj3bbLMWqSTRW5bSzEoVNjVNc_dbH4R11-nnZU0tt1uvWNZZH-3OmrZA7Of9BI2W-GO6r9fk2sa0l1AaiCX7yclG2kXAQgxwTTMOfqI6OgYtKDRvEAgHW71w2LycESK3xVuxTP8C_4OwhFEQBg-GYcq-juoe5qeZNBhiBJsWJxqqxpcPBJ4RgeL7l-emnlG9DHqFk-R3LRDnAi2RBXsr9UWmcVwa3vtlXLLHPcJp07uBDPKz4d-Mm2nKPcjlQVQhaN9ooT5Ccvr5LergYRv4XRMI-u6z0gDYMpZUwLSFvQt3wUEXmQyM0PaazwX1fKjUVF9oKnU8yH5BGpy87uR3_QxfE7UP864PWqdaDg6ni3o7qOLnhZGJ-K6Z0oVu9f48BKRPWWX8-pvHRhe7Is6XnxD1_XVWxjAfXeWAxD8R0URqU6v8JvCgKZXJvwP0vjsXLeXtSsEEd2VFyrGk5UHhHuG1cc7YFGTXJBDMPW7E88LI7pkbsecM-Va5X4IrtoHLppMBOeihBj3Cxz9nHLF5X6fkIMRE7BWKh8TQxRTyNhvdNg-zlMAep2boMwQmBbFpyCH8VN_UxroErjaAixJsEdJyphoYPuIkyHOHfvxREsu5A5IR8aowrQMv1Kc04_bMU-BacYCTCAB8jeRv-I3CtuqetXls4DzC49crLDFyewmwIUdUkkWJS79ml_L3vh82vqjJfHnYs5GHYmqdGLlFPBD19hIjcrAcuOTNXFH5et8ShMfMaa1TRSJCTqeDmonbdH2Uv-_u1RAu_SMHDik2N2NCfEi4E-GvN6vw_kvce_yrLyzeVRCseHo0JmgWxpc4C9nwcLsxa7qN16Q6Gh9aa8RP3Od80tDN8SOOLeVStjcKopJU91ui1rSIdv4pncKWImnwOm7xBG2srVefDb7twP--WYZWeX5PyumDLw9-Nb_QN-LpET3ELxsF0eY6GmFYLFWdt38Aosc4hNFzDFQEkZ5KDnbLdvfMeKqXKDGdV9e5OaaynLyQNmvp4ZlLEVdD3LUraSw_TB-JClxIiw9AIYdhn1vlmilk2CsDIbyfF4n2WsxoVs0k0_RsiAcqaL6jEx31_shLwpGidHXgVUYrNaQ3URMf2Y_KUqB9RPIh0e8_aGoFxbthoxKQmyLMRtLutwvwmqQXx5sCEAfwQ2")
wait = WebDriverWait(driver,30)
# Options available on the page
options = driver.find_elements_by_xpath("//div[#id='reserve-container']//div[#id='site-types']")
# Get details of the 1st option.
viewdetails = options[0].find_element_by_xpath(".//div[contains(#class,'reserve-sitetype-main-row')]//a")
viewdetails.click()
# Get the heading
heading = wait.until(EC.visibility_of_element_located((By.XPATH,"//div[#class='fancybox-stage']//h3")))
print(heading.text)
# Get the details under the heading
details = driver.find_elements_by_xpath("//div[#class='fancybox-stage']//h3//parent::div//following-sibling::div//li")
for detail in details:
print(detail.get_attribute("innerText"))
# Close the pop-up window
driver.find_element_by_xpath("//div[#class='fancybox-stage']//button[contains(#class,'wts-details-close')]").click()
AUSTIN EAST KOA HOLIDAY
6029 Blue Bluff Road Austin, TX 78724
Email: austineast#koa.com
Reserve: 512-732-2812
Info: 800-562-8743
I have tried to make my script click the Purchase/Buy Family button on the Spotify Checkout Page. No matter what class, CSS, XPath, ID, or whatever I put in, it's just saying it could not find the object.
This is the button. It's not in an iframe:
<div class="sc-fzXfOu cvoJMt">
<button id="checkout_submit" class="Button-oyfj48-0 kaOWUo sc-fzXfOv tSdMK">
Buy Premium Family
</button>
</div>
My code:
time.sleep(3)
buy = driver.find_element_by_xpath("/html/body/div[3]/div/div/div/div/div/div/div[3]/div/div/div[2]/form/div[2]/button").click()
I am able to click the button,by a different xpath
driver.findElement(By.xpath("//button[#id='checkout_submit']")).click();
Edit -
Your xpath also works for me only when i load the page initially and there is no change in the dom - /html/body/div[3]/div/div/div/div/div/div/div[3]/div/div/div[2]/form/div[2]/button
It doesn't work when some new event or error is displayed and the dom structure changes.
Why use such relative XPath when the element has distinguishable attributes?
The problem here is the form is not static you have to wait loading of all elements.
And page loads with pane to accept cookies who can mask elements you want to run action on.
The best way in that case is first accept cookies and then run all actions you need.
Try to adapt this code for your needs, that run with success on my test.
In case you run this code you have to brake execution to login first, when the driver get the page.
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.remote.webelement import WebElement
# change this line
path_driver = "your_path_to_chrome_driver"
by, buy_selector, cookies_selector = By.CSS_SELECTOR, 'button#checkout_submit', "button#onetrust-accept-btn-handler"
driver = webdriver.Chrome(path_driver)
driver.maximize_window()
actions = ActionChains(driver)
driver.get("https://www.spotify.com/us/purchase/offer/premium-family/?country=US")
# wait for loading buy button
sls = wait.until(EC.presence_of_all_elements_located((by, buy_selector)))
if sls:
# get accept cookies button element and click
cookies_accept = driver.find_element_by_css_selector(cookies_selector)
if isinstance(cookies_accept, WebElement):
cookies_accept.click()
# get buy button element, move to element and click
buy = driver.find_element_by_css_selector(buy_selector)
if isinstance(buy, WebElement) and buy.is_displayed() and buy.is_enabled():
actions.move_to_element(buy).click(buy).perform()
I have a problem navigating a website using selenium. This is my code:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('https://webtrader.binary.com/v2.2.8/main.html#')
resources = driver.find_element_by_id('ui-id-1')
resources.click()
However, I get the exception:
selenium.common.exceptions.ElementNotInteractableException: Message: Element <ul id="ui-id-1" class="ui-menu ui-widget ui-widget-content ui-menu-icons"> could not be scrolled into view
I don't understand where I went wrong. I am trying to access 'Historical data' from the dropdown menu labeled "Resources". Could someone please help me access it. Maybe I got the id for Resources wrong. You could also check that out.
The element you want to click to open the dropdown is the previous sibling of the element resources
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.support.select import By
from selenium.webdriver.support.ui import WebDriverWait
wait = WebDriverWait(driver, 10)
driver.find_element_by_class_name('resources').click()
historical_data = wait.until(ec.visibility_of_element_located((By.ID, 'ui-id-4')))
There are many things happening here. First of all, your code is lacking a wait. Without it will always fail, as the page is dynamically loaded. Read about Waits.
Secondly, here resources = driver.find_element_by_id('ui-id-1') you are finding the element from the dropdown menu, and then you are trying to click it. But the dropdown menu is not opened. You should click on it, then wait for the option to appear, only then click on the 'Historical data'.
I've written a script in python in combination with selenium to get some information from a webpage. To reach the content it is necessary to hurdle few steps, as in accepting condition, fill in the inputbox, click on the search button to populate results and finally click on the first grid (the first tr, more specifically) within the populated table. As soon as any click is initiated on the first tr, a new page (containing desired information) opens up.
My script can do the first three steps successfully. What I can't do are:
perform a click on the first tr
focus to the newly opened tab (containing information I'm after)
To reach the content:
This is the link to follow. There is an accept button to click first. Then there is an inputbox Name to be filled in with HMC DESIGN GROUP. Now, pressing the search button, the result should appear below within a table. From there I need to click on the first tr.
This is what I've tried so far:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
link = "https://officialrecords.broward.org/AcclaimWeb/search/SearchTypeName"
def get_information(driver,url):
driver.get(url)
wait.until(EC.element_to_be_clickable((By.ID, "btnButton"))).click()
wait.until(EC.presence_of_element_located((By.ID,"SearchOnName"))).send_keys("HMC DESIGN GROUP")
wait.until(EC.presence_of_element_located((By.ID, "btnSearch"))).click()
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,".t-grid-content table tr"))).click()
if __name__ == "__main__":
driver = webdriver.Chrome()
wait = WebDriverWait(driver,10)
try:
get_information(driver,link)
finally:
driver.quit()
Currently the script neither clicks on the grid the first tr of the newly generated table nor throws any error. It quits the browser gracefully.
There is another table under the div with the same class name "t-grid-content". It appears as "Loading...". So after you submit the search you really make a click on node with selector ".t-grid-content table tr", but it just don't make a proper effect
You just need more specific selector.
Try, for instance
wait.until(EC.element_to_be_clickable((By.XPATH, "//td[contains(., 'HMC DESIGN GROUP')]")))
To switch to new window, try update your function body as
driver.get(url)
current = driver.current_window_handle
wait.until(EC.element_to_be_clickable((By.ID, "btnButton"))).click()
wait.until(EC.presence_of_element_located((By.ID,"SearchOnName"))).send_keys("HMC DESIGN GROUP")
wait.until(EC.presence_of_element_located((By.ID, "btnSearch"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//td[contains(., 'HMC DESIGN GROUP')]"))).click()
driver.switch_to.window([window for window in driver.window_handles if window != current][0])
Just replace this css selctor :
.t-grid-content table tr
to this :
.t-grid-content table tr.t-state-selected:first-child
for clicking on first tr.
Before clicking on first tr, store the window handle as :
window_before = driver.window_handles[0]
after clicking on first tr, store the window handle of newly opened window as:
window_after = driver.window_handles[1]
Now all you have to do is to switch the focus of your web driver to newly opened windows.Like this :
driver.switch_to_window(window_after)
This is a follow-up question to this:
WebDriver element found, but click returns nothing
I am trying to scrape data from the URL in the code after making selections in the drop-down menu. I first click on Progress Monitoring and then Physical and Financial Project Summary. Then I make the following selections: State, District, Block, Year, Batch, and Collaboration. I would also like to check the Road Wise button and then click on the view button. After the table loads, I would like to click on the save button and download the excel file. In the code below I also loop through different selections under "State" item. Here is my code:
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import time
import os
chromedriver = r"C:\Users\yuppal\chromedriver"
os.environ["webdriver.chrome.driver"] = chromedriver
browser = webdriver.Chrome(chromedriver)
browser.implicitly_wait(10)
browser.get("http://omms.nic.in")
browser.maximize_window()
#Click on the item Progress Monitoring
progElem = browser.find_element_by_link_text("Progress Monitoring").click()
#Click on the item Physical and Financial Project Sumamry
summElem = browser.find_element_by_link_text("Physical and Financial Project Summary").click()
#Find the element for state and create a list of different selection options
stateElem = browser.find_element_by_xpath("//select[#name='StateCode']")
state_options = stateElem.find_elements_by_tag_name("option")
#delete the first option in the list
del state_options[0]
def select_option(xpath, text):
'''
This function will select the remaining dropd-down menu items.
'''
elem = browser.find_element_by_xpath(xpath)
Select(elem).select_by_visible_text(text)
#run the loop for each option in the list of states
for option in state_options:
select_state = Select(stateElem).select_by_value(option.get_attribute("value"))
# Select the district.
select_option("//select[#name='DistrictCode']","All Districts")
# Select the block.
select_option("//select[#name='BlockCode']","All Blocks")
# Select the year.
select_option("//select[#name='Year']","All Years")
# Select the batch.
select_option("//select[#name='Batch']","All Batches")
# Select the funding agency.
select_option("//select[#name='FundingAgency']","Regular PMGSY")
# Check the road wise box.
time.sleep(10)
checkElem = WebDriverWait(browser, 120).until(EC.element_to_be_clickable((By.XPATH, "//input[#title='Road Wise']")))
browser.execute_script("arguments[0].click();", checkElem)
# Click on the view button.
time.sleep(10)
browser.find_element_by_xpath("//input[#type='button']").click()
# Switch to a new frame.
time.sleep(10)
frame = browser.find_element_by_xpath("//div[#id='loadReport']/iframe")
browser.switch_to.default_content()
#browser.switch_to.frame(frame)
WebDriverWait(browser, 120).until(EC.frame_to_be_available_and_switch_to_it(frame))
#browser.switch_to.frame(browser.find_element_by_xpath("//*[#id='loadReport']/iframe"))
# click on the save button
time.sleep(10)
WebDriverWait(browser, 120).until(EC.element_to_be_clickable((By.XPATH, "//a[#title='Export drop down menu']"))).click()
# Within the save button, Click on the "Excel" option.
time.sleep(10)
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//div/a[#title='Excel']"))).click()
# Switch back to the main content.
time.sleep(20)
browser.switch_to.default_content()
My issue is the "Road Wise" checkbox gets clicked only for some states. Thus the loop proceeds without clicking the checkbox for some states. I checked the HTML code and it is the same for all checkboxes.
I thought the problem might be that the "View" button gets clicked before the road wise button is clickable. So I put some waiting period before both road wise and view buttons. But that doesn't seem to help. So I can't really understand why the checkbox button isn't clicked for some iterations in the loop.
Before clicking on the checkbox, check that is already selected or not:
# Check the road wise box.
time.sleep(10)
checkElem = WebDriverWait(browser, 120).until(EC.element_to_be_clickable((By.XPATH, "//input[#title='Road Wise']")))
if checkElem.is_selected() != True:
browser.execute_script("arguments[0].click();", checkElem)
PS: In your case, the click will be only in the first iteration of the loop.