I'm trying to find a specific element on a page.
The page will automatically change the page content after the page is loaded and after certain validations.
So, after the page is loaded, we try to determine whether the page has been loaded or not based on whether there is a specific element in the changed page content.
but it doesn't work
I want to do the following code is executed If the element is found within the specified time.
If I can't find it by that time, Close Selenium Object and the script.
#Waiting for detect Element
try:
UserListElement = WebDriverWait(browser, 5).until(
EC.presence_of_element_located((By.ID, "mainUserSearchDiv"))
)
except TimeoutException: #Can't find element during time.
print("Cannot find element")
browser.close #Close selenium.
finally: #Can find Element
print("Complite Load Page")
Looks like your problem is with browser.close.
Instead of close it should be close().
So please change from
browser.close
to
browser.close()
And your code should be correct
Related
I am attempting to write a for loop to select product UPC's on a website. Essentially I have a list of UPC's in an Excel file that I want to loop through to EITHER: (1) Find the item through the search bar and select it OR (2) Item is not found and I want to add it to a list of Not Found UPCs. This is where I am at so far. I am unsure if I should use an IF ELSE statement or something else. I would really appreciate any thoughts or guidance.
for upc in upc_list:
driver.find_element_by_xpath('xpath for search bar').clear()
driver.find_element_by_xpath('xpath for search bar').send_keys(upc)
WebDriverWait(driver, 10).until(ec.visibility_of_element_located((By.XPATH, '//*[#id="'+upc+'"]'))).click()
if the element is not found an exception will be thrown and you have to catch it with a try except
upc_list=[]
not_found_list=[]
for upc in upc_list:
driver.find_element_by_xpath('xpath for search bar').clear()
driver.find_element_by_xpath('xpath for search bar').send_keys(upc)
try:
WebDriverWait(driver, 10).until(ec.visibility_of_element_located((By.XPATH, '//*[#id="'+upc+'"]'))).click()
except:
print('no results for '+upc)
not_found_list.append(upc)
continue
print('results for '+upc)
# utc is found
# do what ever you want with it
I'm trying to scrape every item on a site that's displayed in a grid format with infinite scrolling. However, I'm stuck on even getting the second item using xpath because it's saying:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[#class='el-card advertisement card is-always-shadow'][2]"}
x = 1
while x < 5:
time.sleep(5)
target = driver.find_element_by_xpath(f"//div[#class='el-card advertisement card is-always-shadow'][{x}]")
target.click()
wait.until(EC.visibility_of_element_located((By.ID, "details")))
print(driver.current_url)
time.sleep(5)
driver.back()
time.sleep(5)
WebDriverWait(driver, 3).until(EC.title_contains("Just Sold"))
time.sleep(5)
x += 1
With my f-string xpath it's able to find the first div with that class and print the URL, but the moment it completes one iteration of the while loop, it fails to find the 2nd div with that class (so 2).
I've tried monitoring it with all the time.sleep() to see exactly where it was failing because I thought maybe it was running before the page loaded and therefore it couldn't be found, but I gave it ample time to finish loading every page and yet I can't find the issue.
This is the structure of the HTML on that page:
There is a class of that name (as well as "el-card__body" which I have also tried using) within each div, one for each item being displayed.
(This is what each div looks like)
Thank you for the help in advance!
(disclaimer: this is for a research paper, I do not plan on selling/benefiting off of the information being collected)
Storing each item in a list using find_elements_by_xpath, then iterating through them did the trick, as suggested by tdelaney.
So I have a selenium script that will automatically enter a series of numbers into a website, and the website will redirect the user to another website based off if the numbers match a PIN. However, the browser takes a short time to redirect the user, in which the next line of code would have already run and returned an error.
I was thinking something like this would work but it doesn't, I don't know why.
def checkElement():
try:
xpath = '//*[#id="name"]'
print("Page is ready!")
except TimeoutException:
print("failed")
checkElement()
I believe that you are looking for WebDriverWait. You can add specific condition in it. Please find the sample code below.
first_result = wait.until(presence_of_element_located((By.XPATH, "//*[#id='name']")))
I'm trying to go on a webpage,
save a set of links of the page I would like to click on, and then
I would like to click on each of the link if a for loop (going back and forth on the page. Here is the code:
from selenium import webdriver
driver = webdriver.Chrome(executable_path='/Applications/chromedriver')
driver.get("webpage link") #insert link to webpage
list_links = driver.find_elements_by_xpath("//a[contains(#href,'activities')]")
for link in list_links:
print(link)
link.click()
driver.goback()
driver.implicitly_wait(10) # seconds
driver.quit()
However, the first time I go back to the homepage I get the error message:
StaleElementReferenceException: stale element reference: element is not attached to the page document.
Can anyone help me to understand why? Suggest a solution?
Thank you. much appreciated.
Your list_links works only on page where it was defined. After you made first click on link DOM re-created and references to elements of list_links becomes invalid. You can apply below solution:
driver.implicitly_wait(10) # seconds
list_links = [link.get_attribute('href') for link in driver.find_elements_by_xpath("//a[contains(#href,'activities')]")]
for link in list_links:
print(link)
driver.get(link)
driver.goback()
driver.quit()
P.S. I assume that goback() is your custom method that was defined already as there is no such method in Selenium built-ins, but just back()
P.P.S. Note that you can call driver.implicitly_wait(10) only once in your code and it will be applicable for all next find_element...() calls
Its simple, you trying to save the elements of an html(links) which cannot be referenced anymore by the code(the loop logic), thats why it throws this error. Most of all those are selenium objects you trying to save which you should not do. It should be that you save the exact value of the link in an array and then loop them.
I'm having problem selecting the iframe and accessing the different elements inside it. The iframe name is dynamically generated (e.g. frame11424758092173 or frame0005809321 or frame32138092173). The problem is that Selenium can't find the iframe no matter what i do....
switching to most recent frame doesn't work:
iframe = driver.find_elements_by_tag_name('iframe')[0]
driver.switch_to_frame(iframe)
Waiting for frame gets a timeout exception:
try:
iframe = WebDriverWait(driver, 5).until(EC.frame_to_be_available_and_switch_to_it(By.TAG_NAME('iframe')))
except:
logger.error(traceback.format_exc())
The following lines of code also times out:
try:
iframe = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.TAG_NAME, u"iframe")))
driver.switch_to_frame(iframe)
except:
logger.error(traceback.format_exc())
I have also tried iterating through the frames but it can't find it. The returned list is empty
iframes = driver.find_elements_by_tag_name('iframe')
#iframes is empty
really need some help...
Have you tried locating the iframe by its XPath and using the contains method?:
iframe = driver.find_element_by_xpath('//iframe[contains(#name, "frame")]')
driver.switch_to_frame(iframe)
Now you can access elements within the iframe.
To exit the iframe use:
driver.switch_to_default_content()
The contains method lets you get an element by a partial attribute value. Pretty useful for dynamically generated IDs, names, etc. You can search by other attributes as well using XPath. For example, say your iframe element has the attribute value = "3". You could use:
iframe = driver.find_element_by_xpath('//iframe[contains(#name, "frame")][#value = "3"]')
driver.switch_to_frame(iframe)
This approach can be used with any number of attributes as well.
You could also try getting the element by its selector. Keep in mind that this limits what you can do with it:
driver.execute_script('document.querySelector("INSERT SELECTOR HERE").doSomething();')
To get the Selector and/or XPath you're going to want to inpect the element using your browser (Chrome in my case). Right click on the element. Click Inspect. Then right click on the HTML element and click Copy > Copy Xpath or Copy > Copy Selector.
If that doesn't work for me, my last resort is to go the url of the iframe.To get that, you need to right-click on the area of the webpage where the iframe exists and click View Frame Source. It'll then lead you to a new page. The url of that page will be shown in the top of the browser after view-source:. You can then simply navigate to that url:
driver.get('insert url of iframe here')
And now you have access to the elements within the iframe. I do not recommend this approach if you are manipulating elements within the iframe and then exiting the iframe. Your changes will get lost. This will only work if you are scraping info off of that iframe, NOT if you are manipulating the elements within. Finding the iframe element and switching into it is usually better and safer.