I am trying to do something i can't find any help on. I want to be able to locate the xpath or other 'address' information in of a particular element for later use by selenium. I have text for the element and can find it using the selenium By.LINK.TEXT methodology. However, i am writing an application where speed is critical so i want to pre-find the element, store the xpath (for later use) and then use the By.XPATH methodology. In general finding an element using the BY.text construction takes .5 seconds whereas the xpath lookup takes on 10 - 20% of that time. I tried the code below but i get an error on getpath (WebElement object has no attribute getpath)
Thanks for any help
temp = br.find_element(By.LINK_TEXT, (str(day_to_book)))
print(temp.getpath())
The Selenium WebElement object received by driver.find_element(ByLocator) is already a reference to the actual physical web element on the page. In other words, the WebElement object is an address of the actual web element you asking about.
There is no way to get a By locator of an already found WebElement
So, in your particular example temp = br.find_element(By.LINK_TEXT, (str(day_to_book))) the temp is an address of the element you can keep for future use (until the page is changed / refreshed)
Related
This is the site I want to scrape.
I want to scrape all the information in the table on the first page:
then click on the second and do the same:
And so on until the 51st page. I know how to use selenium to click on page two:
link = "http://www.nigeriatradehub.gov.ng/Organizations"
driver = webdriver.Firefox()
driver.get(link)
xpath = '/html/body/form/div[3]/div[4]/div[1]/div/div/div[1]/div/div/div/div/div/div[2]/div[2]/span/a[1]'
find_element_by_xpath(xpath).click()
But I don't know how to set the code up so that it cycles through each page. The process of me getting the xpath is a manual one in the first place (I go on to Firefox, inspect the item and copy it into the code), so I don't know how to automate that step in and of itself and then the following ones.
I tried going a level higher in the webpage html and choosing the entire section of the page with the elements I want, and cycling through them, but that doesn't work because it's a Firefox web object(see below). Here'a a snapshot of the relevant part of the page source:
By calling the xpath of the higher class like so:
path = '//*[#id="dnn_ctr454_View_OrganizationsListViewDataPager"]'
driver.find_element_by_xpath(path)
and trying to see if I can cycle though it:
for i in driver.find_element_by_xpath(path):
i.click()
I get the following error:
Any advice would be greatly appreciated.
This error message...
...implies that you are trying to iterate through a WebElement where as only list objects are iterable.
Solution
Within the for() loop to create a list to iterate through it's elements, instead of using find_element* you need to use find_elements*. So your effective code block will be:
for i in driver.find_elements_by_xpath(path):
i.click()
driver.get("https://www.zacks.com/")
driver.find_element_by_xpath("//*[#id='search-q']")
i am trying to find search box on zacks website with selenium but I am getting StaleElementReferenceException
The reason why you're getting this error is simply, the element has been removed from the DOM. There are several reasons for this:
The page itself is destroying/recreating the element on the fly, maybe even rapidly.
Parts of the page have been updated (replaced), but you're still having and old reference.
You navigate to a new page but holding an old reference.
To avoid this, try to keep the element reference as short as possible. If the content is rapidly changing, make the operation directly without the round trip to the client, via javascript:
driver.executeScript("document.getElementById('serach-q').click();");
Maybe you're trying to find while the page and this exact search box are loading. Try to implement wait mechanism for this element, smth like that:
WebDriverWait wait = new WebDriverWait(driver, timeoutInSeconds);
wait.until(ExpectedConditions.visibilityOfElementLocated(locator));
I am trying to write a Selenium test but the issue is I have learned that the page is generated with PrimeFaces, thus the element IDs randomly change from time to time. Not using IDs is not very reliable. Is there anything I can do?
Not having meaningful stable IDs is not a problem, as there are always alternative ways to locate elements on a page. Just to name a few options:
partial id matches with XPath or CSS, e.g.:
# contains
driver.find_element_by_css_selector("span[id*=customer]")
driver.find_element_by_xpath("//span[contains(#id, 'customer')]")
# starts with
driver.find_element_by_css_selector("span[id^=customer]")
driver.find_element_by_xpath("//span[starts-with(#id, 'customer')]")
# ends with
driver.find_element_by_css_selector("span[id$=customer]")
classes which refer/tell some valuable information about the data types ("data-oriented locators"):
driver.find_element_by_css_selector(".price")
driver.find_element_by_class_name("price")
going sideways from a label:
# <label>Price</label><span id="65123safg12">10.00</span>
driver.find_element_by_xpath("//label[.='Price']/following-sibling::span")
links by link text or partial link text:
driver.find_element_by_link_text("Information")
driver.find_element_by_partial_link_text("more")
And, you can, of course, get creative and combine them. There are more:
Locating Elements
There is also this relevant thread which goes over best practices when choosing a method to locate an element on a page:
What makes a good selenium locator?
For each vendor in an ERP system (total # of vendors = 800+), I am collecting its data and exporting this information as a pdf file. I used Selenium with Python, created a class called Scraper, and defined multiple functions to automate this task. The function, gather_vendors, is responsible for scraping and does this by extracting text values from tag elements.
Every vendor has a section called EFT Manager. EFT Manager has 9 rows I am extracting from:
For #2 and #3, both have string values (crossed out confidential info). But, #3 returns null. I don’t understand why #3 onward returns null when there are text values to be extracted.
The format of code for each element is the same.
I tried switching frames but that did not work. I tried to scrape from edit mode and that didn’t work as well. I was curious if anyone ever encountered a similar situation. It seems as though no matter what I do I can’t scrape certain values… I’d appreciate any advice or insight into how I should proceed.
Thank you.
Why not try to use
find_element_by_class_name("panelList").find_elements_by_tag_name('li')
To collect all of the li elements. And using li.text to retrieve their text values. Its hard to tell what your actual output is besides you saying "returns null"
Try to use visibility_of_element_located instead of presence_of_element_located
Try to get textContent with javascript fo element Given a (python) selenium WebElement can I get the innerText?
element = driver.find_element_by_id('txtTemp_creditor_agent_bic')
text= driver.execute_script("return attributes[0].textContent", element)
The following is what worked for me:
Get rid of the try/except blocks.
Find elements via ID's (not xpath).
That allowed me to extract text from elements I couldn't extract from before.
You should change the way of extracting the elements on web page to ID's, since all the the aspects have different id provided. If you want to use xpaths, then you should try the JavaScript function to find them.
E.g.
//span[text()='Bank Name']
I am trying to make a basic form filler using Selenium on nike.com. I have completed most of it but I am having trouble clicking on the button to select the gender. I have attempted to use many examples of find_element_by_xxxxx code bit none of it has worked. Finding elements by id and xpath haven't come to much either. A typical error i get is Message: no such element: Unable to locate element. I am very new to coding so I could very easily have made an error, but any idea on how you guys would solve it would be much appreciated
That XPATH is very long and you can simplify.
By the looks of it , I would guess those Ids are changing every time there is a new session.
A more straightforward XPATH selector could be...
"//span[text() = 'Male']"
// specifies to search the entire document
span specifies the type of element to search for
text() specifies text that needs to be inside the element
(this will give you the span element but it should still work)
or
"//span[text() = 'Male']/parent::li//input"
(this will give you the actual input button)
Also , like Ollin Boer Bohan suggested, look into using waits before performing actions on your elements.
#cavan answer is correct, Also you can use xpaht like this too
//input[#type='button']//following::span[text()='Male']
Here we can use following to locate the male, same you can do for female button